click the screen · press Enter
← back to blog
Kubernetes Journey · Topic 2

How Kubernetes Works: the Mental Model That Made It Click

Topic 2: How Kubernetes Works - Kubernetes Journey

You can't secure something you can't picture. So before I get anywhere near security concerns, I want to be able to draw how Kubernetes actually works from a developer's point of view. Security issues and their remediation only make real sense when you know what is happening behind the scenes and how developers interact with the platform day to day.

This post focuses on building a clear mental model of Kubernetes rather than jumping straight into commands or security controls.

The problem Kubernetes solves

Modern applications are no longer single servers running a single process. Developers build applications as multiple services, each running in containers. These containers need to be deployed, scaled, restarted when they fail, and connected to each other. Doing this manually becomes painful very quickly.

Kubernetes exists to solve this operational problem - running containers reliably, at scale, across many machines, while giving developers a consistent way to deploy and manage applications.

Kubernetes at a high level

In simple terms, Kubernetes runs and manages your containers for you. Instead of manually starting, stopping or fixing applications, you tell Kubernetes how your application should run - how many copies you need and which image to use. It then keeps checking the system and automatically fixes things if something goes wrong.

Two ideas are key:

  • Declarative configuration - you describe what you want, not the step-by-step process to get there.
  • Continuous reconciliation - Kubernetes constantly watches the cluster and fixes drift between the current state and the desired state.

The cluster concept

A Kubernetes setup is called a cluster, made of two main parts: the control plane and the worker nodes. As a developer you mostly interact with the control plane, even though your application actually runs on the worker nodes.

Control plane - the brain

The control plane makes all the decisions. It doesn't run your application containers directly, but decides where and how they run.

API server

The front door of Kubernetes. Every request goes through it - kubectl, a CI pipeline, or another component. From a security perspective this is critical, because authentication, authorisation and admission control all happen here.

Scheduler

Decides which worker node should run a new Pod, based on available resources, constraints and policies.

Controller manager

Controllers continuously monitor the cluster and fix drift. Declare three replicas and one Pod crashes? A controller creates a new one automatically.

etcd

The key-value store that holds the entire state of the cluster - workloads, configuration and secrets. One of the most sensitive components from a security testing point of view.

Worker nodes - where apps run

Worker nodes are the machines that actually run your containers. Each node includes:

  • Kubelet - talks to the control plane and ensures the containers in Pod specs are running correctly.
  • Container runtime - the software that runs containers, such as containerd or CRI-O.
  • Kube-proxy - manages networking rules so Pods and Services can communicate across the cluster.

Pods - the smallest unit

A Pod is the most basic thing you can create in Kubernetes, and the unit it actually runs. You don't run containers by themselves - Kubernetes always runs them inside Pods. Most of the time a Pod has one container; sometimes more, when they need to share the same network and storage closely.

You don't deploy containers directly in Kubernetes. You deploy Pods - or resources like Deployments that create and manage Pods for you.

Deployments and desired state

A Deployment is the most common way developers run applications. You define the container image, the number of replicas and the update strategy, and Kubernetes ensures the desired number of Pods is always running. This abstraction is useful, but it can hide what's really happening - so from a security point of view it's important to know exactly what Kubernetes is creating and managing for you.

Services and networking

Pods are temporary - created and destroyed at any time - so they can't be relied on directly for networking. A Service provides a stable endpoint and routes traffic to the correct Pods. Misconfigured Services, ingress resources and missing network policies are common real-world security issues.

Why this matters for security

When testing Kubernetes security, you're not just attacking containers. You're testing the control plane, configuration objects, and the trust boundaries between components.

Closing thoughts

Kubernetes is powerful because it abstracts away servers and infrastructure - but that can also make security issues harder to notice if you don't understand what it's doing behind the scenes. In upcoming posts I'll connect real-world Kubernetes security issues to these core components, with practical remediation steps using examples from the Kubernetes Goat project.

FAQ

What are the main parts of a Kubernetes cluster?

Two halves. The control plane is the brain (API server, scheduler, controller manager, etcd) and the worker nodes are where your pods run, via the kubelet and a container runtime. The control plane decides, the workers do.

What does the Kubernetes API server do?

It is the front door to the cluster. Every command, from kubectl or a controller, goes through the API server, which authenticates it, validates it and writes the desired state to etcd. Nothing else talks to etcd directly.

What is etcd used for?

etcd is the cluster database. It stores the entire desired and observed state as key-value data. If etcd is lost, the cluster loses its memory, which is why backing it up and securing it matters so much.