click the screen · press Enter
← back to blog
Kubernetes Journey · Topic 9

Kubernetes Authorization Isn't One System, It's a Pipeline

Kubernetes authorization and RBAC explained - Kubernetes Journey

Topic 8 ended on a cliffhanger. Once the cluster knows who you are, the very next thing it decides is what you're allowed to do. That second decision is authorization, and it's the bit that stops a junior dev from casually deleting a production node. This is the post I've been looking forward to writing, because RBAC is where all those groups, resources and verbs from last time finally do something.

Same deal as always: for each idea I'll give you what it is, how it works, why it matters, and one line to take away. Nothing fancy.

Two doors: who you are, then what you can do

Quick recap so this stands on its own. When you run a command, the API server checks two separate things, in order.

Authentication is the first door. It asks "who are you?" and looks at your certificate or token to confirm it. That was Topic 6 and Topic 8.

Authorization is the second door, and it's today. It asks "okay, you're allowed in, but are you allowed to do this?" You might be a real, authenticated user and still get told no.

Here's what "no" looks like. A cluster admin can do pretty much anything:

kubectl get pods
kubectl get nodes
kubectl delete node worker-2
# node "worker-2" deleted

Hand that same last command to a developer account and the door slams:

kubectl delete node worker-2
# Error from server (Forbidden): nodes "worker-2" is forbidden:
# User "developer" cannot delete resource "nodes"

That "Forbidden" is authorization working. The user is genuine, they're just not allowed to touch nodes. Authentication proves who you are; authorization decides what you can touch. Different doors, different failures.

The menu of authorization modes

Kubernetes doesn't have one way to make that decision. It has a few, and you pick which ones run. Here's the whole menu before I go through each.

Five ways the API server can decide
WHO'S ALLOWED TO DO WHAT?
Node · trusts the kubelets on your nodes
ABAC · a JSON file of who-can-do-what
RBAC · roles you bind to users (the default)
Webhook · ask an external brain like OPA
AlwaysAllow / AlwaysDeny · blanket yes or no
Most real clusters run Node plus RBAC. The rest are situational.

The two blunt ones first, because they're easy. AlwaysAllow waves everyone through with no checks at all. Any authenticated user can do anything. It's handy for a throwaway lab and a disaster anywhere near production. AlwaysDeny is the opposite, it refuses everything. You set the mode as a flag on the API server:

kube-apiserver \
  --authorization-mode=AlwaysAllow \
  ...

If you ever see AlwaysAllow on a real cluster, that's a finding on its own. It means the second door has been propped open.

You can stack modes, and order matters

You're not limited to one mode. You pass a comma-separated list, and this is the part people trip on: the API server walks the list left to right and the first mode that gives a clear yes or no wins. As soon as one decides, the rest never run.

--authorization-mode=Node,RBAC,Webhook
01
Node

Is this a kubelet doing kubelet things? Decide. If it can't tell, pass it on.

02
RBAC

Does a role allow this user? Allow or deny. If no rule matches, pass it on.

03
Webhook

Last word from an external policy engine. Allow or deny.

First clear decision stops the chain. A mode that "isn't sure" just hands the request to the next one.

On a normal kubeadm cluster this list is short and sensible:

kube-apiserver \
  --authorization-mode=Node,RBAC \
  ...

The mode list is an order of priority, not a set of equals. Read it left to right and the behaviour stops being mysterious.

Node authorization: trusting your own kubelets

Node mode exists for one specific group of callers: the kubelets. A kubelet is the agent running on every worker node, and it constantly talks to the API server to do its job. It reads pod and service details so it knows what to run, and it reports the node's health back up.

Those requests need permission too, but you don't want to write RBAC rules for every node by hand. So Node authorization handles them with a simple identity check. The API server trusts a request as a kubelet only if both of these are true:

  • the username starts with system:node: (for example system:node:worker-1), and
  • the user is in the system:nodes group.

Get both right and the kubelet is allowed to do its normal node-level work, nothing more. Node mode is a purpose-built lane for the cluster's own agents, so RBAC doesn't have to babysit them.

ABAC: the JSON file that RBAC replaced

ABAC stands for attribute-based access control. The idea: you grant permissions by matching attributes of a request, things like the user, their group, the namespace and the resource. You write those matches as lines in a JSON policy file. One rule per line, roughly "this user can touch this resource":

{"kind":"Policy","spec":{"user":"dev-user","namespace":"*","resource":"pods","apiGroup":"*"}}
{"kind":"Policy","spec":{"group":"dev-users","namespace":"*","resource":"pods","apiGroup":"*"}}
{"kind":"Policy","spec":{"user":"security-1","namespace":"*","resource":"csr","apiGroup":"*"}}

The flow is dead simple. A request comes in, the API server scans the policy file, and if a line matches, it's allowed. No match, denied.

Sounds fine until you actually run it. Every time you add a user or change a permission, you edit that file by hand and restart the API server for it to take effect. Multiply that by a growing team and it turns into a brittle mess nobody wants to touch. That pain is exactly why RBAC took over.

Why you'll rarely meet ABAC It works, but managing a flat file of rules and bouncing the API server on every change doesn't scale. RBAC keeps everything as normal cluster objects you can create and edit live. That's the whole reason it won.

Webhook: outsourcing the decision

Sometimes you want your own logic making the call, not Kubernetes. Webhook mode lets you do that. When a request arrives, the API server packages up the details and sends them to an external service. A common choice is OPA, the Open Policy Agent, which is basically a policy engine you feed rules to.

OPA looks at the request against its own policies and answers with one word: allow and the API server lets it through, or deny and it's rejected. It's the "ask a specialist" option, useful when your access rules are more complicated than roles alone can express. Handy to know it exists, though most people never need it day to day.

RBAC: the one you'll actually use

Right, the main event. RBAC is role-based access control and it's the default on basically every real cluster. The trick that makes it nice: you don't hand permissions to people directly. You put permissions in a role, then bind that role to a user or group. Change the role once and everyone attached to it updates.

Two objects do all the work:

  • A Role is a bundle of permissions. Each rule says: on this API group, for these resources, you may run these verbs (get, list, create, delete, and so on).
  • A RoleBinding is the link. It says "give this role to this user". Without the binding, a role does nothing at all.

One thing to keep straight from the start: both of these are namespaced. A Role you create in the default namespace only grants access inside default. Same for the binding. So when you write one, you're always working inside a namespace whether you say so or not.

Building a role and binding it

Say I want a dev-user who can list, create and delete pods in default, and nothing else. Two ways to do it. The fast way is straight from the command line:

# the role: what's allowed
kubectl create role developer \
  --namespace=default \
  --verb=list,create,delete \
  --resource=pods
# role.rbac.authorization.k8s.io/developer created

# the binding: who gets it
kubectl create rolebinding dev-user-binding \
  --namespace=default \
  --role=developer \
  --user=dev-user
# rolebinding.rbac.authorization.k8s.io/dev-user-binding created

The clearer way, and the one I prefer for anything I want to keep, is YAML. It reads almost like a sentence once you know the shape:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: developer
rules:
- apiGroups: [""]          # "" means the core group
  resources: ["pods"]
  verbs: ["list", "create", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: dev-user-binding
  namespace: default
subjects:
- kind: User
  name: dev-user
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: developer
  apiGroup: rbac.authorization.k8s.io

Look at the pieces. The Role's rules are the group + resources + verbs combo I banged on about last post. The RoleBinding's subjects is who gets the access, and roleRef points at the role by name. Apply it with kubectl create -f file.yaml and dev-user can now do exactly those three things to pods, in that one namespace.

The empty string trips everyone up apiGroups: [""] isn't a mistake. The empty string is the name of the core API group, the one that holds pods, services, configmaps and the like. Named groups such as apps go in the quotes instead.

Reading what a cluster already allows

Before you change anything, look at what's there. First, confirm which modes the cluster even runs. The API server runs as a pod on the control plane, so describe it and read its flags:

kubectl describe pod kube-apiserver-controlplane -n kube-system
# look for --authorization-mode

Buried in the command list you'll find the line that matters:

--authorization-mode=Node,RBAC
--enable-admission-plugins=NodeRestriction

Now list the roles. Roles are per-namespace, so pick one, or ask for all of them at once:

kubectl get roles                 # just this namespace
kubectl get roles -A              # every namespace

To see what a specific role actually grants, describe it. Here's the real kube-proxy role from kube-system:

root@controlplane ~
$ kubectl describe role kube-proxy -n kube-system Name: kube-proxy PolicyRule: Resources Non-Resource URLs Resource Names Verbs --------- ----------------- -------------- ----- configmaps [] [kube-proxy] [get] == one resource, one verb: read the kube-proxy configmap ==
A tight role: it can only "get" one named configmap. That's least privilege done right.

And to see who a role is handed to, describe its binding:

kubectl describe rolebinding kube-proxy -n kube-system
# Role:     kube-proxy
# Subjects:
#   Kind   Name
#   Group  system:bootstrappers:kubeadm:default-node-token

The Role tells you the what. The RoleBinding tells you the who. You need both in front of you to understand a single permission.

can-i: the fastest question in the cluster

This is my favourite command in the whole authz story, and it's the one I reach for on an engagement. kubectl auth can-i answers a yes/no permission question instantly, without you having to trigger the action for real.

kubectl auth can-i create deployments
# yes

kubectl auth can-i delete nodes
# no

The genuinely useful bit is --as. It lets you ask the question as someone else without logging in as them, so an admin can test exactly what a given user can and can't do:

check what dev-user can really do
$ kubectl auth can-i create deployments --as=dev-user no $ kubectl auth can-i create pods --as=dev-user yes $ kubectl auth can-i --list --as=dev-user == dumps every permission that user has ==
--as impersonates a user for the check. --list dumps their whole effective permission set. Add --namespace to scope it.

From an attacker's chair this is gold. Land on a box with some identity and can-i --list tells you your entire reach in one shot, so you know instantly whether you can read secrets, create pods, or mint certificates before you touch anything noisy.

Defender takeaway Run kubectl auth can-i --list --as=<user> against your own service accounts. If a "read-only" account can create pods or read secrets cluster-wide, that's the gap an attacker will walk straight through. Impersonation itself (the --as verb) should also be locked down.
Try itRBAC simulator: what can this Role actually do?
Runs entirely in your browser. Tick a wildcard and watch how much it quietly opens up.

Narrowing a role to named resources

By default a role covers all resources of a type. Grant get on pods and you've granted it on every pod in the namespace. Sometimes that's too much. RBAC lets you pin a rule to specific objects by name with resourceNames:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: developer
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "create", "update"]
  resourceNames: ["blue", "orange"]

Now the user can act on the pods called blue and orange, and no others. It's a neat way to hand someone exactly one app without giving them the run of the namespace. One catch worth knowing: resourceNames doesn't play with list or watch, because those are collection-wide by nature, so it fits get, update and delete style verbs.

The bit that finally clicked

The thing that finally clicked: authorization isn't one system, it's a little pipeline of them, and RBAC is just the friendliest station on the line. Roles hold the permissions, bindings hand them out, and can-i lets you interrogate the whole thing without breaking anything. I'm still light on the Webhook and OPA side, I've only read about it, not built a policy engine and pointed a cluster at it, so if you've run OPA in anger I'd love to hear how it went. Next up I get into ClusterRoles and the cluster-wide version of all this, plus service account permissions, which is where a lot of real-world escalation actually lives.

If this made RBAC click for you, come say hi on LinkedIn or the contact page, and tell me what you want broken down next.

Further reading

FAQ

What is RBAC in Kubernetes?

Role-Based Access Control decides what an authenticated user or service account is allowed to do. You bind roles (sets of permissions) to subjects, so identity plus binding equals access. It is the default authorizer on most clusters.

How is authentication different from authorization?

Authentication proves who you are; authorization decides what you may do. Kubernetes runs them as two separate gates: the API server first checks identity, then checks RBAC to allow or deny the action.

How do I test what permissions a user has?

Use kubectl auth can-i, with --as to impersonate a user and --list to dump their whole effective permission set. It is the quickest way to spot who can do more than they should.