
The belief I want to poke at in this one is the quiet assumption that a manifest which applies cleanly is a manifest that's fine. kubectl apply returns created, the pod goes Running, and it feels like a pass. It isn't. Kubernetes will happily run a Deployment with no resource limits, a root container and a single point of failure, because "will this schedule" and "is this a good idea" are completely different questions and the API server only answers the first one.
KubeLinter answers the second one, before anything reaches the cluster. It's a static analysis tool: it reads your YAML on disk and checks it against a set of rules, the same way a code linter reads your source. This is a lab log of me running it over a stock nginx Deployment from my course, watching it find five things, fixing them, and then hitting a genuinely annoying surprise where the "fixed" manifest passed the linter and refused to start.
What I set out to do, and the kit
The goal was small and concrete: take one ordinary Deployment, the sort you'd copy off a getting-started page, and see what a linter thinks of it. The manifest was a three-replica nginx Deployment pinned to nginx:1.14.2, with a container port and nothing else. No resource requests, no security context, no probes. It's the kind of thing that works on your machine and worries nobody until it's 2am and one node is on fire.
KubeLinter is a single Go binary from StackRox, so setup is a download and a move onto your path. This was the release the course pinned, so treat the version as theirs, not a recommendation to pin to an old one:
curl -LO https://github.com/stackrox/kube-linter/releases/latest/download/kube-linter-linux.tar.gz
tar -xvf kube-linter-linux.tar.gz
sudo mv kube-linter /usr/local/bin/
One command to run it. Point it at a file or a directory and it lints everything it finds:
kube-linter lint nginx.yml
# or keep the report
kube-linter lint nginx.yml > analyze
Five findings on a Deployment that applied cleanly
Here's the run, trimmed to the finding lines. The full remediation text is long, so I've kept the check name and the one line that matters:
Read them as a group and a shape appears. Two are about not falling over (spread the replicas, declare what you need). Three are about not being an easy target (don't run as root, don't let the process write its own filesystem, set limits so a runaway can't eat the node). None of them stopped the thing deploying. That's the whole argument in one screen: five real problems, zero of them fatal to apply.
One thing it did not flag is worth a note, because it surprised me. This manifest used nginx:1.14.2, a pinned tag, so the latest-tag check stayed quiet. Swap that to nginx:latest and you'd earn a sixth finding. The linter only barks at what's actually there, which sounds obvious until you rely on it to catch a class of problem you didn't leave an example of.
Fixing them, one field at a time
The nice part of KubeLinter is that every finding names the field to change, so the fixes are mechanical. First, resources. A request is what the scheduler reserves for you; a limit is the ceiling the kubelet enforces. Setting both makes the pod a known quantity and pulls it out of the BestEffort eviction bucket:
resources:
requests:
cpu: "250m"
memory: "64Mi"
limits:
cpu: "500m"
memory: "128Mi"
Next, the security context, which is where the run-as-non-root and no-read-only-root-fs findings get answered. This block tells the kubelet to run the process as a specific non-root user and to mount the container's own filesystem read-only:
securityContext:
runAsNonRoot: true
runAsUser: 1000
readOnlyRootFilesystem: true
Last, anti-affinity, the fix for no-anti-affinity. Three replicas are worthless for availability if the scheduler stacks all three on one node and that node dies. This rule says "don't put two pods with this label on the same host":
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values: ["nginx"]
topologyKey: "kubernetes.io/hostname"
Re-lint after all three and it's clean. Zero errors, exit code zero, the pipeline would go green. If the story stopped here it'd be a tidy little post about a tidy little tool, and it'd also be lying to you.
required vs preferred anti-affinity The required form above is a hard rule: if there aren't enough nodes, replicas that can't be placed stay Pending forever. On a small cluster that's a self-inflicted outage. preferredDuringScheduling is usually the saner choice, and KubeLinter is happy with either.Where it bit me: the linter passed, the pod didn't
Here's the thing nobody mentions. That security context KubeLinter asked for will stop the stock nginx image from starting.
Two reasons, both real. The official nginx image runs its master process as root so it can bind port 80, because binding anything below 1024 needs privilege a user with UID 1000 simply doesn't have. And nginx writes to /var/cache/nginx and /var/run at startup, which readOnlyRootFilesystem: true forbids. So the manifest that satisfies every rule produces a pod that crash-loops with a permission-denied on port 80 or a read-only-filesystem write error, depending on which bites first.
The real fix isn't to back the security context out, it's to give the app what it needs to live inside those constraints. Use an image built to run unprivileged, like nginxinc/nginx-unprivileged, which listens on 8080 and expects a non-root user. Then mount writable scratch space for the paths nginx has to write, so the root filesystem stays read-only but the cache directory isn't:
spec:
containers:
- name: nginx
image: nginxinc/nginx-unprivileged:1.27
ports:
- containerPort: 8080
securityContext:
runAsNonRoot: true
runAsUser: 101
readOnlyRootFilesystem: true
volumeMounts:
- { name: cache, mountPath: /var/cache/nginx }
- { name: run, mountPath: /var/run }
volumes:
- { name: cache, emptyDir: {} }
- { name: run, emptyDir: {} }
That's the lesson the clean lint hides. KubeLinter checks the shape of the YAML, not whether the workload runs. It was right that the container shouldn't be root. It has no idea that this particular container needs a different base image to honour that. The green tick means "your manifest matches these rules", never "this will work".
Wiring it into CI, where it actually earns its keep
Running a linter by hand once is a party trick. The value is having it block a bad manifest in a pull request, before a human review even starts. KubeLinter exits non-zero on findings, so any CI system turns that into a failed stage for free. In GitHub Actions it's three lines of real work:
name: Lint Kubernetes manifests
on: [push]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run KubeLinter
uses: stackrox/kube-linter-action@v1
with:
directory: manifests
Put this stage before the build and deploy stages. A manifest problem caught in the pull request costs a comment; the same problem caught after deploy costs an incident. Same check, wildly different price depending on where in the pipeline it fires.
.kube-linter.yaml. You can switch off rules that don't fit (single-replica is fine for a batch job) and, better, turn on ones that aren't in the default set. Treat the defaults as a starting point you tighten, the same way you'd build a Gatekeeper policy, not a finished standard.What this means from the other side
Reading manifests is one of the first things I do on any cluster review, and a linter is a shortcut to the good stuff. Run KubeLinter across a repo of manifests and the findings are a map of where to push: a container with no security context is a container that might be root with a writable filesystem, which is a much shorter walk to a node than a locked-down one. no-read-only-root-fs means an attacker who lands in the pod can drop a binary and stay. run-as-non-root unset is the difference between a container escape being hard and being a Tuesday.
The catch, and it's the same catch from the defender's side, is that a clean report is not a clean cluster. The linter can't see the image's contents, the RBAC around the pod, or whether that read-only filesystem is undermined by a writable hostPath mount two lines down that no default rule covers. It narrows where I look. It doesn't tell me I'm safe, and I wouldn't trust anyone who told you it did.
The one thing I'd tell someone starting with this
Run it, but run it with your eyes open about what it is. KubeLinter is a floor, not a ceiling. It catches the boring, common, entirely preventable mistakes that make up most of what I find on a real review, and it catches them in the pull request where they're cheap. That alone makes it worth the ten minutes to wire in.
What it won't do is tell you the app still works after you take its advice, and it won't tell you the manifest is secure just because it's quiet. Those two gaps are the same gap, really: a tool that reads files can only ever know about files. The bit I'm still chewing on is how hard to fail the build. Block every finding and developers route around you; warn on everything and nobody fixes anything. I don't have a clean answer yet, and if you've found the line that keeps a team both shipping and honest, I'd genuinely like to hear where you put it.
Next in the series I'm picking up the thread this post left dangling: where those images come from, private registries, and making the cluster refuse anything that isn't from one you trust.
References
- KubeLinter documentation
- StackRox KubeLinter on GitHub
- Kubernetes docs: configure a security context for a pod
- Kubernetes docs: inter-pod affinity and anti-affinity
- nginxinc/nginx-unprivileged image
FAQ
What is KubeLinter?
KubeLinter is an open source static analysis tool from StackRox that reads Kubernetes YAML and Helm charts and flags configuration and security problems before they are applied. It checks things like missing resource limits, containers running as root, no anti-affinity and use of the latest tag, and prints a remediation for each finding.
Does KubeLinter check a running cluster?
No, and that is the point. KubeLinter reads manifest files on disk, so it runs in a pull request or a pipeline before anything reaches the cluster. It never talks to the API server. That makes it fast and safe to run anywhere, but it also means it only sees the YAML, not whether the app actually works.
Why does nginx crash when you set runAsNonRoot and readOnlyRootFilesystem?
The stock nginx image runs its master as root and binds port 80, which a non-root user cannot do. It also writes to /var/cache/nginx and /var/run, which a read-only root filesystem blocks. The manifest passes KubeLinter but the container fails to start. Use an unprivileged nginx image and mount writable emptyDir volumes.
Does a manifest passing KubeLinter mean it is secure?
No. KubeLinter checks the shape of the YAML against a set of rules, not the behaviour of the workload. A manifest can pass every default check and still run a vulnerable image, mount a dangerous host path a rule does not cover, or fail to start. Treat a clean lint as a floor, not a certificate.
How do I add KubeLinter to CI?
Install the binary and run kube-linter lint against your manifest directory as a pipeline step. It exits non-zero when it finds problems, so the stage fails and blocks the merge. Run it early, before the build and deploy stages, so a bad manifest is caught in the pull request rather than in production.
Related reading
- Topic 37: Your Scanner Doesn't Delete Anything (the image side of the same trust question)
- Topic 30: Policy as Code with OPA Gatekeeper (enforcing at admission what a linter only warns about)
- Topic 29: Pod Security Policies Are Gone (the built-in successor for securityContext rules)
- Topic 36: A Quota Is a Ceiling, Not a Referee (why unset resources land you in BestEffort)
- My Kubernetes pentest notes (the manifest-review checklist without the story)
- Browse the whole Kubernetes Journey