Containers Are Isolated from Each Other, Not from the Kernel

Start a container that does nothing but sleep. Then go and look for it on the host.

It's right there in ps, as an ordinary process, with an ordinary PID. You can kill it from the host and the container dies. That's not a leak or a misconfiguration, that's just what a container is, and once you've seen it, the reason gVisor and Kata Containers exist stops needing an explanation.

A container is a process wearing a costume

A container is a normal Linux process, started by the host kernel, with a few kernel features switched on around it so it can't see much.

Two ideas do nearly all the work. Namespaces change what a process can see: its own process list, its own network interfaces, its own view of the filesystem, its own hostname. cgroups change how much it can take: CPU, memory, I/O. Put both around a process and it feels like it's alone on a machine.

Feeling alone and being alone are different things. Every container on a node is talking to the same kernel. One kernel, one set of bugs, shared by everything running there.

So containers are isolated from each other. They are not isolated from the kernel, because there is only one of those and they're all using it.

Seeing it: the same process, two PIDs

This takes about a minute on any box with Docker on it. Start something that just sits there:

docker run -d --name napper busybox sleep 3000

That's a PID namespace: a private numbering scheme so the container's main process gets to be PID 1 and can't see anything outside. Useful, real, and entirely one-directional. The container can't look out. The host can look straight in, and act.

What a virtual machine does differently

A VM boots its own kernel. There's a hypervisor underneath managing the hardware, and the guest kernel inside is a separate copy with a separate memory space. Break the guest kernel and you're inside the guest, which is not the same as being on the host.

So the interesting question isn't "which is better". It's whether you can get some of row one back without giving up all of row four. That's exactly what sandboxing runtimes are trying to sell you.

Everything we've stacked so far, and its ceiling

Worth pausing here, because this series has been building one answer to this problem for a dozen topics without ever naming the limit.

Each of those narrows what the container is allowed to ask for. Seccomp cuts down which system calls it can make. AppArmor and capabilities cut down which files those calls may touch and which privileged operations are on the table. Least privilege is the thread through all of it.

And on the choice between the two styles of rule, which comes up every time you write one of these profiles:

Here's the ceiling, though. Every control on that list makes the door narrower. Not one of them changes what's behind the door. The syscall that does get through goes to the same host kernel that every other pod on that node is using, and if that kernel has a bug in the handler for a call your profile allows, you've hardened your way right up to the edge of the actual problem.

gVisor: answer the syscalls yourself

gVisor is a sandboxing runtime from Google, run as runsc, that puts a second kernel between the container and the real one. That second kernel is a normal user-space program.

When the container makes a system call, it doesn't reach the host kernel. It's caught and handed to a process called the Sentry, which is gVisor's own implementation of the Linux system call interface, written in Go. The Sentry answers most calls itself. When it genuinely needs the host, it makes a much smaller, more predictable request of its own. Two other pieces round it out: the Gofer, a separate process that does file access on the container's behalf so the container never touches the host filesystem directly, and Netstack, a user-space network stack so packets don't go straight into the host's networking code either.

Linux has hundreds of system calls and an enormous amount of code behind them. That whole surface is what a container gets to poke at under a normal runtime. Under gVisor, most of it is unreachable, because the thing answering is a smaller program written for this one job.

Which is the bit worth holding onto: gVisor doesn't restrict which syscalls you can make. It changes who answers them.

Getting it running is genuinely easy, which surprised me:

# install runsc (see the gVisor docs for the current install snippet)
sudo runsc install       # registers runsc as a Docker runtime
sudo systemctl restart docker

docker run --rm --runtime=runsc alpine dmesg | head -5
docker run --rm --runtime=runsc alpine uname -r

That uname -r is the tell I like best. The host is running something modern, the container thinks it's on 4.4.0, and neither is lying exactly. The Sentry just presents a fixed version because it isn't the host kernel and never was.

One more thing worth knowing since most older write-ups get it wrong. gVisor has several "platforms", which is how it traps the syscalls in the first place. It used to default to ptrace, which worked everywhere and was slow enough to be a real objection. Since 2023 the default has been systrap, which is much faster and still needs no virtualisation, and ptrace is deprecated. There's also a kvm platform that's quicker on bare metal. If you read a blog post from 2021 saying gVisor is unusably slow, it was probably measuring the platform nobody uses any more.

Kata Containers: give each pod its own kernel

Kata Containers is a runtime that boots a small, fast virtual machine and runs the workload inside it. Not emulating a kernel. An actual Linux kernel, its own copy, booted per sandbox.

The runtime starts a lightweight VM using a hypervisor (QEMU by default, with Cloud Hypervisor and Firecracker as alternatives), boots a stripped-down kernel and a tiny agent inside it, and runs your containers there. Files come in through a virtio filesystem daemon rather than a bind mount. From the workload's point of view nothing is unusual, it's Linux, it just isn't the host's Linux.

The boundary stops being a software policy and starts being the same hardware virtualisation boundary a cloud provider uses between two customers' VMs. A kernel exploit inside gets you a kernel you're already allowed to be in.

gVisor shrinks the host kernel surface. Kata removes the sharing entirely.

The catch is hardware. Kata needs real virtualisation support on the node, which means /dev/kvm has to be there and usable:

ls -l /dev/kvm
grep -c -E 'vmx|svm' /proc/cpuinfo     # non-zero means the CPU can do it
kata-ctl check                          # the runtime's own preflight

On a bare metal node that's fine. In the cloud it often isn't, because your node is already a VM, so running Kata inside it needs nested virtualisation, a VM inside a VM. Some providers support it on some instance types, plenty don't, and where it does work it's slower than the real thing. Google Cloud documents how to turn it on, for instance, but it isn't a default anywhere and it's the first thing to check before planning around Kata. Bare metal node pools are where Kata is genuinely comfortable.

The Kubernetes bit: choosing per pod with RuntimeClass

This is the part most explanations of gVisor and Kata leave out, and it's the part that makes them usable. You don't pick one runtime for the cluster. You pick it per pod.

1. Tell containerd the runtime exists. On the node, in /etc/containerd/config.toml:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata]
  runtime_type = "io.containerd.kata.v2"

Then systemctl restart containerd. Check your containerd version before copying that: the plugin key changed in containerd 2.x, so if the runtime never gets picked up, the config schema is the first thing I'd look at. For Kata there's also a kata-deploy DaemonSet that does the node setup for you, which is what I'd use rather than editing config by hand on every node.

2. Create the RuntimeClass. A tiny cluster-scoped object that maps a friendly name to that handler:

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: gvisor
handler: runsc
overhead:
  podFixed:
    cpu: "50m"
    memory: "64Mi"
scheduling:
  nodeSelector:
    sandbox.unixsingh.dev/runsc: "true"

The two optional blocks are the ones that make this production-shaped rather than a demo. scheduling.nodeSelector means a pod asking for this class only lands on nodes that actually have the runtime installed, instead of failing mysteriously on the ones that don't. overhead tells the scheduler that the sandbox itself costs something, so it stops packing nodes as if it were free. Skip that second one with Kata and you will overcommit your nodes, because a VM per pod is not a rounding error.

3. Ask for it in the pod. One line:

apiVersion: v1
kind: Pod
metadata:
  name: untrusted-job
spec:
  runtimeClassName: gvisor
  containers:
    - name: worker
      image: alpine
      command: ["sleep", "3600"]

4. Verify it actually landed in a sandbox, rather than quietly running on the default runtime:

And this is the shape I'd actually reach for: default runtime for the stuff you wrote and trust, a sandboxed RuntimeClass for the things you don't. Customer-supplied code, CI jobs building arbitrary repos, anything running a binary someone uploaded. You pay the overhead exactly where the risk is. If you want to force that choice rather than hope for it, a Gatekeeper constraint can require runtimeClassName on pods in a given namespace, which is a nicer control than a wiki page nobody reads.

Which one, honestly

Working out what you've landed in

If I've got execution in a pod on an authorised test, this is roughly what I'd check first, because it decides whether a kernel exploit is even worth trying:

uname -a                        # 4.4.0 with no distro string smells like gVisor
dmesg 2>/dev/null | head -3     # "Starting gVisor..." is the giveaway
cat /proc/version
ls /dev/kvm 2>/dev/null         # present in a Kata guest, usually absent in a pod
systemd-detect-virt 2>/dev/null # "kvm" from inside a Kata VM

# and from the API side, if you have read access
kubectl get runtimeclass
kubectl get pods -A -o custom-columns=NS:.metadata.namespace,POD:.metadata.name,RC:.spec.runtimeClassName

That last command is the useful one for a review rather than a break-in. It tells you instantly whether anybody is using the sandboxing that got installed six months ago, and in my experience the answer is often "one namespace, and not the risky one".

What shifted while writing this

The thing that actually shifted for me writing this is that hardening and sandboxing aren't two points on the same scale. I'd been thinking of them as weak, medium, strong. They're not. Seccomp and AppArmor narrow the door. gVisor and Kata change what's behind it. You want both, and adding more of the first will never get you the second.

My honest opinion on where the effort goes: for most people, most of the time, sandboxing everything is the wrong call. The overhead is real, the compatibility risk is real, and a cluster running your own reviewed code on a patched kernel with sensible seccomp profiles is not the thing keeping me up. Where I'd absolutely reach for it is anywhere you're running code you didn't write and can't review, and that's a smaller slice of most clusters than people assume. RuntimeClass exists precisely so you can make that a per-pod decision, and I think that's the most underused object in this whole area.

What I'm not sure about: I've run gVisor in a homelab and read a lot about Kata, but I have never operated either under real load with a real on-call rota. Every write-up says "small performance overhead" and I can't personally vouch for what that means when a service is busy at 3am. If you've run one in production, I'd genuinely like to hear which one and what broke.

Next topic, back to something more hands-on: what's actually inside your container image, and how much of it nobody chose.

References

• gVisor: architecture guide (Sentry, Gofer, Netstack)
• gVisor: platform guide
• gVisor: releasing Systrap, the new default platform
• gVisor: Docker quick start
• Kata Containers
• Kata Containers on GitHub
• Kubernetes docs: RuntimeClass
• Google Cloud: nested virtualisation

FAQ

What is container sandboxing?

Container sandboxing puts an extra boundary between a container and the host kernel, rather than just narrowing what the container may ask that kernel to do. gVisor answers most system calls in user space, and Kata Containers gives each pod a real kernel of its own inside a lightweight virtual machine.

How is a container different from a virtual machine?

A virtual machine boots its own guest kernel on a hypervisor, so a kernel bug inside it stays inside it. A container is just a process on the host, wrapped in namespaces and cgroups, using the same kernel as every other container on that node. That shared kernel is the whole difference.

What is the difference between gVisor and Kata Containers?

gVisor intercepts system calls and answers most of them in a user-space kernel called the Sentry, so it needs no hardware virtualisation. Kata boots a real Linux kernel per pod inside a lightweight VM, which is a stronger boundary but requires hardware virtualisation support on the node.

How do I run a pod under gVisor or Kata in Kubernetes?

Register the runtime with containerd on the node, create a RuntimeClass object naming its handler, then set runtimeClassName on the pod. It is a per-pod choice, so untrusted workloads can be sandboxed while everything else keeps running on the default runtime.

Does gVisor slow containers down?

It depends entirely on the workload. Anything that makes heavy use of system calls or file I/O pays the most, because every one of those crosses the Sentry. CPU-bound work that mostly stays in user space barely notices. Benchmark your own workload rather than trusting a general number.

Related reading

• Topic 25: Seccomp, blocking the syscalls a container never needs (narrowing the door)
• Topic 26: AppArmor and Linux capabilities (narrowing which files those calls reach)
• Topic 24: Linux syscalls, watching the kernel door (what the Sentry is standing in front of)
• Topic 30: OPA and Gatekeeper (how you'd require a RuntimeClass rather than hope for one)
• Browse the whole Kubernetes Journey