Default SSH Leaves the Front Door of Every Node on the Latch

Every Kubernetes node I run has one door I keep coming back to, and it isn't the API server. It's SSH. That's the way I get onto the box to poke at logs, restart the kubelet, or fix the thing that broke at 11pm. Which means it's also the way an attacker gets on if they find it soft. And out of the box, SSH is softer than most people think.

The default setup lets you log in with a password, and it'll happily let root log in too. Both of those are gifts to anyone scanning the internet for port 22. In this one I want to close that door properly: keys instead of passwords, no direct root, and password login switched off once keys work. It's not clever, it's just tidy, and it's the sort of hardening that stops the boring attacks that actually get people. Let me walk you through it, and we'll do it as a lab so you can copy along.

What SSH actually is (in one breath)

SSH (Secure Shell) is how you get a command line on another machine over the network, safely. You sit on your laptop (the client), the server runs a small program called sshd that listens on port 22, and once you prove who you are, you get a shell on that box as if you were sat in front of it.

You run ssh someuser@somehost, the two machines set up an encrypted tunnel so nobody on the wire can read what you type, and then the server checks your identity. That check is the important bit for this post. You can prove yourself with a password, or with a key pair, and the whole game of hardening is about which one you allow.

Whoever can log in over SSH can run commands as that user. On a Kubernetes node that user is usually one sudo away from root, and root on the node sits right next to the kubelet and every pod running there. So SSH isn't just admin convenience, it's a direct route to the cluster if it's left open.

SSH is the front door to the node. Default SSH leaves that door on the latch.

Why this bites harder on a node

A regular server going down is one server's problem. A node is different. It's a shared machine that runs other people's workloads, and it holds the kubelet, the agent that talks to the control plane and can read the pods and secrets scheduled onto it. Get root on a node and you're not looking at one app, you're looking at a foothold in the cluster.

Here's the part people miss: attackers rarely bother breaking the crypto. They log in the lazy way. Password spraying (trying a few common passwords against lots of accounts) and brute-forcing root are still some of the most common things you'll see hitting an exposed box, because they still work often enough. Hardening SSH is you refusing to play that game. No password to spray, no root to target. It's the same idea as least privilege, just pointed at the login itself.

Step 1: log in the plain, password way

Before hardening anything, let's see the default. To reach a host from your laptop you run ssh with the username and the address. If you leave the username off, SSH just reuses your local one, which is a nice shortcut but easy to forget when the remote user is different.

# both of these do the same thing if the remote user is "mark"
ssh mark@node01
ssh -l mark node01

# and if your laptop user is already "mark", this is enough
ssh node01

The first time you connect you'll get a prompt to accept the host's fingerprint (that's your client remembering the server so it can warn you if it ever changes), then it asks for the remote user's password. Type it, and you're in. That's the whole default experience, and it's exactly what we're about to make better.

Step 2: swap the password for a key pair

What a key pair is: two matching files, a private key that never leaves your laptop and a public key that you hand out to servers. Anything the public key locks, only the private key can open. So the server keeps your public key, and when you connect it sets a little challenge only your private key can answer. Nothing secret ever crosses the wire, and there's no password sitting on the server to steal or guess.

Generate the pair

You make the pair once, on the client, with ssh-keygen. I'll use Ed25519, which is a modern key type that's small and fast (older guides say rsa, and that's fine too, but Ed25519 is the nicer default now).

It asks for a passphrase, and yes, add one. It feels like it defeats the point of a passwordless login, but it doesn't: the passphrase encrypts the private key file on your disk, so if your laptop is nicked, the key is useless without it. You type it once when you first use the key, an ssh-agent holds it for the session, and you're not typing it every login. Skip it and a stolen laptop is a stolen key.

Put the public key on the server

Now copy the public half to node01. The ssh-copy-id tool does the fiddly part for you: it logs in with your password this one last time and appends your public key to the right file on the server.

# the easy, correct way
ssh-copy-id mark@node01

# it lands here on the server, one key per line:
cat ~/.ssh/authorized_keys

That file, ~/.ssh/authorized_keys in the remote user's home, is the server's guest list. Every public key in it is allowed to log in as that user. Once your key's in there, connect again and watch what happens: no password prompt, straight to a shell. That's the key pair doing its job.

Step 3: lock down sshd_config

Keys work, but the server will still accept passwords and still let root in until you tell it not to. That config lives in one file on the server, /etc/ssh/sshd_config, and you edit it as root (or with sudo). Three changes do most of the work.

Turn off direct root login

Why: root is the one username that exists on every Linux box, so an attacker already knows half your login. Take it off the table. You log in as a normal user and step up with sudo when you need to, which also leaves a trail of who did what.

# in /etc/ssh/sshd_config
PermitRootLogin no

Turn off password login

Why: now that your key works, passwords are pure downside. This one line is what actually kills password spraying and brute-force against the box, because there's simply no password path left to try.

# in /etc/ssh/sshd_config
PasswordAuthentication no

While you're in the file, a couple of quick extras I always set: cap how long someone gets to authenticate and how many tries they get. Small thing, but it trims the noise from bots.

# optional but nice: fewer, shorter attempts
MaxAuthTries 3
LoginGraceTime 20

Save it, then tell sshd to re-read the config. Restarting the service does that.

sudo systemctl restart ssh   # some distros call it sshd

Don't lock yourself out (read this bit)

Here's the quick check I run in that second terminal. If it drops me to a shell with no password prompt, hardening's done and it's safe to walk away.

What a defender watches for

Hardening is half the job. The other half is noticing when someone's trying the door. On the server, SSH login activity gets logged, usually to /var/log/auth.log on Debian and Ubuntu, or via journalctl -u ssh. A wall of failed logins for root or for users that don't exist is the exact fingerprint of the spraying I mentioned earlier.

# failed attempts, newest last
sudo grep "Failed password" /var/log/auth.log | tail

# who actually got in, and how
sudo grep "Accepted" /var/log/auth.log | tail

Two things worth doing once you've read those logs: put SSH behind the internal network or a bastion so port 22 isn't facing the whole internet, and add something like fail2ban that watches those logs and temporarily blocks an IP after a few failures. Neither is in the original hardening steps, but they're the obvious next moves once you've seen how much noise a public port 22 attracts. Score the whole node against a benchmark too, the CIS checks flag weak SSH config automatically.

Where this leaves the node

So that's the front door sorted. Keys instead of passwords, so there's nothing to spray. No root login, so the one guaranteed username is off the table. Password auth off, so the lazy attacks have nowhere to land. And a passphrase on the private key, so a lost laptop isn't a lost cluster. None of it is fancy, and that's sort of the point, most nodes get done in by the boring stuff, not clever zero-days.

One honest caveat: I've done this by hand on a couple of VMs, which is great for learning but doesn't scale. Doing it across a real fleet means baking it into an image or a config-management tool so every node comes up hardened, and I haven't wired that into my homelab properly yet. That's the bit I want to get right next, because a hardening step you have to remember to run is a hardening step you'll eventually forget. Shout if you've got a setup you like for that, I'm keen to compare notes.

Further reading

If you want to go deeper than this, the OpenSSH manual pages (man sshd_config) are the real source of truth for every directive, and the CIS Benchmark for your distro has a full, opinionated list of SSH settings worth applying. Both are more thorough than one post can be.

FAQ

What is SSH hardening?

SSH hardening means tightening how you log into a Linux box over SSH so it is harder to break into. In practice that is three moves: swap passwords for a key pair, stop the root account logging in directly, and turn password login off once keys work. Fewer ways in, less to guess.

Why use SSH keys instead of passwords?

A password can be guessed, sprayed or phished. An SSH key is a huge random pair of files, and only the public half sits on the server, so there is nothing useful to steal there and nothing to brute-force. You also stop typing a password every login, which is faster and safer at once.

Should I disable root login over SSH?

Yes. Root is the one username every attacker already knows, so leaving it open hands them half the login for free. Set PermitRootLogin no, log in as a normal user, and use sudo for admin work. You get the same power plus a clear record of who did what.

How do I avoid locking myself out when hardening SSH?

Keep your current session open. After you change sshd_config and restart the service, open a second terminal and log in with your key before you close the first one. If the new session works, you are safe. If it does not, you still have the old session to fix it.

Does SSH hardening still matter on a private network?

Yes. Private does not mean safe. If one pod or box on that network gets popped, the attacker is now inside with you, and a node that still takes root and passwords is the easiest next hop. Hardened SSH means that internal foothold does not quietly become the whole cluster.

Related reading

• Topic 19: Least Privilege (the mindset this whole post is an example of)
• Topic 11: Kubelet Security (what sits right behind the node you just locked down)
• Topic 5: CIS Benchmarks & Kube-Bench (score your SSH config automatically)
• Browse the whole Kubernetes Journey