Password authentication over SSH is the largest exposed surface an ordinary server has. Neither Fail2ban nor any threshold helps against it: a thousand addresses at one attempt an hour will never reach a limit, and they will keep guessing for exactly as long as guessing remains possible at all. Keys remove the possibility itself, and of the whole hardening list this is the one setting that changes the situation qualitatively.

Only one thing about it is frightening: switching the password off and being left outside. Here is the order in which that cannot happen.

The key

Generated on your own machine, not on the server:

ssh-keygen -t ed25519 -C "work laptop"

ed25519 rather than RSA: shorter, faster, and with no questions about length. RSA is needed only if you have to reach something very old; then -t rsa -b 4096.

A passphrase on the key is worth setting: the key file can be stolen off a laptop, and without a passphrase it works immediately. You will not have to type it every time — ssh-agent handles that, and on macOS and most desktop Linux systems it is already running.

Copying it to the server takes one command, while the password still works:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@203.0.113.25

If ssh-copy-id is unavailable, the contents of the .pub file are appended as a line to ~/.ssh/authorized_keys on the server. Permissions matter: the .ssh directory 700, the file 600, owned by the user. With anything else sshd silently refuses to read the file, and that is the most common cause of "the key does not work".

The check that settles it

Before switching anything off, open a second connection without closing the first:

ssh -o PasswordAuthentication=no user@203.0.113.25

The option forbids the client from falling back to a password — so if the login succeeded, it succeeded by key. Until that command works, do not move on. And keep the first connection open to the very end: it is what fixes anything you manage to break.

Switching passwords off, and one non-obvious catch

Settings go in a separate file so that a package update cannot overwrite them. But the file name matters:

sudo tee /etc/ssh/sshd_config.d/10-hardening.conf <<'EOF'
PasswordAuthentication no
PermitRootLogin no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
LogLevel VERBOSE
EOF

The thing is that OpenSSH uses the first value it obtains for a parameter, not the last, as almost everywhere else. Files in sshd_config.d are read alphabetically, and Ubuntu cloud images put 50-cloud-init.conf there, which frequently contains PasswordAuthentication yes. Your file numbered 90 will do nothing in that situation: the setting appears to be in place while the password keeps working. Hence the number 10 — it is read earlier.

You can see what actually came out without guessing:

sudo sshd -T | grep -Ei 'passwordauth|permitrootlogin|pubkeyauth'

That command prints the effective configuration after all the includes. Trust it, not the contents of the files.

Then a syntax check and a gentle reload of the service:

sudo sshd -t && sudo systemctl reload ssh

sshd -t is mandatory: a typo in the config with restart leaves the service down and you outside. reload also does not break open sessions.

Ubuntu 24.04: the port is not where you think

A separate trap on recent Ubuntu: sshd there is started through a systemd socket. In that mode the Port line in sshd_config does nothing — the socket listens, not the daemon. If you are changing the port, that is what has to be edited:

sudo systemctl edit ssh.socket

and ListenStream= (an empty value resets the previous one) has to name the new port. The symptom that gives it away: the config is changed, the service restarted, and the server still answers on 22.

While we are on the subject of changing the port: as protection it does not work — scanners find a service on any port within minutes. Its only effect is quieter logs, because mass guessing only goes to 22. That is convenient, but do not confuse it with security.

Who may log in at all

A useful addition is an explicit list:

AllowUsers deploy admin

Anything not listed is cut off before the key is even checked. That also covers the case where some package creates a system user with a shell and a home directory.

A spare way in

One key on one laptop is a single point of failure. The disk dies, the laptop is lost, and the server becomes unreachable for good. A sensible minimum:

  • a second key from another device in authorized_keys;
  • a copy of the private key in a password manager or on an encrypted stick;
  • tested console access at your provider — VNC or serial. Tested means you have logged in through it at least once, not that "there is a button somewhere in the panel".

While you are there, look at what is already in authorized_keys — both your user's and root's. Somebody else's key there survives any password change and remains working access.

Afterwards

Password guessing will not go away, it simply becomes useless: the logs will keep thousands of Failed password lines, none of which can end in success. What is worth watching now is the successful logins: from which address, under which user, at what time. A successful key login from an unfamiliar address is a far more important event than a million failures, and much harder to notice in the general stream. That is the picture the demo page below shows.