SSH Keys: Setting Up Passwordless Server Login

An SSH key is a pair of files: the private key stays with you, and the public key goes on the server. When you connect, the server checks that you hold the private key matching one of the allowed public keys, and lets you in without a password. This is more convenient and far more secure than a password: guessing a key by brute force is practically impossible. Below is SSH key setup from generation to disabling password login, with the errors people run into most often.

How key authentication works

Keys are asymmetric. The public key (id_ed25519.pub) can be shown to anyone — it cannot be used to recover the private key. The private key (id_ed25519) never leaves your machine and is never sent over the network.

When you log in, the client tells the server which key it wants to use. The server looks for this public key in the user's ~/.ssh/authorized_keys file, and if it finds it, asks the client to sign the session data with the private key. The signature is checked against the public key, and the login is allowed. No password is transmitted at all.

Step 1. Generate a key: ssh-keygen

On your computer (Linux, macOS or Windows 10/11 with built-in OpenSSH) run:

ssh-keygen -t ed25519 -C "alena@laptop"

The -t flag sets the algorithm, -C sets a comment that is added to the end of the public key and helps you later see whose key it is. Ed25519 is the modern recommended algorithm: keys are short, fast and secure. If the server is very old and does not support Ed25519, use RSA with at least 3072 bits: ssh-keygen -t rsa -b 4096.

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/alena/.ssh/id_ed25519):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/alena/.ssh/id_ed25519
Your public key has been saved in /home/alena/.ssh/id_ed25519.pub
The key fingerprint is:
SHA256:Xq3v0m6c1y2pVv8m3N0uYv9yQm0e6dFv5gH2rZt1aBc alena@laptop

Do you need a passphrase

The passphrase encrypts the private key on disk. If your laptop is stolen or the file leaks, the key is useless without it. For a personal key it is worth setting a passphrase, and to avoid typing it on every connection, use ssh-agent: it keeps the decrypted key in memory until the end of the session.

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
ssh-add -l      # list loaded keys

Keys without a passphrase are justified for automation — for example, for CI or a backup script. Such a key should be restricted on the server (see below).

Step 2. Copy the public key to the server: ssh-copy-id

The simplest way is ssh-copy-id. It connects with your password one last time, appends your public key to ~/.ssh/authorized_keys on the server and sets the correct permissions.

ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@203.0.113.10

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/alena/.ssh/id_ed25519.pub"
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
deploy@203.0.113.10's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'deploy@203.0.113.10'"
and check to make sure that only the key(s) you wanted were added.

If the server listens on a non-standard port, add -p 2222.

If ssh-copy-id is not available

There is no ssh-copy-id utility on Windows. The key can be transferred manually with one command:

cat ~/.ssh/id_ed25519.pub | ssh deploy@203.0.113.10 "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Cloud providers usually ask for the public key when you create a server — it lands in authorized_keys through cloud-init.

Step 3. Test the login

ssh deploy@203.0.113.10

If you are no longer asked for a password (or only for the key's passphrase), everything works. On the first connection to a new server, SSH shows the host key fingerprint and asks whether to trust it. After you answer yes, the fingerprint is saved in ~/.ssh/known_hosts, and from then on SSH warns you if the server's key changes.

To see which keys the client tries and why the server refuses, run the connection with verbose output:

ssh -v deploy@203.0.113.10

The ~/.ssh/config file: short names for servers

When you have several servers, typing the user, IP, port and key path every time is tedious. Describe all of this in ~/.ssh/config:

Host prod
    HostName 203.0.113.10
    User deploy
    Port 2222
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes

Host bastion
    HostName bastion.example.com
    User alena

Host db-internal
    HostName 10.0.0.15
    User admin
    ProxyJump bastion

Host *
    ServerAliveInterval 60
    AddKeysToAgent yes

Now ssh prod is enough, and scp, rsync and Git understand these names too: rsync -av ./site/ prod:/var/www/.

• IdentitiesOnly yes — use only the specified key. Without this line the client tries every key from the agent, and the server may disconnect you for too many attempts before it reaches the right one.
• ProxyJump — connect to an internal server through an intermediate one (a bastion) in a single command.
• ServerAliveInterval — send a "ping" every 60 seconds so the connection does not drop when idle.

The file permissions must be 600, otherwise SSH may refuse to read it: chmod 600 ~/.ssh/config.

Agent forwarding: when it is needed and why it is risky

Sometimes you need to connect onward from a server — for example, to run git pull from a private repository. You must not copy your private key there. Instead, use ssh -A or ForwardAgent yes: the server gets access to your agent for the duration of the session. But anyone with root on that server can also use the agent while you are connected. So enable forwarding only for trusted hosts, and for hopping through an intermediate server prefer ProxyJump: it does not expose the agent.

Step 4. Disable password login

As long as password login is allowed, the server remains a target for brute-force attacks. Once you have confirmed that you log in with a key, disable passwords. Important: do not close your current session until you have tested the login in a new window, otherwise you may lose access.

Open /etc/ssh/sshd_config (or create a file in /etc/ssh/sshd_config.d/ if the main config includes it with an Include directive) and set:

PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
PermitRootLogin no

PermitRootLogin no forbids logging in as root at all: administrators log in with their own users and gain privileges through sudo. If you cannot do without root login, use PermitRootLogin prohibit-password — key-based only.

Check the syntax and restart the service:

sudo sshd -t
sudo systemctl reload ssh      # on Debian/Ubuntu; on RHEL-like systems — sshd

The command sshd -t prints nothing if the config is correct. Now, in a new terminal, make sure key login works and that the server refuses password login:

$ ssh -o PubkeyAuthentication=no deploy@203.0.113.10
deploy@203.0.113.10: Permission denied (publickey).

Additional protection

Disabling passwords is the main measure. Next, access to port 22 is usually restricted with a firewall (ufw allow from 198.51.100.0/24 to any port 22) and fail2ban is installed, which bans an IP address after a series of failed attempts. The SSH protection status is shown by sudo fail2ban-client status sshd.

A key used for automation can be restricted directly in authorized_keys with options before the key: for example, from="198.51.100.7",no-port-forwarding,no-pty ssh-ed25519 AAAA... backup allows login only from a single address and without an interactive shell.

Common errors and how to fix them

Permission denied (publickey)

The most common error. Causes, from most to least frequent:

1. Wrong permissions on the server. sshd ignores authorized_keys if the directory or file is writable by others. Fix: chmod 700 ~/.ssh, chmod 600 ~/.ssh/authorized_keys, and the home directory must not be writable by the group or others.
2. Wrong owner. If you created files with sudo, they belong to root. Fix: sudo chown -R deploy:deploy /home/deploy/.ssh.
3. The key went to the wrong user. You copied the key for root, but log in as deploy, or the other way around.
4. The client offers the wrong key. Check ssh -v: the Offering public key lines show which keys are tried. Specify the right one with -i or IdentityFile.
5. The key was damaged while copying. The public key must be a single line. A line break introduced by the clipboard breaks it.

The server-side reason for the refusal is shown in the log: sudo journalctl -u ssh -n 30 (on RHEL — -u sshd). Lines like this appear there: Authentication refused: bad ownership or modes for directory /home/deploy/.ssh.

UNPROTECTED PRIVATE KEY FILE

The client refuses to use a private key that other users can read. Fix: chmod 600 ~/.ssh/id_ed25519.

REMOTE HOST IDENTIFICATION HAS CHANGED

The host key does not match the one saved in known_hosts. This happens after reinstalling a server or changing its IP, but it looks the same as a "man-in-the-middle" attack. Make sure the server really was rebuilt, and remove the old entry: ssh-keygen -R 203.0.113.10.

Too many authentication failures

The agent offered the server too many keys, and it closed the connection because of MaxAuthTries. The solution is IdentitiesOnly yes and an explicit IdentityFile in ~/.ssh/config.

Summary

Setting up SSH keys takes five minutes: ssh-keygen -t ed25519, ssh-copy-id, testing the login, then PasswordAuthentication no and reloading sshd while a second session stays open. The ~/.ssh/config file saves you from long commands, and the 700/600 permissions solve most "Permission denied" problems. SSH, rsync and server protection with a firewall and fail2ban are covered in practice in our Linux course.