Secure and efficient access to Linux servers is a cornerstone for any SysAdmin, DevOps Engineer, or developer. For years, passwords were the standard, but they present significant security and usability limitations. Every time we type a password, we expose it to potential interception and brute-force attacks, while also slowing down daily operations. If you manage even a handful of servers, you know how frustrating and inefficient it is to enter credentials repeatedly.
Fortunately, SSH (Secure Shell) offers a superior alternative: public key authentication. This method not only enhances security by eliminating the need for weak or reused passwords, but it also streamlines your workflow, making logins fast and automation-ready. In this hands-on guide, I’ll walk you step-by-step through configuring passwordless SSH on Linux, from key generation to advanced management, covering even the most common pitfalls I’ve encountered in the field while working with complex infrastructures—like a public healthcare agency with hundreds of VMs.
Prerequisites / Test Environment
To follow along, you’ll need two Linux servers (or a Linux workstation and a remote server) with SSH access enabled and a sudo user. Our reference distribution is Ubuntu 22.04, but the concepts and commands apply universally across most Linux distributions. Make sure you have a non-root user for logging in. Our main focus is passwordless SSH on Linux, and we’ll see how to implement it in a real-world environment.
How Public Key Authentication Works
Public key authentication relies on a cryptographic key pair: a private key and a public key. The private key is generated on your local system (the client) and must remain secret, protected by restrictive permissions and, ideally, a passphrase. The public key, on the other hand, can be freely shared and is copied to the remote server you want to access.
When you attempt to connect, the client signals the server that it wants to authenticate using a key. In response, the server encrypts a message using the public key associated with your account and sends it to the client. Only your corresponding private key can decrypt this message. If the client successfully decrypts the message and sends the correct response, the server grants access. This process completely eliminates the need to transmit passwords or password hashes, drastically reducing the risk of interception and brute-force attacks.
Generating the Key Pair with ssh-keygen
The first step is generating your SSH key pair on the client system. We’ll use the Ed25519 algorithm, which is more modern and secure than traditional RSA, offering better performance and smaller key sizes.
Open a terminal on your client and type:
ssh-keygen -t ed25519 -C "nomeutente@hostname_client"You’ll be prompted to save the key (the default ~/.ssh/id_ed25519 is recommended) and to set a passphrase. It is highly recommended to set a strong passphrase to protect your private key. Even though the goal is passwordless SSH on Linux to the server, the passphrase protects your private key on the client. If someone gains access to your client, they cannot use your private key without the passphrase.
This command creates two files in the ~/.ssh/ directory:
id_ed25519: your private key.id_ed25519.pub: your public key.
Copying the Public Key to the Remote Server
Once the key pair is generated, you need to copy the public key to the remote server. The simplest and most secure tool for this is ssh-copy-id.
ssh-copy-id utente_remoto@ip_server_remotoYou will be prompted for the remote user’s password the first time. ssh-copy-id will take care of creating the ~/.ssh/ directory on the remote server (if it doesn’t exist), setting the correct permissions, and appending your public key to the ~/.ssh/authorized_keys file. This is the preferred method because it automatically handles all the necessary configurations, reducing the chance of errors.
Manual Method with authorized_keys
If for some reason ssh-copy-id is unavailable or you prefer a manual approach, you can copy the public key directly. This method requires closer attention to permissions.
To copy the public key to the remote server and append it to the authorized_keys file, use the following command:
bash
cat ~/.ssh/id_ed25519.pub | ssh utente_remoto@ip_server_remoto "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"This command executes several actions in sequence on the remote server:
mkdir -p ~/.ssh: creates the.sshdirectory if it doesn’t exist.chmod 700 ~/.ssh: sets the correct permissions for the.sshdirectory (readable, writable, and executable only by the owner).cat >> ~/.ssh/authorized_keys: appends the contents of your public key to theauthorized_keysfile.chmod 600 ~/.ssh/authorized_keys: sets the correct permissions for theauthorized_keysfile (readable and writable only by the owner).
It is crucial that these permissions are set correctly; otherwise, SSH will ignore the key for security reasons, preventing passwordless SSH on Linux.
Verifying the Connection
After copying the key, try connecting to the remote server:
ssh utente_remoto@ip_server_remotoIf everything is configured correctly, you should gain access without being prompted for a password (only the private key passphrase, if you set one). If you are still prompted for the remote user’s password, there is a configuration issue (likely permissions).
Configuring ~/.ssh/config for Connection Aliases
To simplify access even further, you can configure a ~/.ssh/config file on your client. This allows you to define aliases for your servers, specifying the user, host, port, and the private key to use. This is particularly useful when managing multiple servers or complex environments, like those I managed for a healthcare client.
Create or edit the ~/.ssh/config file (if it doesn’t exist, create it with touch ~/.ssh/config and chmod 600 ~/.ssh/config):
Host server_web
Hostname 192.168.1.100
User utente_web
IdentityFile ~/.ssh/id_ed25519
Host db_prod
Hostname 10.0.0.5
User utente_db
Port 2222
IdentityFile ~/.ssh/chiave_db_privataNow you can connect simply with ssh server_web or ssh db_prod.
Correct Permissions on .ssh (Most Common Error)
As mentioned, permissions are critical. SSH is extremely strict about this. If the ~/.ssh/ directory or the ~/.ssh/authorized_keys file (or the private key ~/.ssh/id_ed25519) have overly permissive permissions, SSH will safely ignore the keys and fall back to password authentication.
Ensure the permissions are as follows:
On the client (for the private key):
chmod 600 ~/.ssh/id_ed25519On the remote server (for the .ssh directory and authorized_keys file):
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keysThese commands ensure that only the owner can read and write these files and directories. [Read also: Hardening SSH on Linux: Practical Guide 2026] to dive deeper into SSH security.
Using ssh-agent to Avoid Typing the Passphrase Every Time
If you protected your private key with a passphrase (which is highly recommended), ssh-agent allows you to unlock it just once per session, avoiding typing it on every connection. ssh-agent is a program that manages your private keys and unlocks them when needed.
To start ssh-agent and add your key:
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519You will be asked to enter the key’s passphrase. Once entered, ssh-agent will keep it in memory for the duration of your session, allowing for fully automated passwordless SSH on Linux.
Connecting to Multiple Servers with the Same Key
A major advantage of public key authentication is the ability to use the same private key (and its corresponding public key) to access multiple servers. This greatly simplifies access management in environments where a user needs access to dozens or hundreds of virtual machines. Simply copy the same public key to all desired servers using ssh-copy-id or the manual method.
Common Errors and Troubleshooting with ssh -vvv
When public key authentication doesn’t work as expected, debugging is essential. The -vvv flag for ssh is your best friend, providing verbose output that shows exactly what is happening during the connection attempt.
ssh -vvv utente_remoto@ip_server_remotoWhat to look for in the output:
debug1: Authentication methods that can continue: publickey,password: Indicates which authentication methods the server is willing to accept.debug1: Trying private key: /home/utente/.ssh/id_ed25519: SSH is attempting to use your private key.debug1: identity file /home/utente/.ssh/id_ed25519 type 3: The key was loaded successfully.debug1: Authentication failed: publickey: This is the sign that public key authentication failed. The most common causes are:- Incorrect permissions: Check the permissions on
~/.ssh/and~/.ssh/authorized_keyson the remote server, and on the private key on the client. - Public key not present: Ensure your public key is actually in the
authorized_keysfile on the server. - Wrong username: The user specified in the
sshcommand must match the user on the server that has the public key configured. - SSH server not configured to accept keys: Rare, but check
/etc/ssh/sshd_configon the server forPubkeyAuthentication yesandAuthorizedKeysFile .ssh/authorized_keys.
For more details on configuring sshd_config, you can check the official documentation: OpenSSH sshd_config.
Conclusion with Key Takeaways
Configuring passwordless SSH on Linux via public key authentication is a crucial step to improve the security and efficiency of your IT operations. It’s not a matter of “if,” but “when” you should implement it. It allows you to eliminate password-related vulnerabilities, speed up logins, and prepare your infrastructure for automation with tools like Ansible. Always remember to protect your private key with a passphrase and keep .ssh file permissions strictly correct. By adopting these practices, you will ensure a more secure and productive environment for you and your organization, reducing the risk of incidents and optimizing your valuable time.
