How to Add an SSH Key on Linux? Generating and Configuring Step by Step

We write this guide from a practitioner's perspective — at Nodea we configure SSH access on Bare Metal and Public Cloud servers every day as part of our Linux administration service. We'll show you step by step how to generate a key pair on Ubuntu, Debian and CentOS, add the public key to the server and test passwordless login. Every command is ready to copy.

What is an SSH key and how does key-based login work?

An SSH key is a pair of files — a public key and a private key — that lets you log in to a Linux server without entering a password. An SSH key replaces password login and is resistant to brute-force attacks, because a cryptographic key can't be guessed. The pair splits into two files:

• the public key goes on the server and can be shared freely,
• the private key stays on your computer and is never shared with anyone.

SSH itself (the SSH protocol) is an encrypted connection to the server — logging in with a key is safer than with a password. The mechanism is simple: the server stores your public key and, at login, checks whether you hold the matching private key. If the two files match, the server lets you in without asking for a password — without sending any secret over the network.

That answers three common questions. The SSH public key is the id_rsa.pub file, which you place on the server and can share. The private key (id_rsa) stays with you and is secret. And why is this safer than a password? A password can be cracked by brute force; a private key thousands of bits long cannot. You'll find more about the protocol itself in our SSH glossary entry.

Generate an SSH key with ssh-keygen

How do you generate an SSH key on Linux? Open a terminal and type ssh-keygen -t rsa -b 4096 — it creates a 4096-bit key pair in the ~/.ssh directory. The basic and extended versions of the command look like this:

• ssh-keygen — the basic version, with default parameters,
• ssh-keygen -t rsa -b 4096 -C "your_email@domain.com" — with a chosen type, length and comment.

The parameters mean, in order: -t rsa is the key type (the RSA algorithm), -b 4096 is the length in bits, and -C adds a comment (usually an email) that makes the key easier to recognize later. The comment is optional.

During generation the system asks two questions. First about the file location — by default ~/.ssh/id_rsa, confirm with Enter. Then about a key passphrase, which you can also leave empty. In the terminal you'll see roughly this:

• Enter file in which to save the key (/home/user/.ssh/id_rsa):
• Enter passphrase (empty for no passphrase):
• Your identification has been saved in /home/user/.ssh/id_rsa
• Your public key has been saved in /home/user/.ssh/id_rsa.pub

When it finishes, the ~/.ssh directory holds two files: the private key id_rsa and the public key id_rsa.pub. RSA isn't the only key type — the available SSH commands also include ssh-copy-id (copy to the server) and ssh (connect), which we cover in the next sections. Choosing the algorithm is explained below.

Check whether you already have an SSH key

Before generating a new SSH key, check whether you already have one — type ls -al ~/.ssh in the terminal. The command lists the contents of the key directory.

• If the output shows id_rsa and id_rsa.pub (or id_ed25519 and id_ed25519.pub), a key already exists.
• Generating a new key in the same location overwrites the old one without warning — access based on the previous key then stops working.

If you already have a key and still use it, skip straight to the section on copying the public key to the server.

Set a passphrase for the private key

A passphrase protects your private key if the id_rsa file falls into the wrong hands — without the passphrase no one can use it. You set the passphrase during generation, in answer to the Enter passphrase prompt.

• A passphrase is optional but recommended on workstations.
• Without one, anyone who obtains the private key file can log in to your server.
• Leave the field empty (just Enter) if the key is meant for automation — deployment scripts or cron jobs where no one types a passphrase by hand.

Which SSH key type to choose — RSA or ED25519?

An SSH key comes in two popular types: RSA (minimum 2048 bits, 4096 recommended) and ED25519 (a fixed length of 256 bits). ED25519 is newer and faster; RSA is more widely supported by older systems. The commands that generate each type are:

• RSA: ssh-keygen -t rsa -b 4096
• ED25519: ssh-keygen -t ed25519

The comparison below helps you pick the right type:

• RSA — 2048 or 4096 bits; slower; maximum compatibility, including with older servers; choose it when you need certainty the key works everywhere.
• ED25519 — a fixed 256 bits; faster and with a shorter key at comparable security; choose it for new systems and modern Linux distributions.

The recommendation in two sentences: for new servers choose ED25519 — it's faster and more modern. When you must support older systems or network devices, stick with SSH RSA at 4096 bits.

Add the public key to your Linux server

To add an SSH key to a Linux server, copy the public key into the ~/.ssh/authorized_keys file on the server — most easily with ssh-copy-id user@server_ip_address. The command creates the .ssh directory, appends the key and sets permissions on its own.

Replace user with the username on the server and server_ip_address with the IP address or hostname. The automatic method goes like this:

1. Type ssh-copy-id user@server_ip_address.
2. Enter the account password on the server (this is the last time you type it).
3. ssh-copy-id copies your SSH public key into the authorized_keys file on the server.

If ssh-copy-id isn't available (common on minimal server images), copy the key manually with a single command:

• cat ~/.ssh/id_rsa.pub | ssh user@server_ip_address "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

This command sends the contents of id_rsa.pub over SSH and appends it to authorized_keys, first creating ~/.ssh if it didn't exist. Both methods lead to the same result: your SSH public key ends up in the ~/.ssh/authorized_keys file on the server — the list of keys the server trusts. Configuring this file correctly is a standard we apply at Nodea on Bare Metal and Public Cloud servers. If you copy files alongside SSH, see how to connect to an FTP account.

Set the correct permissions on the key directory and file

The SSH server will refuse key-based login if the ~/.ssh directory and the authorized_keys file have overly loose permissions — set chmod 700 ~/.ssh and chmod 600 ~/.ssh/authorized_keys. Run both commands on the server:

1. chmod 700 ~/.ssh — a directory accessible only to the owner.
2. chmod 600 ~/.ssh/authorized_keys — a key file readable and writable only by the owner.

The SSH daemon deliberately rejects keys when permissions are too broad — a safeguard against another server user appending their own key. This is the most common reason for "I did everything and the key still doesn't work." A diagnostic sentence to remember: if login asks for a password despite the key being added, check the permissions on the ~/.ssh directory and the authorized_keys file.

Test passwordless SSH key login

To check whether the SSH key works, connect to the server with ssh user@server_ip_address — if the configuration is correct, you'll log in without entering a password. SSH key login should happen instantly and drop you straight into the server shell.

• Success means logging in without being asked for the server password (if the key has no passphrase).
• If you set a passphrase, the system asks for the key passphrase — not the server password. That's an important difference.
• If the server still asks for the server password despite the added key, go back to the permissions section and check chmod on ~/.ssh and authorized_keys.

This difference confuses beginners: the "server password" is the account password on the server, while the "key passphrase" encrypts the local id_rsa file. If the system asks for a passphrase after you add the key, it means the key WORKS — it's asking for the key passphrase, not the server password. We cover connecting to a server over SSH in more depth in our SSH entry.

Set up ~/.ssh/config and ssh-agent for convenience

The ~/.ssh/config file lets you log in to a server with a short alias instead of the full command — instead of ssh user@192.168.0.255 you type ssh myserver. Just add a few lines to the config file:

• Host myserver
•   HostName 192.168.0.255
•   User user
•   IdentityFile ~/.ssh/id_rsa

The fields mean, in order: Host is the alias you'll use in the command, HostName is the server's IP or name, User is the username, and IdentityFile is the path to the private key. From now on you log in with ssh myserver.

The second convenience is ssh-agent — a tool that keeps the private key passphrase in memory so you don't type it on every login. Start it and add the key with two commands:

1. eval "$(ssh-agent -s)" — starts the agent in the current terminal session.
2. ssh-add ~/.ssh/id_rsa — adds the private key to the agent (you enter the passphrase once).

From then on ssh-agent supplies the key passphrase for you until the session closes. If you also set up access from Windows, see how to add an SSH key with PuTTY on Windows. Need help configuring your server? Nodea offers a Linux administration service — we'll handle SSH, the firewall and the whole setup for you.