Managing Multiple SSH Keys And Servers

Either you are using SSH keys to connect multiple machines or managing multiple accounts in Github or other services, you end up with many ssh keys sitting somewhere like a trash. Let’s manage them in effective way and let’s get over that mess of trash.

Scenario of Using Multiple SSH Keys

Here are the few scenarios where you may are using multiple SSH keys:

• Using EC2 instances or VPCs on the cloud
• Using two or more accounts in services like Github and others
• Managing any kind of remote servers or working remotely

Whatever may be your scenario of having these many keys, trust me we all ended up in having a key mess — many keys sitting there and not knowing what are those for. And we’ll solve this.

How to Manage Multiple SSH Keys?

The idea is very simple. OpenSSH has provided us a way of managing multiple keys using config file which is located at ~/.ssh/config in Linux and $HOME\.ssh\config in Windows. Let’s not start by creating these file right away. Yes, it will provide the solution but hides the problem. So, it’s better to start with the problem we are facing now.

Before: What We are Doing Wrong Now?

For simplicity, let’s say we have two pair of keys to manage, namely:

india_server
india_server.pub
us_server
us_server.pub

These keys are created using:

ssh-keygen -t ed25519 -C "[email protected]" -f us_server
ssh-keygen -t ed25519 -C "[email protected]" -f india_server

It will create two pair of keys listed above with the strongest algorithm type ed25519 which is the recommended one.

Now when connecting to the server, we are providing these keys with -i (providing IdentityFile) option of ssh client:

ssh -i us_server [email protected]
ssh -i india_server [email protected]

Everytime you are connecting to any of the server, you must provide Identityfile (with -i). This doesn’t seem a problem or headache until:

• you are doing this repeatedly
• you have have tens of more server to manage
• you may need to invoke those servers from some kind of script.

Another subtle often ignored point is that these Identityfile may not always live in one standard location (~/.ssh/), they are often scattered especially if you are managing on-the-fly or ad-hoc kind of projects like testing, learning or whatever.

After: The Solution

There is one naive and one better solution to this problem. If you are in a hurry and working in temporary project, naive solution might be okay for your but still it’s all good to setup with better solution for any kind of taks you are doing. Let’s go one by one.

ssh-agent: The Naive Solution

You can fire up ssh-agent to start ssh agent and eval it so that it configures the environment of the running shell to point to that agent. It does this by setting some enviornment variables (env vars). We need ssh-agent because now we can ask it to hold our ssh keys (using ssh-add see below) which we were manually providing before.

eval $(ssh-agent -s)

Let’s add our keys to the agent:

ssh-add us_server
ssh-add india_server

Agent signals:

Identity added: india_server ([email protected])
Identity added: us_server ([email protected])

Now that we have added our keys to the agent, we have gained the following advantages:

1. We don’t need to provide IdentityFile now, just skip the -i part.

ssh us_server [email protected]
ssh india_server [email protected]

2. ssh-agent also holds the credentials (the password you setup for your key while generating keys), this means you don’t have to type password each time you connect to the server.

But we still got problems to solve

1. If you terminate current shell session, you have to do it again. It doesn’t even persist past session, reboot is far down the road.
2. ssh-agent still doesn’t know which key is for which server, for two keys it is max of 2 tries per host at worst case scenario. Multiply this with number of keys and hosts you are using. You will soon start seeing “Too many authentication failures” on some servers.

ssh_config: The Better Solution

The config file approach solves all the problem we talked about. Edit or create $HOME/.ssh/config file:

Host developer-us_server
    HostName 100.0.0.2
    User developer
    IdentityFile /path/to/us_server
Host hacker-india_server
    HostName hacker.com
    User hacker
    IdentityFile /path/to/india_server

The config file hold everything we need to connect to remote server using ssh:

ssh -i <IdentityFile> <User>@<Hostname>

See every directive in config file maps to exactly what’s required to complete the command. And this mapping is what Host <name> part does. So, ssh Host hacker-india_server literally means:

ssh -i /path/to/india_server [email protected]

That’s it, noting more. If you are confused on what goes on this config file, see the man page.

Why ssh_config is Best of All?

Why use ~/.ssh/config?

• Persistent solution
• Shorter commands
• Avoids trying every key in your agent (which can cause “Too many authentication failures” on some servers) because it knows the key beforehand. No second guess, No Hit and Trial.
• Lets you configure other options (port, proxy, forwarding, etc.) per host

A Basic Advance configuration to show why is it the best. Even if you have 100s of servers in your *.example.com subdomain which all uses same key (although not recommended to use), this is what you specify in the config file. Three lines working for you.

Host *.example.com
    User ubuntu
    IdentityFile /path/to/ssh_key

Missing HostName part means ssh will match whatever you pass in command line with *.example.com which matches abc.example.com, xyz.example.com and more but not example.com itself.

You may not use one key for 100s of servers but it is still useful in testing in controlled environment where you end up spinning multiple identical machines for some test purposes and uses one temporary key for small period of time.

The Common Github Pattern

In .git/config file, you will see something like this as clone url or just url:

[remote "origin"]
	url = git@test-github:testaccout/somerepo.git

The interesting part in url is git@test-github. This simply means, anyone using this git config must have set something similar to the following in ~/.ssh/config file:

Host test-github
    HostName github.com
    IdentityFile /path/to/ssh_key

Since git is glready provided as user in git@test-github, it is likely User is not specified in the ssh config file. If the url has just test-github:testaccout/somerepo.git pattern, it is likely User is then set as git in the ssh config file:

Host test-github
    HostName github.com
    User git
    IdentityFile /path/to/ssh_key