How to Set Up a Raspberry Pi Cluster
By: Richard Youngkin
Tags: Kubernetes Cluster, Raspberry Pi, Containers, Programming, Kubernetes

4. Set Up Keyless SSH Access Between Hosts

When this section is completed, you’ll be able to freely ssh between the hosts in your cluster, and from selected hosts in the external network, with ease. This section is comprised of two sub-sections:

Set up keyless SSH access

Set up a reverse-SSH tunnel

Set up keyless SSH access

Some of the information in this section was taken from “How to configure passwordless login in Mac OS X and Linux.” It is very important to note that these steps are directional, i.e., they give keyless access from a specific host to a remote host — for example, from the cluster host that you are performing these steps on to a remote host.

The public key of a host’s user is copied to a remote host. During the copy process ssh-copy-id , will be required to complete the normal authentication process to the remote host. After copying has completed, the public key that was copied is now considered authenticated. For more information on how the public/private keys are used during keyless access, see “Understanding the SSH Connection and Encryption Process.”

From the host permitting access, the steps are as follows:

ssh-keygen -t rsa — This will generate a private/public key pair. If you already have a public/private key pair, you can skip this step.

The key pair is used to provide mutual authentication between the hosts involved in the SSH session. -t specifies that the generated key type is RSA.

There will be several prompts. The first will be the location/name of the generated pair. The defaults are ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub (the public key). If you change these names, substitute the name you used for ~/.ssh/id_rsa wherever these names are used.

The next prompts will be for the key’s password. Take the defaults if you don’t want to have to enter a password every time you ssh into the remote host (the point of this section is to avoid entering a password). Note: Empty passwords are considered a security risk, but in some cases, like this one, it is acceptable.

The gist below shows an example of ssh-keygen :

Creating a public/private key pair

2. ssh-copy-id -i ~/.ssh/id_rsa.pub user@remotehost — This will copy the public key, in this example id_rsa.pub , to the remote host.

In our cluster, you will be able to ssh-copy-id from a host in the cluster network to a host in the external network, but you won’t be able to ssh or ssh-copy-id from a host in the external network to a host in the cluster network yet. The next step, “Set up a reverse-ssh tunnel, enables this.

Repeat the above steps for each host in the network that you want to enable keyless access for. Note that Remote Host can also be in the cluster network.

Here’s a pictorial illustration of the above:

Set up a reverse-SSH tunnel

Having a reverse-SSH tunnel from a cluster host(s) to a host in the external network is incredibly useful, almost mandatory. I do all my cluster configuration from my MacBook. It’s possible to do this from the Pi router, but nowhere near as convenient, especially if you want multiple tmux or i2cssh windows into the cluster.

The previous step, Set up keyless SSH access, is a prerequisite for this step. You’ll have to have already created and ssh-copy-id ’d the keys from the permitting hosts to the remote host(s) on the external network. The permitting hosts in my cluster are kube-node1 through kube-node4 . On my network the host on the external network is located at 10.0.0.223 on the external network.

Some of the information below was taken from “Start AutoSSH on Boot.”

The steps are as follows:

sudo apt-get install autossh . autossh is a tool that will set up a configured reverse-ssh tunnel. autossh will also monitor the tunnel and re-establish it if it fails. This step installs autossh onto the Raspberry Pi. It’s possible to configure autossh to set up the tunnel at boot time. This is what we’ll do next. Edit your /etc/rc.local file, adding the following line (with changes as appropriate):

autossh -o StrictHostKeyChecking=no -i /home/pi/.ssh/id_rsa -fN -R <port>:localhost:22 <user>@<remotehost>

Here’s what the above line specifies:

-o flag is used to specify configuration options that have no command line analog. -o StringHostKeyChecking=no directs autossh to ignore the ~/.ssh/known_hosts file, i.e., connect even if the host isn’t in known_hosts . The See the Debian SSH_CONF and SSH main pages for more details.

flag is used to specify configuration options that have no command line analog. directs to ignore the file, i.e., connect even if the host isn’t in . The See the Debian SSH_CONF and SSH main pages for more details. -i tells autossh where to find the key file.

tells where to find the key file. -f directs autossh to run in the background.

directs to run in the background. -N tells autossh that there is no command to be run on the remote host, only to set up the tunnel.

tells that there is no command to be run on the remote host, only to set up the tunnel. -R is the forwarding specification. In the above command, <port> in <port>:localhost:22 specifies which port on the remote host to use for the tunnel. localhost:22 in <port>:localhost:22 instructs autossh to use port 22 to terminate the connection on this host.

On my cluster hosts, I use the following:

autossh -o StrictHostKeyChecking=no -i /home/pi/.ssh/id_rsa -fN -R 13003:localhost:22 rich_youngkin@10.0.0.223

Port 13003 is arbitrary. It can be any unused port on the remote host, usually above 1024 . Keep note of it, as it will be needed in a moment. Reboot the host to ensure autossh starts up as expected on boot. To verify the tunnel works as expected, on the remote host (in my case 10.0.0.223 which is my MacBook), run:

ssh pi@localhost -p 13003

-p specifies the port used by the reverse tunnel. (Note: localhost is used because we’re using the tunnel port on the current host, local host, to connect to the remote host via the tunnel port 13003 .)

You’ll be prompted for a password. What? The point of this is to be able to ssh without a password! Well, we haven’t yet copied the public key of the host in the external network to the host in the cluster network. We couldn’t do this since we didn’t have connectivity between the two networks yet. The reverse-tunnel allows us to do that. So now we can enable keyless access from the external host to the cluster network:

ssh-copy-id -i ~/.ssh/id_rsa.pub pi@localhost -p <tunnelPort>

In this section the tunnel port is 13003 , so the command will be:

ssh-copy-id -i ~/.ssh/id_rsa.pub pi@localhost -p 13003

Now ssh into the cluster host. If all went well, you won’t be prompted for a password.

ssh pi@localhost -p 13003

Repeat these steps for each cluster host from which you want to enable autossh . You’ll need to choose a different tunnel port for each cluster host (e.g., kube-node1 , kube-node2 ). This is because the host on the external network, e.g., my MacBook at 10.0.0.223 , needs to have a unique tunnel port number for each cluster host it will ssh into. In my configuration, I use ports 13000 through 13003 from my MacBook. So for example:

On kube-node1 I specify port 13000 in the autossh command. On kube-node2 , I specify port 13001 . On kube-node3 , I specify port 13002 . And so on.

One other thing I did to simplify my life was to configure my ~/.ssh/config to allow me to use shortcuts to ssh into other hosts. Specifically, it provides a shorthand for ssh 'g over a reverse tunnel. Here’s my ~/.ssh/config file:

Taking the first entry, Host master ... , here’s what each line means:

Host — This is the alias for the specification that follows.

— This is the alias for the specification that follows. Hostname — This is the IP address for the local host to connect to.

— This is the IP address for the local host to connect to. Port — This is the port number on the remote host. Typically, this is port 22, the standard ssh port and the port we specified in the autossh command above.

— This is the port number on the remote host. Typically, this is port 22, the standard port and the port we specified in the command above. User — This is the user to use for the login.

Here’s how you use it as a substitute for ssh pi@localhost -p 13004 :

ssh pi-node1

That’s it.