SSH & SCP in Python with Paramiko
Tags: python, automation, devops
Published: 2022-05-24 02:35:16

---
title: SSH & SCP in Python with Paramiko
published: true
date: 2020-01-03 08:06:00 UTC
tags: Python,Automation,DevOps
canonical_url: https://hackersandslackers.com/automate-ssh-scp-python-paramiko/
---

![SSH & SCP in Python with Paramiko](https://cdn.hackersandslackers.com/2020/11/paramiko.jpg)

Cloud providers have made a killing from neatly-packaged managed services for years. Whether it be databases or message brokers, developers like ourselves don't seem to have a problem paying a bit extra to have things taken care of. But wait, aren't we typically the _last_ people to opt for less optimization and less control? Why is this the time we decide otherwise? If I had to make a guess, I'd wager it's partly because server-side DevOps kind of sucks.

As a developer, configuring or debugging a VPS is usually work which is unaccounted for, and it isn't particularly rewarding. At best, your application will probably end up running the same as your local environment. How could we make this inevitable part of our jobs better? Well, we could automate  it.

[Paramiko](https://www.paramiko.org/) and [SCP](https://github.com/jbardin/scp.py) are two Python libraries we can use together to automate tasks we'd want to run on a remote host such as restarting services, making updates, or grabbing log files. We're going to take a look at what scripting with these libraries looks like. Fair warning: there's a sizable amount of code in this tutorial, which tends to make me excited enough to coerce others into my manic code-tutorial episodes. If you start to feel lost, the full repo can be found here:


{% github hackersandslackers/paramiko-tutorial %}


<!--kg-card-begin: html-->
 Paramiko leans heavily on the "in-the-weeds" side of Python libraries. If you're looking for something easy that can just get the job done, [pyinfra](https://pyinfra.com/) is supposedly a great (and easy) alternative. 
<!--kg-card-end: html-->

## Setting up SSH Keys

To authenticate an SSH connection, we need to set up a **private RSA SSH key**. We can generate a key using the following command:

```shell
$ ssh-keygen -t rsa
```
<figcaption>Generate an RSA key</figcaption>

This will prompt us to provide a name for our key. Name it whatever you like:

```shell
>> Generating a public/private rsa key pair.
>> Enter the file in which you wish to save they key (i.e., /home/username/.ssh/id_rsa):
```
<figcaption>RSA prompt</figcaption>

Next, you'll be prompted to provide a password (recommended, but up to you). This will result in the creation of two keys: a **private key** (which I'll refer to as `sshkey`) and a **public key** which takes the naming scheme of `sshkey.pub`:

```shell
>> Enter passphrase (empty for no passphrase):
>> Enter same passphrase again: 
>> Your identification has been saved in /Users/username/.ssh/sshkey
>> Your public key has been saved in /Users/username/.ssh/sshkey.pub
>> The key fingerprint is:
SHA256:UFYfdydftf6764576787648574 user@My-MacBook-Pro.local
>> The key's randomart image is:
+---[RSA 3072]----+
|o*-HHFTEf.       |
|++o=++.+E.       |
|OoO + ..         |
|.= + o           |
|+ . o . S        |
|.. o o .         |
|o .+ O .         |
|.+               |
|..o.             |
+----[SHA256]-----+
```

Consider the **private key** to be sacred; this key and everything about it should remain on your machine (the example I posted above is fake). The corresponding **public key** is what we put on remote hosts in our possession to authenticate a connection.

The easiest way to do this is by using `ssh-copy-id`, which is a command that exists for this exact purpose:

```shell
$ ssh-copy-id -i ~/.ssh/mykey user@example.com
```
<figcaption>Copy key to remote host</figcaption>

In the above example, `~/.ssh/mykey` is the file path of the **public key** we created. `user@example.com` should of course be replaced with the address of your remote host, where `user` is the username of the preferred user to connect with.

### Verifying our SSH Key

It's always best to check our work. Go ahead and SSH into your remote host; if you didn't set a password when creating your key, you may be surprised to find that you're no longer prompted for a password. That's a good sign.

Check your host's `.ssh` directory for the public key we created:

```shell
$ ssh user@example.com
$ cd ~/.ssh
$ ls
```
<figcaption>Check /.ssh directory</figcaption>

We're looking for keys that begin with the following header:

```shell
-----BEGIN RSA PRIVATE KEY-----

...

-----END RSA PRIVATE KEY-----
```
<figcaption>id_rsa</figcaption>

Feel free to do the same on your VPS.

## Starting our Script

Let's install our libraries. Fire up whichever virtual environment you prefer and let em rip:

```shell
$ pip3 install paramiko scp
```
<figcaption>Install paramiko & scp</figcaption>

Just one more thing before we write some meaningful Python code! Create a config file to hold the variables we'll need to connect to our host. Here are the barebones of what we need to get into our server:

- **Host**: The IP address or URL of the remote host we're trying to access.
- **Username**: This is the username you use to SSH into your server.
- **Passphrase** (optional): If you specified a passphrase when you created your ssh key, specify that here. Remember that your SSH key passphrase is _not_ the same as your user's password.
- **SSH Key**: The file path of the key we created earlier. On OSX, these live in your system's **~/.ssh** folder. SSH key we're targeting must have an accompanying key with a **.pub** file extension. This is our public key; if you were following along earlier, this should have already been generated for you.

If you're trying to upload or download files from your remote host, you'll need to include two more variables:

- **Remote Path**: The path to the remote directory we're looking to target for file transfers. We can either upload things to this folder or download the contents of it.
- **Local Path**: Same idea as above, but the reverse. For our convenience, the local path we'll be using is simply **/data**, and contains pictures of cute fox gifs.

Now we have everything we need to make a respectable **config.py** file:

```python
"""Remote host configuration."""
from os import getenv, path
from dotenv import load_dotenv

# Load environment variables from .env
BASE_DIR = path.abspath(path.dirname( __file__ ))
load_dotenv(path.join(BASE_DIR, ".env"))

# SSH Connection Variables
ENVIRONMENT = getenv("ENVIRONMENT")
SSH_REMOTE_HOST = getenv("SSH_REMOTE_HOST")
SSH_USERNAME = getenv("SSH_USERNAME")
SSH_PASSWORD = getenv("SSH_PASSWORD")
SSH_KEY_FILEPATH = getenv("SSH_KEY_FILEPATH")
SCP_DESTINATION_FOLDER = getenv("SCP_DESTINATION_FOLDER")

# Local file directory
LOCAL_FILE_DIRECTORY = f"{BASE_DIR}/files"
```
<figcaption>config.py</figcaption>

## Creating an SSH Client

We're going to create a class called `RemoteClient` to handle the interactions we'll be having with our remote host. Before we get too fancy, let's just start things off by instantiating the `RemoteClient` class with the variables we created in **config.py** :

```python
"""Client to handle connections and actions executed against a remote host."""

class RemoteClient:
    """Client to interact with a remote host via SSH & SCP."""

    def __init__ (
        self,
        host: str,
        user: str,
        password: str,
        ssh_key_filepath: str,
        remote_path: str,
    ):
        self.host = host
        self.user = user
        self.password = password
        self.ssh_key_filepath = ssh_key_filepath
        self.remote_path = remote_path
        self.client = None
        self._upload_ssh_key()
```
<figcaption>server.py</figcaption>

You'll notice I added a few things to our constructor, besides the config values we pass in:

- `self.client`: **self.client** will ultimately serve as the connection objection in our class, similar to how you have dealt with terminology like `conn` in database libraries. Our connection will be `None` until we explicitly connect to our remote host.
- `self._upload_ssh_key()` isn't a variable, but rather a function to be run automatically whenever our client is instantiated. Calling `_upload_ssh_key()` is telling our **`RemoteClient`** object to check for local ssh keys immediately upon creation so that we can try to pass them to our remote host. Otherwise, we wouldn't be able to establish a connection at all.

## Uploading SSH Keys to a Remote Host

We've reached the section of this exercise where we need to knock out some devastatingly inglorious boilerplate code. This is typically where emotionally inferior individuals succumb to the sheer dull obscurity of understanding SSH keys and maintaining connections. Make no mistake: authenticating and managing connections to _anything_ programmatically is overwhelmingly dull... unless your tour guide happens to be an enchanting wordsmith, serving as your loving protector through perilous obscurity. Some people call this post a _tutorial_. I intend to call it _art_.

**`RemoteClient`** will start with two private methods: `_get_ssh_key()`  and `_upload_ssh_key()`. The former will fetch a locally stored public key, and if successful, the latter will deliver this public key to our remote host as an olive branch of access. Once a locally created public key exists on a remote machine, that machine will then forever trust us with our requests to connect to it: no passwords required. We'll be including proper logging along the way, just in case we run into any trouble:

```python
"""Client to handle connections and actions executed against a remote host."""
from os import system
from paramiko import SSHClient, AutoAddPolicy, RSAKey
from paramiko.auth_handler import AuthenticationException, SSHException
from scp import SCPClient, SCPException
from .log import logger

class RemoteClient:
    """Client to interact with a remote host via SSH & SCP."""

    ...

    def _get_ssh_key(self):
        """Fetch locally stored SSH key."""
        try:
            self.ssh_key = RSAKey.from_private_key_file(
                self.ssh_key_filepath
            )
            LOGGER.info(
                f"Found SSH key at self {self.ssh_key_filepath}"
            )
            return self.ssh_key
        except SSHException as e:
            LOGGER.error(f"SSHException while getting SSH key: {e}")
        except Exception as e:
            LOGGER.error(f"Unexpected error while getting SSH key: {e}")

   def _upload_ssh_key(self):
        try:
            system(
                f"ssh-copy-id -i {self.ssh_key_filepath}.pub {self.user}@{self.host}>/dev/null 2>&1"
            )
            LOGGER.info(
                f"{self.ssh_key_filepath} uploaded to {self.host}"
            )
        except FileNotFoundError as e:
            LOGGER.error(
                f"FileNotFoundError while uploading SSH key: {e}"
            )
        except Exception as e:
            LOGGER.error(
                f"Unexpected error while uploading SSH key: {e}"
            )
```
<figcaption>server.py</figcaption>

`_get_ssh_key()` is quite simple: it verifies that an SSH key exists at the path we specified in our config to be used for connecting to our host. If the file does in fact exist, we happily set our `self.ssh_key` variable, so this key can be uploaded and used by our client from here forward. Paramiko provides us with a submodule called `RSAKey` to easily handle all things RSA key related, like _parsing a private key file into a usable connection authentication._ That's what we get here:

```python
RSAKey.from_private_key_file(self.ssh_key_filepath)
```
<figcaption>Read RSA key from local file</figcaption>

If our RSA key were incomprehensible nonsense instead of a real key, Paramiko's `SSHException` would have caught this and raised an exception early on explaining just that. Properly utilizing a library's error handling takes a lot of the guesswork out of "what went wrong," especially in cases like where there's potential for numerous unknowns in a niche space neither of us mess with often.

`_upload_ssh_key()` is where we get to jam our SSH key down the throat of our remote server while shouting, "LOOK! YOU CAN TRUST ME FOREVER NOW!" To accomplish this, I go a bit "old school" by passing bash commands via Python's `os.system`. Unless somebody makes me aware of a cleaner approach in the comments, I'll assume this is the most badass way to handle passing keys to a remote server.

The standard non-Python way of passing keys to a host looks like this:

```shell
ssh-copy-id -i ~/.ssh/mykey user@host
```
<figcaption>Pass SSH key to remote host</figcaption>

This is precisely what we accomplish in our function in Python, which looks like this:

```python
system(
    f"ssh-copy-id -i {self.ssh_key_filepath} \
       {self.user}@{self.host}>/dev/null 2>&1"
 )
```

I suppose you won't let me slip that `/dev/null 2>&1` bit by you? Fine. If you _must_ know, here's some guy on StackOverflow explaining it better than I can:

> `>` is for redirect `/dev/null` is a black hole where any data sent, will be discarded. `2` is the file descriptor for Standard Error. `>` is for redirect. `&` is the symbol for file descriptor (without it, the following `1` would be considered a filename). `1` is the file descriptor for Standard O.

So we're basically telling our remote server we're giving it something, and it's all like "where do I put this thing," to which we reply "nowhere in physical in space, as this is not an _object_, but rather an _eternal symbol of our friendship_.  Our remote host is then flooded with gratitude and emotion, because yes, computers _do_ have emotions, but we **can't be bothered by that right now**.

### Connecting to our Client

We'll add a method to our client called `connect()` to handle connecting to our host:

```python
...

class RemoteClient:
    """Client to interact with a remote host via SSH & SCP."""

    ...

    @property
    def connection(self):
        """Open connection to remote host. """
        try:
            client = SSHClient()
            client.load_system_host_keys()
            client.set_missing_host_key_policy(AutoAddPolicy())
            client.connect(
                self.host,
                username=self.user,
                password=self.password,
                key_filename=self.ssh_key_filepath,
                timeout=5000,
            )
            return client
        except AuthenticationException as e:
            LOGGER.error(
                f"AuthenticationException occurred; did you remember to generate an SSH key? {e}"
            )
        except Exception as e:
            LOGGER.error(
                f"Unexpected error occurred while connecting to host: {e}"
            )
```
<figcaption>server.py</figcaption>

Let's break this down:

- `client = SSHClient()` sets the stage for creating an object representing our SSH client. The following lines will configure this object to make it more useful.
- `load_system_host_keys()` instructs our client to look for all the hosts we've connected to in the past by looking at our system's **known\_hosts** file and finding the SSH keys our host is expecting. We've never connected to our host in the past, so we need to specify our SSH key explicitly.
- `set_missing_host_key_policy()` tells Paramiko what to do in the event of an unknown key pair. This is expecting a "policy" built-in to Paramiko, to which we're going to specific `AutoAddPolicy()`. Setting our policy to "auto-add" means that if we attempt to connect to an unrecognized host, Paramiko will automatically add the missing key locally.
- `connect()` is **SSHClient**'s most important method (as you might imagine). We're finally able to pass our **host** , **user** , and **SSH key** to achieve what we've all been waiting for: a glorious SSH connection into our server! The `connect()` method allows for a ton of flexibility via a vast array of optional keyword arguments too. I happen to pass a few here: setting **look\_for\_keys** to `True` gives Paramiko permission to look around in our **~/.ssh** folder to discover SSH keys on its own, and setting **timeout** will automatically close connections we'll probably forget to close. We could even pass variables for things like **port** and **password** , if we had elected to connect to our host this way.

### Disconnecting

We should close connections to our remote host whenever we're done using them. Failing to do so might not necessarily be _disastrous_, but I've had a few instances where enough hanging connections would eventually max out inbound traffic on port 22. Regardless of whether your use case might consider a reboot to be a disaster or mild inconvenience, let's just close our damn connections like adults as though we were wiping our butts after pooping. No matter your connection hygiene, I advocate for setting a **timeout** variable (as we saw earlier). Anyway. voila:

```python
class RemoteClient:
    ...

    def disconnect(self):
        """Close ssh connection."""
        if self.client:
            self.client.close()
        if self.scp:
            self.scp.close()
```
<figcaption>server.py</figcaption>

Fun fact: setting `self.client.close()` actually sets `self.client` to equal `None`, which is useful in cases where you might want to check if a connection is already open.

## Executing Unix Commands

We now have a wonderful Python class that can find RSA keys, connect, and disconnect. It _does_ lack the ability to do, well, anything useful.

We can fix this and finally begin doing "stuff" with a brand new method to execute commands, which I'll aptly dub `execute_commands()` (that's correct, "commands" as in potentially-more-than-one, we'll touch on that in a moment). The legwork of all this is done by the Paramiko client's built-in `exec_command()` method, which accepts a single string as a command and executes it:

```python
...

class RemoteClient:
    ...

    def execute_commands(self, commands: List[str]):
        """
        Execute multiple commands in succession.

        :param List[str] commands: List of unix commands as strings.
        """
        for cmd in commands:
            stdin, stdout, stderr = self.connection.exec_command(cmd)
            stdout.channel.recv_exit_status()
            response = stdout.readlines()
            for line in response:
                LOGGER.info(
                    f"INPUT: {cmd}\n \
                    OUTPUT: {line}"
                )
```
<figcaption>server.py</figcaption>

The function we just created `execute_commands()` expects a **list of strings** to execute as commands. That's partially for convenience, but it's also because Paramiko won't run any "state" changes (like changing directories) between commands, so each command we pass to Paramiko should assume we're working out of our server's root. I took the liberty of passing three such commands like so:

```python
...

def execute_command_on_remote(remote):
    """Execute UNIX command on the remote host."""
    remote.execute_commands(
        [
            "cd /var/www/ && ls",
            "tail /var/log/nginx/access.log",
            "ps aux | grep node",
            "cd /uploads/ && ls",
        ]
    )
```
<figcaption>__init__.py</figcaption>

I can view the contents of a directory by chaining `cd path/to/dir && ls`, but running `cd path/to/dir` followed by `ls` would result in nothingness because `ls` the second time returns the list of files in our server's root.

You'll notice `client.exec_command(cmd)` returns three values as opposed to one: this can be useful to see which _input_ produced which _output_. For example, here are the full logs for the example I provided where I passed three commands to `remote.execute_commands()`:

```shell
2020-01-02 23:20:16.103 | paramiko_tutorial.client:execute_cmd:95 - INPUT: cd /var/www/ && ls | OUTPUT: django
2020-01-02 23:20:16.103 | paramiko_tutorial.client:execute_cmd:95 - INPUT: cd /var/www/ && ls | OUTPUT: ghost
2020-01-02 23:20:16.103 | paramiko_tutorial.client:execute_cmd:95 - INPUT: cd /var/www/ && ls | OUTPUT: hackers-hbs
2020-01-02 23:20:16.103 | paramiko_tutorial.client:execute_cmd:95 - INPUT: cd /var/www/ && ls | OUTPUT: html
2020-01-02 23:20:16.104 | paramiko_tutorial.client:execute_cmd:95 - INPUT: cd /var/www/ && ls | OUTPUT: hustlers
2020-01-02 23:20:16.104 | paramiko_tutorial.client:execute_cmd:95 - INPUT: cd /var/www/ && ls | OUTPUT: pizza
2020-01-02 23:20:16.104 | paramiko_tutorial.client:execute_cmd:95 - INPUT: cd /var/www/ && ls | OUTPUT: toddbirchard

2020-01-02 23:20:16.196 | paramiko_tutorial.client:execute_cmd:95 - INPUT: tail /var/log/nginx/access.log | OUTPUT: 134.209.37.49 - - [03/Jan/2020:03:42:34 +0000] "GET / HTTP/2.0" 404 139 "-" "Mozilla/5.0 (compatible; NetcraftSurv
eyAgent/1.0; +info@netcraft.com)"
2020-01-02 23:20:16.196 | paramiko_tutorial.client:execute_cmd:95 - INPUT: tail /var/log/nginx/access.log | OUTPUT: 54.36.148.187 - - [03/Jan/2020:03:43:49 +0000] "GET /robots.txt HTTP/1.1" 404 149 "-" "Mozilla/5.0 (compatible; Ah
refsBot/6.1; +https://ahrefs.com/robot/)"
2020-01-02 23:20:16.196 | paramiko_tutorial.client:execute_cmd:95 - INPUT: tail /var/log/nginx/access.log | OUTPUT: 54.36.150.104 - - [03/Jan/2020:03:43:50 +0000] "GET / HTTP/1.1" 404 139 "-" "Mozilla/5.0 (compatible; AhrefsBot/6.
1; +https://ahrefs.com/robot/)"
2020-01-02 23:20:16.196 | paramiko_tutorial.client:execute_cmd:95 - INPUT: tail /var/log/nginx/access.log | OUTPUT: 46.229.168.146 - - [03/Jan/2020:03:48:49 +0000] "GET /robots.txt HTTP/1.1" 200 92 "-" "Mozilla/5.0 (compatible; Se
mrushBot/6~bl; +https://www.semrush.com/bot.html)"
2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT: tail /var/log/nginx/access.log | OUTPUT: 46.229.168.153 - - [03/Jan/2020:03:48:50 +0000] "GET /the-art-of-technical-documentation/ HTTP/1.1" 200 9472 "-" "
Mozilla/5.0 (compatible; SemrushBot/6~bl; +https://www.semrush.com/bot.html)"
2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT: tail /var/log/nginx/access.log | OUTPUT: 157.55.39.171 - - [03/Jan/2020:03:50:20 +0000] "GET /robots.txt HTTP/1.1" 200 94 "-" "Mozilla/5.0 (compatible; bin
gbot/2.0; +https://www.bing.com/bingbot.htm)"
2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT: tail /var/log/nginx/access.log | OUTPUT: 40.77.167.220 - - [03/Jan/2020:03:50:24 +0000] "GET / HTTP/1.1" 200 3791 "-" "Mozilla/5.0 (compatible; bingbot/2.0
; +https://www.bing.com/bingbot.htm)"
2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT: tail /var/log/nginx/access.log | OUTPUT: 54.36.150.64 - - [03/Jan/2020:03:56:47 +0000] "GET /the-ruin-of-feeling-powerless/ HTTP/2.0" 200 9605 "-" "Mozilla
/5.0 (compatible; AhrefsBot/6.1; +https://ahrefs.com/robot/)"
2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT: tail /var/log/nginx/access.log | OUTPUT: 54.36.150.187 - - [03/Jan/2020:04:05:01 +0000] "GET /bigquery-and-sql-databases/ HTTP/2.0" 404 146 "-" "Mozilla/5.
0 (compatible; AhrefsBot/6.1; +https://ahrefs.com/robot/)"
2020-01-02 23:20:16.197 | paramiko_tutorial.client:execute_cmd:95 - INPUT: tail /var/log/nginx/access.log | OUTPUT: 46.229.168.149 - - [03/Jan/2020:04:13:41 +0000] "GET /robots.txt HTTP/1.1" 502 182 "-" "Mozilla/5.0 (compatible; S
emrushBot/6~bl; +https://www.semrush.com/bot.html)"

2020-01-02 23:20:16.322 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux | grep node | OUTPUT: ghost 1354 39.2 3.4 1223844 140172 ? Sl 04:20 0:05 /usr/bin/node current/index.js
2020-01-02 23:20:16.323 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux | grep node | OUTPUT: ghost 1375 36.8 3.3 1217696 135548 ? Sl 04:20 0:04 /usr/bin/node current/index.js
2020-01-02 23:20:16.323 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux | grep node | OUTPUT: ghost 1395 46.8 3.6 1229824 147384 ? Sl 04:20 0:06 /usr/bin/node current/index.js
2020-01-02 23:20:16.323 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux | grep node | OUTPUT: ghost 1410 37.7 3.2 1216320 132912 ? Sl 04:20 0:04 /usr/bin/node current/index.js
2020-01-02 23:20:16.323 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux | grep node | OUTPUT: root 1848 0.0 0.0 13312 3164 ? Ss 04:20 0:00 bash -c ps aux | grep node
2020-01-02 23:20:16.323 | paramiko_tutorial.client:execute_cmd:95 - INPUT: ps aux | grep node | OUTPUT: root 1850 0.0 0.0 14856 1104 ? S 04:20 0:00 grep node
```
<figcaption>Output</figcaption>

Some beautiful stuff here. Now you can see which sites are on my server, which bots are spamming me, and how many node processes I'm running.

I don't want to waste much more time on the art of executing commands, but it's worth mentioning the presence of why we call `stdout.channel.recv_exit_status()` after each command. Waiting for `recv_exit_status()` to come back after running `client.exec_command()` forces our commands to be run _synchronously,_ otherwise there's a likely chance our remote machine won't be about to decipher commands as fast as we pass them along.

## Uploading (and Downloading) Files via SCP

SCP refers to both the protocol for copying files to remote machines (secure copy protocol) as well as the Python library, which utilizes this. We've already installed the SCP library, so import that shit.

The **SCP** and **Paramiko** libraries complement one another to make uploading via SCP super easy. **`SCPClient()`** creates an object which expects  "transport" from Paramiko, which we provide with `self.conn.get_transport()`. Creating an SCP connection piggybacks off of our SSH client in terms of syntax, but these connections are _separate_. It's possible to close an SSH connection and leave an SCP connection open, so don't do that. Open an SCP connection like this:

```shell
self.scp = SCPClient(self.client.get_transport())
```
<figcaption>Open an SCP connection</figcaption>

Uploading a single file is boring, so let's upload an entire directory of files instead. `bulk_upload()` accepts a list of file paths:

```python
class RemoteClient:
    ...

    def bulk_upload(self, filepaths: List[str]):
        """
        Upload multiple files to a remote directory.

        :param List[str] filepaths: List of local files to be uploaded.
        """
        try:
            self.scp.put(
                filepaths,
                remote_path=self.remote_path,
                recursive=True
            )
            LOGGER.info(
                f"Finished uploading {len(filepaths)} files to {self.remote_path} on {self.host}"
            )
        except SCPException as e:
            LOGGER.error(
                f"SCPException during bulk upload: {e}"
            )
        except Exception as e:
            LOGGER.error(
                f"Unexpected exception during bulk upload: {e}"
            )
```
<figcaption>server.py</figcaption>

Our method is expecting to receive a list of strings, each representing the local path to a file, we'd like to upload.

SCP's `put()` method will upload any number of files to a remote host. This will replace existing files with the same name if they happen to exist at the destination we specify. That's all it takes!

### Downloading Files

The counterpart to SCP's `put()` is the `get()` method:

```python
class RemoteClient:

    ...

    def download_file(self, file: str):
        """Download file from remote host."""
        self.scp.get(file)
```
<figcaption>server.py</figcaption>

## Our Big Beautiful Script

We now have a sick Python class to handle SSH and SCP with a remote host... let's put it to work! The following snippet is a quick way to test what we've built so far. In short, this script looks for a local folder filled with files (in my case, I filled the folder with fox gifs 🦊).

Check out how easy it is to create a **main.py** that handles complex tasks on remote machines thanks to our **`RemoteClient`** class:

```python
"""Perform tasks against a remote host."""
from config import (
    host,
    local_file_directory,
    password,
    remote_path,
    ssh_key_filepath,
    username,
)

from .client import RemoteClient
from .files import fetch_local_files

def run():
    """Initialize remote host client and execute actions."""
    client = RemoteClient(
        SSH_REMOTE_HOST,
        SSH_USERNAME,
        SSH_PASSWORD,
        SSH_KEY_FILEPATH,
        SCP_DESTINATION_FOLDER,
    )
    upload_files_to_remote(client)
    execute_command_on_remote(
        client,
        [
            "mkdir /uploads",
            "cd /uploads/ && ls",
        ],
    )

def upload_files_to_remote(client: RemoteClient):
    """
    Upload files to remote via SCP.

    :param RemoteClient client: Remote server client.
    """
    local_files = fetch_local_files(LOCAL_FILE_DIRECTORY)
    client.bulk_upload(local_files)

def execute_command_on_remote(client: RemoteClient, commands: List[str]):
    """
    Execute a UNIX command remotely on a given host.

    :param RemoteClient client: Remote server client.
    :param List[str] commands: List of commands to run on remote host.
    """
    client.execute_commands(commands)

```
<figcaption>__init__.py</figcaption>

Here's the output of the our upload function:

```shell
2020-01-03 00:00:27.215 | paramiko_tutorial.client:__upload_single_file:85 - Uploaded data/fox1.gif to /uploads/
2020-01-03 00:00:27.985 | paramiko_tutorial.client:__upload_single_file:85 - Uploaded data/fox2.gif to /uploads/
2020-01-03 00:00:30.015 | paramiko_tutorial.client:__upload_single_file:85 - Uploaded data/fox3.gif to /uploads/
2020-01-03 00:00:30.015 | paramiko_tutorial.client:bulk_upload:73 - Finished uploading 3 files to /uploads/ on 149.433.117.1425
```
<figcaption>Foxes uploaded successfully</figcaption>

It worked! Don't believe me? Why don't we check for ourselves by running `remote.execute_commands(['cd /var/www/ && ls'])`?

```shell
2020-01-03 00:08:55.955 | paramiko_tutorial.client:execute_commands:96 - INPUT: cd /uploads/ && ls | OUTPUT: fox1.gif
2020-01-03 00:08:55.955 | paramiko_tutorial.client:execute_commands:96 - INPUT: cd /uploads/ && ls | OUTPUT: fox2.gif
2020-01-03 00:08:55.956 | paramiko_tutorial.client:execute_commands:96 - INPUT: cd /uploads/ && ls | OUTPUT: fox3.gif
```
<figcaption>Output of cd /var/www/ && ls</figcaption>

There you have it. Straight from the fox's mouth.

## Take It And Run With It

This is where I'd like to take a moment to thank all of you, and apologize that you're still here. I swore an oath to myself to stop posting tutorials over two thousand words long, and this one is looking to push five thousand words of nonsense. I'll work on that. New year, new me.

For your convenience, I've uploaded the source for this tutorial to [Github](https://github.com/hackersandslackers/paramiko-tutoral). Feel free to take this and run with it! To close things out, I'll leave you with the meat and potatoes of the **`RemoteClient`** class we put together:

```python
"""Client to handle connections and actions executed against a remote host."""
from os import system
from typing import List

from paramiko import AutoAddPolicy, RSAKey, SSHClient
from paramiko.auth_handler import AuthenticationException, SSHException
from scp import SCPClient, SCPException

from log import LOGGER

class RemoteClient:
    """Client to interact with a remote host via SSH & SCP."""

    def __init__ (
        self,
        host: str,
        user: str,
        password: str,
        ssh_key_filepath: str,
        remote_path: str,
    ):
        self.host = host
        self.user = user
        self.password = password
        self.ssh_key_filepath = ssh_key_filepath
        self.remote_path = remote_path
        self.client = None
        self._upload_ssh_key()

    @property
    def connection(self):
        """Open SSH connection to remote host."""
        try:
            client = SSHClient()
            client.load_system_host_keys()
            client.set_missing_host_key_policy(AutoAddPolicy())
            client.connect(
                self.host,
                username=self.user,
                password=self.password,
                key_filename=self.ssh_key_filepath,
                timeout=5000,
            )
            return client
        except AuthenticationException as e:
            LOGGER.error(
                f"AuthenticationException occurred; did you remember to generate an SSH key? {e}"
            )
        except Exception as e:
            LOGGER.error(f"Unexpected error occurred while connecting to host: {e}")

    @property
    def scp(self) -> SCPClient:
        conn = self.connection
        return SCPClient(conn.get_transport())

    def _get_ssh_key(self):
        """Fetch locally stored SSH key."""
        try:
            self.ssh_key = RSAKey.from_private_key_file(self.ssh_key_filepath)
            LOGGER.info(f"Found SSH key at self {self.ssh_key_filepath}")
            return self.ssh_key
        except SSHException as e:
            LOGGER.error(f"SSHException while getting SSH key: {e}")
        except Exception as e:
            LOGGER.error(f"Unexpected error while getting SSH key: {e}")

    def _upload_ssh_key(self):
        try:
            system(
                f"ssh-copy-id -i {self.ssh_key_filepath}.pub {self.user}@{self.host}>/dev/null 2>&1"
            )
            LOGGER.info(f"{self.ssh_key_filepath} uploaded to {self.host}")
        except FileNotFoundError as e:
            LOGGER.error(f"FileNotFoundError while uploading SSH key: {e}")
        except Exception as e:
            LOGGER.error(f"Unexpected error while uploading SSH key: {e}")

    def disconnect(self):
        """Close SSH & SCP connection."""
        if self.connection:
            self.client.close()
        if self.scp:
            self.scp.close()

    def bulk_upload(self, filepaths: List[str]):
        """
        Upload multiple files to a remote directory.

        :param List[str] filepaths: List of local files to be uploaded.
        """
        try:
            self.scp.put(filepaths, remote_path=self.remote_path, recursive=True)
            LOGGER.info(
                f"Finished uploading {len(filepaths)} files to {self.remote_path} on {self.host}"
            )
        except SCPException as e:
            LOGGER.error(f"SCPException during bulk upload: {e}")
        except Exception as e:
            LOGGER.error(f"Unexpected exception during bulk upload: {e}")

    def download_file(self, filepath: str):
        """
        Download file from remote host.

        :param str filepath: Path to file hosted on remote server to fetch.
        """
        self.scp.get(filepath)

    def execute_commands(self, commands: List[str]):
        """
        Execute multiple commands in succession.

        :param List[str] commands: List of unix commands as strings.
        """
        for cmd in commands:
            stdin, stdout, stderr = self.connection.exec_command(cmd)
            stdout.channel.recv_exit_status()
            response = stdout.readlines()
            for line in response:
                LOGGER.info(
                    f"INPUT: {cmd}\n \
                    OUTPUT: {line}"
                )
```
<figcaption>server.py</figcaption>

The full source code for this tutorial can be found here:

{% github hackersandslackers/paramiko-tutorial %}