rsync and SSM agent port forwarding for file transfers without private key
Tags: aws, cloud, cloudcomputing, ec2
Published: 2023-06-10 04:32:05

**THIS IS AN INTERESTING HACKING PROJECT DON'T DO THIS IN A HIGH COMPLIANCE ENVIRONMENT**

Out of curiosity I was trying to see about copying files using the SSM agent. What I found unfortunately was that you [kind of need an `authorized_keys` available pem/private key file](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-enable-ssh-connections.html) for the easy SCP route to work. At that point I felt like well that kind of defeats the purpose of the whole don't keep SSH keys laying around on your instance by using SSM. Then I came across an [interesting gist](https://gist.github.com/lukeplausin/4b412d83fb1246b0bed6507b5083b3a7) talking about how to do file transfer using `netcat`. Pondering for a moment I realized `rsync` could possibly work. Note that all commands here assume an Amazon Linux 2 system. Installation of the SSM agent may be required for other systems.

## Permissions

For all this magic to work permissions will needed to be added to the source instance transferring the file:

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ssm:StartSession",
            "Resource": [
                "arn:aws:ec2:*:[account_number]:instance/*",
                "arn:aws:ssm:*:*:document/AWS-StartPortForwardingSession"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "ssm:TerminateSession",
            "Resource": "arn:aws:ssm:*:[account_number]:session/*"
        }
    ]
}
```

The destination will also need the `arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore` managed policy (or similar policy that gives the proper permissions) as part of instance profile (I generally recommend this for easy EC2 instance management). 

## Port Forwarding

It turns out that SSM agent has a plugin for port forwarding which you can find the code for [in around this area](https://github.com/aws/amazon-ssm-agent/blob/768015398c24d187a653a4441006d97f4a0c4f1b/agent/session/plugins/port/). For this to work we first need to install the session manager plugin to start the actual session via the AWS CLI on the source machine:

```bash
$ sudo yum install -y https://s3.amazonaws.com/session-manager-downloads/plugin/latest/linux_64bit/session-manager-plugin.rpm
```

Next on the destination machine is the installation of the rsync daemon. This allows for rsync traffic to work over a TCP port instead of the usual SSH method (which again, would require potential private key access):

```bash
$ sudo yum install rsync-daemon
```

From there I worked together a simple configuration which has an `ec2-upload` directory that owns files as the ec2-user to an `uploads` path in the home directory. `read only = false` is needed to allow the uploads to work.

```ini
# /etc/rsyncd: configuration file for rsync daemon mode
[ec2-upload]
  path = /home/ec2-user/uploads
  comment = Instance Upload
  uid = ec2-user
  read only = false
```

Due to session manger console access being done through `ssm-user`, a bit of sudo was required to create the uploads folder and own it for the `ec2-user`:

```bash
$ sudo mkdir /home/ec2-user/uploads
$ sudo chown ec2-user /home/ec2-user/uploads
```

Finally it's time to start the rsyncd service:

```bash
$ sudo systemctl start rsyncd
```

Now on the source machine:

```bash
$ sudo aws ssm start-session --target [instance-id-here] --document-name AWS-StartPortForwardingSession --parameters '{"portNumber":["873"],"localPortNumber":["873"]}' &
```

`[instance-id-here]` should be replaced in entirety with the instance ID of the destination instance. Due to 873 being a restricted port root access is required via `sudo` for port binding to work. The `&` at the end sets it as a background process so other commands work (otherwise it blocks terminal interaction). You'll want to know the PID of the resulting process so you can shut it down later via `kill -15` or `kill -9`. So now that it's hopefully setup, simply run rsync against a file with `rsync://localhost/[module-here]` which is `ec2-upload` in this case. For an example here we'll go ahead and transfer the Google home page:

```bash
$ wget google.com
$ rsync index.html rsync://localhost/ec2-upload/
```

Now on the destination system:

```bash
# ls -lah /home/ec2-user/uploads/
total 20K
drwxr-xr-x. 2 ec2-user root      24 Jun 10 04:02 .
drwx------. 4 ec2-user ec2-user  89 Jun 10 03:31 ..
-rw-r--r--. 1 ec2-user nobody   19K Jun 10 03:46 index.html
```

Success! No PEM file required!

## Conclusion

Note that if you open up 873 on your security group well... anyone with access to the port will now be able to upload files with no remorse. Doing this through the SSM agent does some tunneling through the SSM service to make opening this port on the security group unnecessary (so some slight security). Though this also means you shouldn't expect insane transfer speeds due to increased network hops. IAM also enables restrictions on `StartSession` to help with too much open access. All and all there are other solutions such as EFS or an S3 proxy that might be a better option, but this could potentially be helpful for more niche cases.