AWS Transfer familyを使ってみた

# What is AWS Transfer family?

https://aws.amazon.com/aws-transfer-family/?nc1=h_ls&whats-new-cards.sort-by=item.additionalFields.postDateTime&whats-new-cards.sort-order=desc

> The AWS Transfer Family provides fully managed support for file transfers directly into and out of Amazon S3 or Amazon EFS. With support for Secure File Transfer Protocol (SFTP), File Transfer Protocol over SSL (FTPS), and File Transfer Protocol (FTP), the AWS Transfer Family helps you seamlessly migrate your file transfer workflows to AWS by integrating with existing authentication systems, and providing DNS routing with Amazon Route 53 so nothing changes for your customers and partners, or their applications. With your data in Amazon S3 or Amazon EFS, you can use it with AWS services for processing, analytics, machine learning, archiving, as well as home directories and developer tools. Getting started with the AWS Transfer Family is easy; there is no infrastructure to buy and set up.

* It can be used to transfer files into and out of S3 and EFS
* SFTP, FTPS, and FTP protocols are supported

# How do we use it?

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/32d054d7-b381-376a-5c75-acca07720347.png)


# Demo

## Visit https://ap-northeast-1.console.aws.amazon.com/transfer/home?region=ap-northeast-1#/

→ Click **Create server**

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/8759f604-b60d-3270-0c95-bd4aee1cc2d3.png)


## Set up configurations

### Choose protocols

We can select one of the following options

* SFTP (**S**SH **F**ile **T**ransfer **P**rotocol)
* FTPS (**F**ile **T**ransfer **P**rotocol **S**ecure)
* FTP (**F**ile **T**ransfer **P**rotocol)

Check **SFTP (SSH File Transfer Protocol) - file transfer over Secure Shell** and click **Next**

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/b935c900-77c3-a447-6c48-a284208f71f4.png)


### Choose an identity provider

We can select one of the following protocols

* Service managed
* Custom

Check **Service managed** and click **Next**

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/78e58107-e89e-a378-30f6-f293a84b7600.png)

### Choose an endpoint

We need to configure endpoint type and custom hostname. We have the following options for an endpoint.

* Publicly accessible
* VPC hosted

Check **Publicly accessible** and click **Next**

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/9a06159a-9a69-d224-9554-1bd0bbc886f9.png)


### Choose a domain

We can use S3 and EFS as the destination or starting point of transfer of files. Click **Amazon S3** and click **Next**.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/641e03fe-e8e6-e4dc-1eb3-88a542bc5921.png)

### Configure additional details

We can attach an existing role or create a new role to allow the transfer family to put logs in CloudWatch Logs. Choose to **Create a new role** and click **Next**
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/1b081b6f-a475-c90e-df6f-2af5026833a0.png)

### Review and create

Click **Create server** below.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/dce1389f-19a3-2f76-b792-6cdcfc3e3115.png)


## Check on console
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/4bd0506a-900d-0c9f-d85c-042adea4bcce.png)


## Add a service managed user

Click **Server ID** of the server that you want to add a user to. Click **Add user**.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/c88b6391-931c-d85e-d6de-13c7623af7bf.png)


### User configuration

Attach to the role `AWSTransferLoggingAccess`
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/87f34075-b4ff-2a2b-aa63-d3dee322538d.png)


Input Username `transfer-family-test`, choose the role `AWSTransferLoggingAccess`. Then, choose S3 bucket where the files are transferred and 
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/307f3406-37e2-b7f3-49c7-551f7a00f0e6.png)


Generate SSH key

```bash
> ssh-keygen -P "" -m PEM -f ~/.ssh/transfer-family-test
Generating public/private rsa key pair.
Your identification has been saved in /Users/Kanata-Miyahana/.ssh/transfer-family-test.
Your public key has been saved in /Users/Kanata-Miyahana/.ssh/transfer-family-test.pub.
The key fingerprint is:
SHA256:zsBLQFnZn7PL+Q8ioX68yge7Nqo2lpCAgE4VcvDzM5A kanata-miyahana@ip-192-168-3-2.ap-northeast-1.compute.internal
The key's randomart image is:
+---[RSA 3072]----+
|.oo+oo.o         |
|o.+.o . .        |
|=  E.    . .     |
|o.  +o    +      |
|..   ++ S  o     |
|o    .+* ..      |
| . .  o++..o.    |
|  =  o+ +.+. .   |
| o.o.o=*.. ....  |
+----[SHA256]-----+

> cat ~/.ssh/transfer-family-test.pub | pbcopy
```

enter the public SSH key and click **Add**.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/f3f53b35-969c-6cec-7ee1-55c3b2b05588.png)


※ You can follow the instruction here to create an SSH public key.

## Confirm that the user is successfully added

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/8211ab01-686d-30d4-caeb-3dfefc47d5e9.png)


## Install client application

### Install Cyberduck

```bash
> brew install cyberduck --cask
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 3 taps (homebrew/cask-versions, homebrew/core and homebrew/cask).
==> New Formulae
snowpack
==> Updated Formulae
Updated 28 formulae.
==> New Casks
devbook                                 goldenpassport                          netnewswire-beta
==> Updated Casks
Updated 26 casks.

==> Downloading https://update.cyberduck.io/Cyberduck-7.8.2.34203.zip
==> Downloading from https://a48823c7ec3cf4539564-60c534a1284a12ce74ef84032e9b4e46.ssl.cf1.rackcdn.com/Cyberduck-7.8.2.
######################################################################## 100.0%
==> Installing Cask cyberduck
==> Moving App 'Cyberduck.app' to '/Applications/Cyberduck.app'
🍺  cyberduck was successfully installed!
```

### Set up configurations

If Cyberduck is successfully installed, open the application. Click + button.

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/0f6f8c9f-e724-35d7-b432-110ca090e705.png)


Set the following configuration

* SFTP
* endpoint → you can get it on AWS console

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/bbaf770e-6312-62e6-8769-d9e8987b50bc.png)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/6d0ff860-3664-2f4d-81e7-1fcf20f4b218.png)


## Test connection

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/42371179-1bee-4a7f-a08b-11c2cad26e41.png)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/f4331ace-d12a-b1e2-c8d1-5d2d6aac06ab.png)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/36e296ea-19b6-5589-09b2-528664658f5e.png)


Check that there is no file on S3 

```bash
> aws s3 ls s3://transfer-family-test-miyahana-local
# There is nothing because we have never transferred any files
```

Open directory where the file you want to transfer locates, drag, and drop the file. You can copy the file from local to S3!!!

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/b79c10f5-825e-9132-4497-92d239a20349.png)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/349999/caf71194-44c6-7188-3efa-932dea261954.png)


Check S3

```bash
> aws s3 ls s3://transfer-family-test-miyahana-local/
2021-03-09 00:04:58         12 test.txt

> aws s3 cp s3://transfer-family-test-miyahana-local/test.txt .
download: s3://transfer-family-test-miyahana-local/test.txt to ./test.txt

> cat test.txt
hello world
```

### Clean up

* Delete the server
* Delete ssh key pairs
* Delete the bucket

You can follow instructions