Connect your Fly.io app to AWS RDS Postgres. Terraform complete walkthrough.
Tags: fly, postgres, aws, elixir
Published: 2024-02-12 18:22:14

I built [choremate.co](choremate.co) with Phoenix and Elixir and the preferred PaaS for the stack nowadays is [Fly.io](fly.io), great CLI, and generous free tier. (Bonus, the creator of the Phoenix Framework works there.)

While their hosting and deployment experience is great, they do not offer a managed database option, so I had to look elsewhere: the good 'ol AWS RDS (Postgres). 

I documented in this post all the necessary steps to use Terraform to create a Postgres instance on AWS and run an EC2 host with PGBouncer and Wireguard to connect your Fly.io app to AWS VPN through a tunnel.

This guide starts from this (repo)[https://github.com/fly-apps/rds-connector?tab=readme-ov-file] which at this point is outdated in some of their parts but still a very good starting point.

Overall we will be creating a Postgres instance not publicly accessible, an ec2 instance that will work as a bastion with PgBouncer functioning as a proxy and WireGuard installed so we'll be able to establish a tunnel between the Fly.io private network and our Aws private network.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5rzkjpqe53njgqljj4w7.png)*Should look something like this*

This guide will also make a couple of assumptions:
- you already have a web application running on Fly.io and an - you already have an active account on AWS
- you are working on a Mac Silicon machine (mostly impacts how you will install dependencies)

**Required dependencies**
```
brew install terraform
brew install awscli
```

**Step 0.5 - AWS local Authentication** 

Not required but strongly suggested https://github.com/99designs/aws-vault allows you to manage your AWS credential on your local machine.

The instructions are pretty straightforward, but in case you need them, here is the [link](https://us-east-1.console.aws.amazon.com/iam/home?region=us-east-1#/security_credentials/access-key-wizard) is where you create you can quickly create a root access key (I know you should use dedicated identity but I can't cover everything here)

**Step 1 - Create SSH key-pair to use in our VPC**

We'll reference those keys in our Terraform file so they will be used inside our VPC and will use it to ssh into our EC2 instance.

```
❯ ssh-keygen -b 4096 -t rsa

Generating public/private rsa key pair.
Enter file in which to save the key (/Users/me/.ssh/id_rsa): aws_terraform
Enter passphrase (empty for no passphrase): #nothing here

❯ ls ~/.ssh
aws_terraform           aws_terraform.pub       config 
```

**Step 2 - Generate the Wireguard configuration**

Wireguard will be used to generate a VPN tunnel between the EC2 instance and the Fly.io VPN, enabling our app to send requests to our database without using the Internet.

```
fly wireguard create
flyctl wireguard create
? Select Organization:  [Use arrows to move, type to filter]
> ChoreMate (choremate)
❯
❯
❯  flyctl wireguard create choremate iad mypostgrespeer wg0.conf
? Select Organization: ChoreMate (choremate)
Creating WireGuard peer "mypostgrespeer" in region "iad" for organization choremate

!!!! WARNING: Output includes private key. Private keys cannot be recovered !!!!
!!!! after creating the peer; if you lose the key, you'll need to remove    !!!!
!!!! and re-add the peering connection.                                     !!!!
? Filename to store WireGuard configuration in, or 'stdout':

```

Will use this file in a moment.


**Step 3 - Clone and customize the demo configuration**

We'll keep everything in a single `.tf` file, but feel free to learn how to split a Terraform project into different files.

```
git clone git@github.com:xantrac/rds-fly-connector.git
```
the folder has the following structure

```
.
├── main.tf 
├── pgbouncer.ini.tmpl
├── terraform.tfvars
└── userlist.txt
```

- `main.tf` includes all the Terrform instructions to create our infrastructure
- `pgbouncer.ini.tmpl` is a template for pgbouncer configuration to run in our ec2 instance
- `terraform.tfvars` contains some variables that you will personalize
- `userlist.txt` is a list of postgres users that will be created with the database, defaults to `postgres`

3.1 Open the `terraform.tfvars` file and edit the existing placeholders:

```
aws_account = "cool-app"
vpc_name = "cool-app-vpc"
postgres_instance_name = "cool-app-prod"
database_name = "cool-app"
```

3.2 Copy your `wg0.conf` file to the root of the `rds-fly-connector` directory

```
cp ~/wg0.conf ~/rds-fly-connector #or wherever your files are located
```

Great, your Terraform file is ready.

**Step 4 - Apply the terraform configuration**

Will split this into two steps, first will *plan* the configuration.

From within the `rds-fly-connector` directory

```
aws-vault exec choremate -- terraform plan -out prod.tfstate
```

This command will analyze the configuration, fill in the templated variables, generate the final configuration object, and generate a `.tfstate` file which is the artifact of how our infrastructure will look at a given time.

Then will apply the `.tfstate` file to create our resources in AWS

```
aws-vault exec choremate -- terraform apply prod.tfstate
```

At this point, (likely 10 minutes later) Terraform is done applying the configuration. Let's verify the result.

- your database should be available at the following link https://us-east-1.console.aws.amazon.com/rds/home?region=us-east-1#databases:

- your ec2 bastion should be here https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:instanceState=running



Then click on *connect*

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kj9d0wx7i4qspljefteg.png)

and copy the public address

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j4pyl24a30c8cfb7ral5.png)

use the public address and the ssh key created in step 1 to connect to the bastion

```
❯  ssh -i ~/.ssh/aws_terraform  ubuntu@ec2-52-202-142-228.compute-1.amazonaws.com
Welcome to Ubuntu 22.04.3 LTS (GNU/Linux 6.2.0-1018-aws x86_64)
```

and attempt to connect to your database, you will need:

- username which is `postgres` unless you changed it
- password, can be found here https://us-east-1.console.aws.amazon.com/secretsmanager/listsecrets?region=us-east-1 (dig a little, should be easy)
- database address, on your database page https://us-east-1.console.aws.amazon.com/rds/home?region=us-east-1#databases: under *Connectivity & security*

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o4l63hmqmy1rkvpg9iz8.png)

putting al together we should have our postgres URL that looks like this

```
psql postgres://postgres:#{password}@#{database_url}:5432/#{database_name}

ubuntu@ip-172-16-101-225:~$ psql postgres://postgres:#########@############.us-east-1.rds.amazonaws.com:5432/chore_mate
psql (14.10 (Ubuntu 14.10-0ubuntu0.22.04.1), server 15.5)
```

If you see this everything is working, so far!

Let's check if WireGuard is working correctly when attempting to access the Postgres instance from within your Fly app.

When you create the WireGuard configuration in step 2 you also register the named peer within the Fly registry.

```
fly wireguard list
```

and the name you assigned to your configuration should be listed there.

Now because our ec2 bastion is running WireGuard and PgBouncer as a proxy our database should be reachable through that host at the `#{peer_name}._peer.internal` and we should be able to compose our postgres URL in this way.

```
postgres://postgres:#{password}@#{peer_name}._peer.internal:5432/#{db_name} 
``` 

```
fly ssh console

apt update
apt upgrade
apt install -y postgresql-client

 psql postgres://postgres:#{password}@#{peer_name}._peer.internal:5432/#{db_name}

psql (14.10 (Ubuntu 14.10-0ubuntu0.22.04.1), server 15.5)
```

If even this step worked you are done.

**Step 5 - Add the DATABASE_URL env to your app**

```
❯ fly secrets set DATABASE_URL=postgres://postgres:#{password}@#{peer_name}._peer.internal:5432/#{db_name} 
```

Enjoy your new AWS database!


P.S. Please let me know if you encounter any hiccups in the process so I can keep this guide updated.