Bastion host in AWS
Tags: 100daystooffload, aws, cloud, devops
Published: 2023-08-26 17:51:46

## What is a Bastion host

I like to see this as a security pattern for temporary access to services on a private network in the cloud. One popular use case is to access the Amazon RDS from a developer machine.

The bastion host is a machine that acts like a proxy/bridge/jump box between the public internet and the private network.

![bastion host](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1wk3j7zxe5am4lsussak.png)

This machine has only one task and that is to create a secure communication channel to a specific service on the private network. Therefore it's important to make it as secure as possible, such as having strong authentication, software up to date, not running unnecessary software or open ports and monitoring.

## How to setup a Bastion host for AWS

There is a short cut available at the great [aws-quickstart](https://aws-quickstart.github.io/quickstart-linux-bastion/) page, which has an already baked cloudformation template file for us to use. Which sets up the following.

> * A highly available architecture that spans two Availability Zones.
* A virtual private cloud (VPC) configured with public and private subnets, according to AWS best practices, to provide you with your own virtual network on AWS.
* In the public subnets:
 * Managed network address translation (NAT) gateways to allow outbound internet access for resources in the private subnets.
 *1–4 Linux bastion hosts in an Amazon Elastic Compute Cloud (Amazon EC2) Auto Scaling group for connecting to Amazon EC2 instances and other resources deployed in public and private subnets.
* An Amazon CloudWatch log group to hold the Linux bastion host shell history logs.
* AWS Systems Manager for access to the bastion host.

There are two [templates](https://github.com/aws-quickstart/quickstart-linux-bastion/tree/main/templates) available, depending on your needs, one for an existing VPC and one for a new VPC.

Both templates deploys a Linux bastion host in an auto-scaling group which make it easy to start/stop the service. We can later enable the feature *scheduled actions* which will stop the bastion host at a specific time, which is perfect, since people often forget to turn it off and this solution should only be used temporarily.

### Step by step guide

This guide will enable access from a developer machine with SSH. The default is to only allow access from [AWS Systems Manager](https://aws.amazon.com/systems-manager/) to the bastion host.

1. Download the template, [linux-bastion-entrypoint-existing-vpc.template.yaml](https://github.com/aws-quickstart/quickstart-linux-bastion/blob/main/templates/linux-bastion-entrypoint-existing-vpc.template.yaml), to your machine
2. Open up a new tab and go the [AWS Management console](https://console.aws.amazon.com/console)
3. Navigate to the *EC2* service, and in the left pane click on the *Key Pairs* menu item. ![Key pairs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/occ5haubw5bnemohc5o8.png)
4. In *Key pairs* view, click the *Create key pair* button and fill out the form. ![Create key pair](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uekkeqn1uadvenxp8tue.png)
5. When the key pairs is created and you have saved the pem file in a secure location, **this certificate should be treated as a secret! Be restrictive of how it's shared, and to whom**, navigate to the CloudFormation service
6. In the *Stacks* view, click the button *Create stack* -> *With new resources (standard)* ![new stack](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2bn8yrgq4nl5yzcck8j4.png)
7. In the *Create stack* dialog, choose *Template is ready* and upload the downloaded template from step one.
![create stack dialog](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mzsy8xozlz2kjtit2wvj.png)
8. Fill out the form, *Specify stack details* ![Specify stack details](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n7rcoo1l8mfo796exbci.png)
 * Network configuration is all about selecting correct VPC and subnets.
 * Under Amazon EC2 configuration, remember to select your key pair
 * The other parameters can use the defaults
9. Click *Next*, and then click *Next* for *Configure stack options*, and finally click *Submit*
10. Wait for the *Cloudformation* to deploy all the resources
11. Done with the first part


To add a scheduled action that turns off the bastion host automatically we will need to do the following.

1. Navigate to the *EC2* service and click on the *Auto Scaling Groups* link in the bottom of the left pane.
2. In the list of auto scaling groups, click on the list item representing the bastion host.
3. In the next view click on the tab, *Automatic scaling* ![Automatic scaling](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2xgbxh3ne4vtb6nse21e.png)
4. At the bottom of the page, click on the button *Create scheduled action*, and fill out the form. ![Create scheduled action](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2k1kief5srdbcfjkqenu.png)
5. Done

One other option is to update the Cloudformation template file and add the action there by adding some *yaml code* to the template;

```yaml
ScheduledActionDown:
  Type: "AWS::AutoScaling::ScheduledAction"
  Properties:
    AutoScalingGroupName:
      Ref: BastionAutoScalingGroup
    MinSize: 0
    MaxSize: 0
    DesiredCapacity: 0
    Recurrence: "0 19 * * *"
    TimeZone: "Europe/Stockholm"
```

The bastion host is now configured to be started on-demand and is taken down automatically at 19:00 Swedish time. To start the bastion host we will need to follow these instructions. 

1. Use EC2 service view and navigate to the *Instances* view in the left pane
2 If the instance *BastionHost* is listed and has state **Running** - it's already started. Otherwise, follow these steps
 1. Go to the *Auto Scaling Groups* view in the left pane
 2. Click on the list item named *bastion-host-...*
 3. Click the Edit button for the *Group details*
 4. Set all values to **1** (Desired capacity, Minimum capacity, Maximum capacity) and click the *Update* button ![Start instance](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qmppl81glsk29khtagqc.png)
 5. The bastion host will now start one instance in 10-30 seconds.

## Connect to the bastion host with SSH

The bastion host should almost be ready to be connected to from your local machine via a SSH-tunnel. To be able to connect to the bastion host you will need to use an IP address that is allowed by the ingress rule on the security group for the bastion host. 

1. Once more jump to the *EC2* service and navigate to the *Security Groups* view via the left pane
2. Locate the security group name *bastion-host-...* in the list of groups and click on the security group id for that item
3. Click on the *Edit inbound rules* button ![Edit inbound rules](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/py3blx48zlf09xst2r5k.png)
4. Add a new rule for your IP address (CIDR block)  ![Add rule](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ac1kp82eazjbsm1eoihy.png)
5. Done, now should SSH traffic be allowed from your IP address

To open up a SSH tunnel to the bastion host from a developer machine that is using Ubuntu we can use this procedure with the command line interface.

1. Now we need to grab the pem file from when we created the key pair in the beginning of this blog
2. Put a copy of the pem file in the folder *~/.ssh/*
3. Make sure that the file has correct permissions ```bash
chmod 600 ~/.ssh/bastion-host.pem
```
4. Open the SSH tunnel ```sh
ssh -i ~/.ssh/bastion-host.pem  ec2-user@[ip-of-bastion-host]
# The [ip-of-bastion-host] is found at
# AWS Console -> EC2 -> Instances -> BastionHost -> Public IPv4 address
```
5. Now the the tunnel is up between your machine and the bastion host.

## Connect to a service on the private network

More configuration is needed to enable us to communicate with a service on the private network via the bastion host. Lets see how we could enable the possibility to communicate with a database hosted by RDS in AWS. What we need to is to enable the correct network traffic in the security group for the bastion host. This is similar to how we configured the inbound rule for allowing our machine to talk to the bastion host, but this time we need to allow outbound traffic.

1. Go to the *EC2* service and navigate to the *Security Groups* view via the left pane
2. Locate the security group name *bastion-host-...* in the list of groups and click on the security group id for that item
3. Click on the *Edit outbound rules* button in the *Outbound rules* tab ![outbound rules](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tvopx7wdh2k58xu9cmis.png)
4. In the edit outbound rules view, click on the *Add rule* button and configure the rule for the outgoing database traffic ![add outbound rule](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vgt6ssuvb2lw7vfrpb35.png)
 * Select a type that match your protocol, the *PostgreSQL* is selected above for communicating with a PostgreSQL database in RDS
 * In the destination you should select the RDS security group
5. Open up a SSH tunnel to the database 
```sh
ssh -i ~/.ssh/bastion-host.pem ec2-user@[ip-of-bastion-host] -L 5432:[internal-dns-name-of-rds-instance]:5432
# Where the [internal-dns-name-of-rds-instance] can be found at
# AWS Console -> RDS -> Databases -> [your database instance] -> Endpoints
```

You should now be able to open your Database management tool and connect to the database on AWS

These rules can of course be added to the Cloudformation template if that feels like a better approach.

```yaml
BastionSecurityGroup:
  Type: AWS::EC2::SecurityGroup
  Properties:
    GroupDescription: Enables access to bastion hosts
    VpcId: !Ref VPCID
    SecurityGroupEgress:
      - IpProtocol: tcp
        FromPort: 5432
        ToPort: 5432
        DestinationSecurityGroupId: !Ref <reference  to your RDS security group>
      - IpProtocol: tcp
        FromPort: 443
        ToPort: 443
        CidrIp: "0.0.0.0/0"
RDSInstanceSecurityGroupIngressRule:
  Type: AWS::EC2::SecurityGroupIngress
  Properties:
    GroupId: !Ref <reference  to your RDS security group>
    IpProtocol: tcp
    FromPort: 5432
    ToPort: 5432
    SourceSecurityGroupId: !Ref BastionSecurityGroup
RDSIngressRule:
  Type: AWS::EC2::SecurityGroupIngress
  Properties:
    IpProtocol: tcp
    FromPort: 5432
    ToPort: 5432
    SourceSecurityGroupId: !Ref BastionSecurityGroup
    GroupId: !Ref <reference  to your RDS security group>
```

The reference to the RDS security group could be given as a parameter to the Cloudformation template like this.

```yaml
RDSInstanceSecurityGroup:
  Type: AWS::EC2::SecurityGroup::Id
  Description: The RDS instance security group 
```

## What else

If you need to create a tunnel often, then this shell alias might be of use.

```sh
# Add these rows in ~/.bashrc 
# NOTE: That I use local port 5434 here instead,
# that is because usually I already have a local database
# on that port and this enable to have both running

rdstunnelup () {
  ssh -i ~/.ssh/bastion-host.pem ec2-user@$1 -L 5434:<internal-dns-name-of-rds-instance>:5432
}
```

Reread the *.bashrc* file

```sh
source ~/.bashrc
```

With that in place, I can use the alias like this.

```sh
rdstunnelup <the ip address to the bastion host>
```

If your are using Windows as your local machine then you can use [PuTTY](https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html) tools to connect to the bastion host. Since PuTTY needs the certificate in the .ppk format we will need to convert the .pem file.

1. Start PuTTYGen (one of the tools in the PuTTY-package)
2. Select *Conversion -> Import Key* from the menu
3. Select the pem file that is used for the bastion host
4. Enter a passphrase (and do not forget this passphrase)
5. Click on the *Save private key* button

When you have a .ppk file you can proceed with next step to setup the SSH authentication agent. Pageant is an SSH authentication agent. It holds your private keys in memory, already decoded, so that you can use them often without needing to type a passphrase.

1. Open Pageant (one of the tools in the PuTTY-package)
2. Click on the *Add key* button
3. Use the ppk file
4. Enter the passphrase for the ppk file when prompted
5. Close the window.

Now everything is set for configuring the PuTTY client for port forwarding.

1. Open the PuTTY client (one of the tools in the PuTTY-package)
2. In the left pane (Category), select *Connection -> SSH -> Auth* and make sure that "Allow agent forwarding" is selected.
3. In the left pane (Category), select *Connection -> SSH -> Tunnels* and do following;
 - In the *Source port* field, add 5434
 - In the *Destination* field, add the [internal-dns-name-of-rds-instance] and port (5432)
 - Click on the *Add* button
4. In the left pane (Category), select *Session* and do following:
 - In the *Host Name (or IP address)* field, add the [ip-of-bastion-host] which is found under *AWS Console -> EC2 -> Instances -> BastionHost -> Public IPv4*
 - In the *Saved Sessions* field, add *bastion-host*
 - Click on the *Save* button
5. Click on the *Open* button at the bottom of the window
6. Enter the user name: *ec2-user*, when prompted the *login as*
7. Now you should be able to connect to *localhost:5434* to talk to the database in AwS.

## Summary

There is a lot of configuration to setup a bastion host, however that is only done once. When everything is at place it will be easy to connect to your service in the private network in AWS.

It is really educational exercise to setup this solution on AWS, if you have the possibility to this then you should.

Happy configuration!