Going Bastion-less: Accessing Private EC2 instance with Session Manager | by Daniel Da Costa | Towards Data Science

In this post we will set up a private EC2 instance (in a private subnet), and use SSM session manager to access the instance that hosts a Jupyter Notebook server. We will then use PostForwarding with AWS Session Manager to access our server from our local machine.
We'll set up this infrastructure without opening inbound ports or setting up bastion hosts or managing SSH keys!.
It is well known that we can not directly connect to a private EC2 instance unless there is VPN Connectivity or Direct Connect or other network connectivity source with the VPC. A common approach to connect to an EC2 instance on a private subnet of your VPC is to use a Bastion Host.
A Bastion Host is a server whose purpose is to provide access to a private network from an external network (such as the Internet). Because of its exposure to potential attacks, a bastion host must minimize the chances of penetrations. When using a bastion host, you log into the bastion host first, and then into your target private instance. With this approach, only the bastion host will have an external IP address.
However, there are some drawbacks:
You will need to allow SSH inbound rule at your bastion
You need to open ports on your private EC2 instance in order to connect it to your bastion
You will need to manage the SSH key credentials of your users: You will need to generate an ssh key pair for each user or get a copy of the same SSH key for your users
Cost: The bastion host also has a cost associated with it as it is a running EC2 instance. Even a t2.micro costs about $10/month.
Session Manager can be used to access instances within private subnets that allow no ingress from the internet. AWS SSM provides the ability to establish a shell on your systems through its native service, or by using it as a tunnel for other protocols, such as Secure Shell (SSH). Advantages:
It will log the commands issued during the session, as well as the results. You can save the logs in s3 if you wish.
Shell access is completely contained within Identity and Access Management (IAM) policies, you won’t need to manage SSH keys
The user does not need to use a bastion host and Public IPs.
No need to open the ports in the security groups
You will need to Install the Session Manager plugin for the AWS CLI in order to use the CLI to start and end sessions that connect you to your managed instances.
You can check on how to install the plugin for different OS here!
Before creating the EC2 instance you will need a VPC with a Public and Private Subnets. Since will be hosting a Jupyter Notebook on our instance located on the Private Subnet, it will need internet access (so that we can install and update Python packages).
In order to give access to the internet to our private subnet we will be using a NAT Gateway. In addition, to enable internet connectivity, this gateway make sure that the internet doesn’t initiate a connection with the instance.
The network configuration that will be used is represented below:
You can create the network directly on the AWS Console or you can build it using Terraform, you can check the Terraform code here.
Our instance will be deployed on the Private Subnet without a Public IP address configuration since we won't need it.
In order to allow Session Manager access to our instance will need to attach the following IAM role: AmazonSSMManagedInstanceCore. This policy grant instances the permissions needed for core Systems Manager functionality.
For the VPC security group, we won't need to include any inbound rules, allowing only outbound traffic.
With everything set up you can access your instance from the command line:
$ aws ssm start-session --target {YOUR_TARGET_INSTANCE_ID}
It's important to notice that the connection was successfully made without opening any port at the EC2 Security Group and without a Bastion Host!
The following image shows the actual configuration that we are using:
Session Manager has an in-built audit log: AWS Session Manager provides audit logs by default; so each command is logged and stored in CloudWatch Logs or an S3 bucket as per necessary security and compliance regulations.
You can also have a Session History on the console:
Many customers use SSH Tunnel to remotely access services not exposed to the public internet. Basically, the SSH client listens for connections on a configured port, and when it receives a connection, it tunnels the connection to an SSH server. The server connects to a configurated destination port, possibly on a different machine than the SSH server.
In OpenSSH, local port forwarding is configured using the -L option:
ssh -L 9999:localhost:8888 user@instance
This example opens a connection to the instance as user user, open port 9999 on the local computer, a forward everything from there to localhost:8888.
Port Forwarding for AWS System Manager Session Manager allows you to securely create tunnels between your instances deployed in private subnets, without the need to start the SSH service on the server, to open the SSH port in the security group, or the need to use a bastion host.
In this post, we won’t go through on how to set up a Jupyter Notebook Server on an EC2 instance, but you can find all the information need to do so on this link.
Once the prerequisites are met, you use the AWS CLI to create the tunnel:
$ aws ssm start-session --target {YOUR_TARGET_INSTANCE_ID} --document-name AWS-StartPortForwardingSession --parameters "portNumber"=["8888"],"localPortNumber"=["8888"]
OBS: While I was using the Jupyter Notebook Server, I faced a very high latency problem; I think it might be because of the Port Forwarding from Session Manager. If you are facing the same problem, please, leave a comment.
In this post, we saw how to use Session Manager to access a private EC2 instance, without the need to add inbound rules to the instance Security Group, manage SSH Keys and use another instance as a Bastion Host. We also learned how to use Post Forwarding using Session Manager.
All the Terraform Code used in this post can be found Here!