AWS PrivateLink for more secure and cost-effective inter-service communication
Tags: aws, devops, privatelink
Published: 2021-09-04 14:00:01

In this article, I'll be explaining how to use AWS PrivateLink to secure and more cost-effective AWS inter-service communication. AWS PrivateLink is a widely discussed area in AWS cloud and, yet I have decided to write my recent experience with AWS PrivateLink.

## AWS PrivateLink ##

AWS PrivateLink was launched in 2019 to allow users to securely access AWS services without exposing their traffic to the public internet.

*__AWS PrivateLink__ is a highly available, scalable technology that enables you to privately connect your VPC to supported AWS services, services hosted by other AWS accounts (VPC endpoint services), and supported AWS Marketplace partner services.*

Please find the AWS PrivateLink whitepaper from [here](https://d1.awsstatic.com/whitepapers/aws-privatelink.pdf).

The following figure describes the usage of PrivateLink. 

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9h69n97vhd2pytmv1yfz.jpg)

As per the above figure, the diagram on the left shows ECS is pulling images from ECR using the public internet even though they are in the same AWS account. AWS PrivateLink, on the other hand, keeps communication with ECS and ECR within the AWS network, making it more secure and cost-effective.

AWS PrivateLink supports both same region and cross-region access. However, For cross-region PrivateLink to work, you may need to use inter-region VPC peering. In this article,  I will only discuss the use of PrivateLink in the same region. More information about cross-region PrivateLink support may be found [here](https://aws.amazon.com/about-aws/whats-new/2018/10/aws-privatelink-now-supports-access-over-inter-region-vpc-peering/).

## VPC Endpoints ##
VPC endpoint allows AWS users to connect to AWS services and VPC endpoint services in a secure manner (services hosted by AWS customers and partners are known as VPC endpoint services). AWS PrivateLink is used to power VPC endpoints. AWS PrivateLink uses private IP addresses to communicate with internal AWS services. VPC Endpoints are categorized into three categories: (1) _Interface Endpoints_,(2) _Gateway Load Balancer Endpoints_ and (3) _Gateway Endpoints_.

##### Interface Endpoints #####
AWS PrivateLink connects services provided by AWS using Interface Endpoints (Interface VPC Endpoints). AWS uses Network Load Balancers (NLB) to link interface endpoints to AWS services in the background while utilizing AWS PrivateLink.

Interface endpoints can be used to connect AWS managed services, customer hosted AWS Services and [endpoint services](https://docs.aws.amazon.com/vpc/latest/privatelink/endpoint-service.html).

##### Gateway Load Balancer Endpoints (GWLB Endpoints)#####
GWLB Endpoints are primarily used to intercept network traffic for security inspection. Only services that are configured to use Gateway Load Balancers are supported by GWLB Endpoints.

##### Gateway Endpoints #####
These types of endpoints are used as a route table target for traffic generated into AWS supported services. At the time of writing this article, only Amazon S3 and DynamoDB are supported for use of Gateway Endpoints.

Differences between GWLB Endpoints and Gateway Endpoints can be found [here](https://docs.aws.amazon.com/AmazonS3/latest/userguide/privatelink-interface-endpoints.html).

Please note that not all AWS services are supported by PrivateLink, and I'll only go through a handful that can utilize PrivateLink to secure and cost-effectively communicate with other services. Please find [this](https://docs.aws.amazon.com/vpc/latest/privatelink/integrated-services-vpce-list.html) link which will be useful in finding services that are compatible with PrivateLink.

### Endpoint for ECR ###
Amazon ECR is a fully managed container registry that works with EKS, ECS, and Lambda. Please find the following steps to create a PrivateLink for ECR service. Without configuring a PrivateLink for ECR, you have been pulling the images from ECR to EKS/ECS or lambda via public internet by leaving the AWS Network. This comes as an added cost if you are using NAT Gateway to communicate with the public internet. 

For said services to pull images in ECR privately, we may need to create two endpoints.
1. Interface Endpoint for ECR Docker
2. Gateway Endpoint for Amazon S3

Let's begin!

Navigate to the VPC service in AWS Console and create an endpoint using the Create Endpoint button.
![Alt Text](https://res.cloudinary.com/dnqnztxfa/image/upload/v1630005082/11_khmu7e.jpg)

Leave the Service category as default 'AWS Services' and search for 'ecr' under service name. Select Interface <code>com.amazonaws.[region].ecr.dkr</code>. (select the region you are currently working on)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gbli46qjn46auz0a57j1.png)

Select the desired VPC and subnets where your instances reside. Leave the _Enable DNS name_ as ticked.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mspx8of8w2tkljiujvr1.png)

Create a new dedicated security group and allow all inbound HTTPS traffic from port 443. This will allow incoming traffic from the subnets of the selected VPC.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rwqq5f32civ0zccqxnj2.png)

Search and select the newly created security group while keeping the default policy as it is. Create a new tag with Key as _Name_ and Value as _ECR_endpoint_ or something you prefer and save the endpoint.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ourl3h2oh4sv9bcet6h9.png)

Initially, the endpoint will be pending.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1egd6y2fxyynciojknm4.png)

The status should change to _accessible_ in a few minutes if everything goes well.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ivf8wwwpqsj1hanir1ld.png)

### Endpoint for S3 ###

As mentioned earlier, for ECR to work privately, we may need to create a Gateway Endpoint for Amazon S3 as ECR uses S3 to store docker images as layers under the hood.

Same as before. create a new endpoint and search for <code>com.amazonaws.[region].s3</code>. (select the region you are currently working on).
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dhr4ct0ns6o9nq1apa4r.png)

Select the VPC where your instances reside and select the appropriate route table attached to the VPC. Leave the rest as default and save the endpoint.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q8ep570p01ao2wcjcntj.png)

Voilà! Both endpoints should be in _available_ state after a while.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zsc3g0icbc16d74qjppq.png)
That should do it for ECR. If everything went well, you should experience a sudden change in the NAT gateways metrics (in Cloudawatch). Furthermore, in AWS Cost Explorer, do notice the cost change for _EC2-Other ($)_ (Cost change might take some time, therefore notice the cost change after few days/weeks).

There are few other services that can help reduce the cost and enhance AWS security.

### Endpoint for Cloudwatch Logs###
When adding logs to the Cloudwatch by services such as ECS/EKS and Lambda, there are sent to the Cloudwatch through the public internet. This increases both the security risk for sensitive data and NAT Gateway cost.

To avoid the above, let's create an Interface Gateway for Cloudwatch Logs.

Head over to the _Endpoints_ and create a new interface endpoint using <code>com.amazonaws.[region].logs</code> select the region you are currently working on.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p84ww6uatza64zrhzn13.png)

Select the desired VPC and subnets where your instances reside. 
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pyyv2t5mjwhwvhh5oyqp.png)

Create a security group same as before by allowing HTTPS traffic through port 443 and save the endpoint.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0ecslfshbz84klg8xwzt.png)
Now your logs should be added to Cloudwatch using the internal AWS network.

### Endpoint for SSM###
There's one more service I would like to highlight and that is AWS SSM. This can be useful if you are having an EC2 instance without access to the public internet. For example, by creating an _Interface Endpoint_ for SSM, your instance can securely access parameters in the _Parameter Store_ without internet access.

Let's create a new VPC endpoint using the <code>com.amazonaws.[region].ssm</code> service. Region should be same as your current selected region in the AWS console.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vokuhb5pah6iuwzqb471.png)

Same as before, create a new security group for SSM Privatelink with allowing HTTPS access through port 443. Keep the rest the same as before and save the endpoint.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lqv96vridorwy41ovkrz.png)

Finally, you should have four VPC endpoints in _available_ state contributing secure inter-service communication to your resources.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/apxutknbz07yt7fo2x53.png)

## AWS PrivateLink with Cross Account ##

You may be wondering how to use AWS PrivateLink when you have multiple AWS accounts for different environments and, there can be resources that are commonly used by other accounts. For example, all your ECR images could be in one account(Let's say master/root account) and, other accounts such as Staging/Production are using the master/account's ECR to store and pull the images.

In that case, creating a PrivateLink is similar to the same account PrivateLink creation. PrivateLinks needs to be created in each account that is trying to pull the images from. 
Please find the diagram I have created which illustrates PrivateLink usage with multi accounts.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mk6w0u20d7swi5815dnk.png)

As per the above diagram, secure cross account access with PrivateLink can be achieved by creating VPC endpoints in each account(same region) where you are trying to access the resources (Make sure to avoid creating endpoints in the root/master account).

#### References ####
1. AWS PrivateLink Whitepaper (https://d1.awsstatic.com/whitepapers/aws-privatelink.pdf)
2. AWS PrivateLink and VPC endpoints (https://docs.aws.amazon.com/vpc/latest/privatelink/endpoint-services-overview.html)
3. Cover photo by <a href="https://unsplash.com/@kalvisuals?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">KAL VISUALS</a> on <a href="https://unsplash.com/s/photos/jungle-dark?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>