Streamlining Your Data Transfer
Tags: aws, devops, cloud, security
Published: 2023-04-05 10:30:16

Security threats have been one of the major challenges faced by many organisations and over the years, huge sums of monies have been lost as a result of one security threat or the other.

According to this **[report by Forbes](https://www.forbes.com/sites/splunk/2022/03/01/the-top-security-threats-of-2022/?sh=39b90da02e5d)**, the top 10 security threats of 2022 include:

- **Credential Reuse Attack**
- **Insider Threat**
- **Man-in-the-Middle Attack**
- **Phishing**
- **Ransomware**
- **Watering Hole Attack**
- **Spyware**
- **Social Engineering Attack**
- **DDoS Attack**
- **Cloud Crypto-mining**

The report indicates that these 10 are the most malicious of [**Top 50 Security Threats](https://www.splunk.com/en_us/form/top-50-security-threats.html)** that must be taken note of.

This post discusses a way of mitigating the third threat; ***Man-in-the-Middle Attack***

**A Man-in-the-Middle** happens when an attacker places himself between two targets and relays messages for them. The communicating parties may think there is a secure and direct communication between them but the attacker eavesdrops and possibly changes the original data stream from the sender before delivering it to the receiver.

The workflow in many organizations involve data transfer from one system to another over a network.
Medical history, transactional information, receipts, bank statements, geographical coordinates etc. are few examples of many sensitive data shared across networks.

When files are sent over the internet, the data traverses the public internet with a high risk of being intercepted and tempered with.
*A Man-in-the-Middle Attack* is likely occur if the transfer is not made over a secure network. The repercussions of this can be dire as lives, money, reputations, security of nations amongst other things may be at risk if these files end up in the wrong hands.

In this session, we’ll explore how to **Securely Transfer Files To And From AWS S3 Buckets By Utilizing VPC Endpoints.**

VPC endpoints are virtual devices that are horizontally scaled, redundant, and highly available components of the Amazon VPC. They enable communication between Amazon VPC instances and AWS services without imposing availability risks or bandwidth constraints on network traffic.

They aid customers to connect privately to AWS by providing a secure, efficient and cost-effective way to access services from within your VPC. 
They ensure security by allowing access to AWS services without going through the internet.
The network architecture is simplified since there is no need for a NAT Gateway. This makes it easy to setup and manage.

Network performance is also improved with a reduction in network cost as well. The ability to provide direct access to AWS services reduces latency and improves overall performance of applications while reducing cost.

## Now let’s dive deep into the tutorial

 1. **Prerequisites: You must have an AWS account.**

This tutorial requires an AWS account and a secure connection established between the account and whatever instance required to reach it.

VPC endpoints are created in specific VPCs and are accessible only to systems that have access to those VPCs. The requests may come from other AWS accounts or on-premise networks. Connectivity must be established between the networks in order to proceed.

Connectivity with other AWS accounts require VPC Peering while connectivity with on-premise networks requires establishing a Site-to-Site VPN connection between the two networks.

Ensure that a [Site-to-Site VPN connection](https://docs.aws.amazon.com/vpn/latest/s2svpn/working-with-site-site.html) or a [VPC Peering connection](https://docs.aws.amazon.com/vpc/latest/peering/create-vpc-peering-connection.html) is established depending on your use case.

The S3 bucket to/from which the files would be accessed must also exit.

2. **Login to your AWS account and in the search bar, type VPC, navigate to the VPC page and click on “Endpoints”**.

![Click on Endpoints on the VPC Page](https://cdn-images-1.medium.com/max/2314/1*vNlJJZVOboP3TMpJTQ5idg.png)

**3. Click on “Create endpoint”.**

![Click on create endpoint](https://cdn-images-1.medium.com/max/3762/1*bSbKJ5_ZJivEnLRiFVVECQ.png)

**4. Fill the details below.**

![](https://cdn-images-1.medium.com/max/2000/1*FND2_x8KrxJpq5mC3danZA.png)

* Enter the name of the endpoint you want to create.

* Select the AWS services in the service category.

* All the AWS services that are accessible via VPCEs are listed here. You can select as many services as you want but in our use case, we only need access to the S3 hence we select only that.

* In the services, select the interface type of the S3 services.

* There are two types of VPC endpoints; **Interface and Gateway **endpoints. We are going to work with interface endpoints because they support a host of services and are accessible via peered VPCs.

**5. Specify the VPC and subnets where the endpoint would be created.**

![Specify the VPC, subnet and security group for the VPCE](https://cdn-images-1.medium.com/max/2000/1*X6bUbxkDxN_ZxGw3pzWsQA.png)

* Select the VPC that as peered in *Step 1.*

* Choose the private subnets in that VPC.

* If you want to limit access to only specific IP addresses in the other account, create a security group and grant access to IPs or range of IPs allowed to reach it on port 443 as well.

**6. Edit the VPCE policy to limit access to specific services.**

![](https://cdn-images-1.medium.com/max/2000/1*z2pQXhJQV7M6_lANITc4UA.png)

Granting full access would give users or services in the source accounts the ability to access all AWS services in the destination account, which is not recommended. In order to implement the *Principle of Least Privilege*, select custom to limit access to only the services they are required to reach and in this case the designated S3 bucket.

Go ahead and create the endpoint. Next, we’ll modify the bucket’s policy to allow access from the VPCE.

**7. Modify the bucket’s ACL to allow access from the VCPE**.

![edit bucket acl](https://cdn-images-1.medium.com/max/2000/1*ww0aUu9G9s6SgImxuZ_Tbw.png)

## Accessing Objects In The Bucket.

Now that we have successfully created our VPCE, we would go ahead and use it to access our S3 bucket and perform various operations on the bucket. Objects in the bucket can be accessed by the AWS CLI or CDK.

### Using The AWS CLI

Below are the steps to follow when connecting with AWS CLI:
First, ensure the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-mac.html#cliv2-mac-install-cmd) is installed on the device trying to reach the bucket.

In the examples below, replace the VPC endpoint ID

***“vpce-1a2b3c4d-5e6f.s3.eu-west-1.vpce.amazonaws.com”***, filename “***file.txt”***, the region and bucket name “***my-bucket”*** with appropriate information.

In the details tab of the VPCE created, copy the DNS name. There are usually 4 DNS names for every VPCE. The first one indicates the endpoint at the regional level whiles the others are at the availability zone level.

Copy he first one to proceed.

Let’s say our file of interest here is file.txt

**List Files In Bucket**

    aws s3 --region eu-west-1 \
    --endpoint-url https://bucket.vpce-1a2b3c4d-5e6f.s3.eu-west-1.vpce.amazonaws.com \
    ls s3://my-bucket/

**Write File To Bucket**

    aws s3 --region eu-west-1 \ 
    --endpoint-url https://bucket.vpce-1a2b3c4d-5e6f.s3.eu-west-1.vpce.amazonaws.com \ 
    cp file.txt s3://my-bucket/

For instance, if there is file on the host called, file.txt. the command above is used to copy it to the bucket.

**Download File From Bucket**

    aws s3 --region eu-west-1 \
    --endpoint-url https://bucket.vpce-1a2b3c4d-5e6f.s3.eu-west-1.vpce.amazonaws.com  \
    cp s3://my-bucket/file.txt file.txt

To download a file “file.txt” to the host, use the command above.

### Using The AWS SDK

Various AWS clients can be employed to access S3 buckets with various programming languages. The clients have to be configured to use an endpoint URL for accessing buckets and performing operations through S3 interface endpoints. Here are some examples of such configurations:

**Using The AWS Python SDK (Boto3)**

    s3_client = session.client(
    service_name='s3',
    region_name='eu-west-1',
    endpoint_url='https://bucket.vpce-1a2b3c4d-5e6f.s3.eu-west-1.vpce.amazonaws.com'
    )

**Using The AWS SDK For Java 1.x**

    // bucket client
    final AmazonS3 s3 = AmazonS3ClientBuilder.standard().withEndpointConfiguration(
            new AwsClientBuilder.EndpointConfiguration(
                    "https://bucket.vpce-1a2b3c4d-5e6f.s3.eu-west-1.vpce.amazonaws.com",
        Regions.DEFAULT_REGION.getName()
            )
    ).build();
    List<Bucket> buckets = s3.listBuckets();

**Using The AWS SDK For Java 2.x**

    // bucket client
    Region region = Region.EU_WEST_1;
    s3Client = S3Client.builder().region(region).endpointOverride(
    URI.create("https://bucket.vpce-1a2b3c4d-5e6f.s3.eu-west-1.vpce.amazonaws.com")
    ).build()

Refer to the official [AWS documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/privatelink-interface-endpoints.html#accessing-bucket-and-aps-from-interface-endpoints) for further details.

*References:*
[https://www.enisa.europa.eu/topics/incident-response/glossary/man-in-the-middle](https://www.enisa.europa.eu/topics/incident-response/glossary/man-in-the-middle).

*Photo Credit : [Sean Patrick on Pexels](https://www.pexels.com/@seanpatrickphotography/)*