Implementing S3 Gateway VPC Endpoints with Terraform
Tags: terraform, infrastructureascode, aws, beginners
Published: 2023-08-19 08:59:24

Recently, in the process of optimizing the security and cost of our service infrastructure, we revisited the way applications placed in private subnets access S3. Until now, access was via the Internet (NAT Gateway). However, by introducing a Gateway VPC Endpoint, we achieved a more secure and cost-effective private access.

## Directory Structure
The directory structure for Terraform is modularized, with configurations separated for each environment (dev, stg, prod). The `modules` directory contains the Terraform modules (like VPC Endpoint, Route Table, etc.) used in the project.


```lua
-- terraform-project/
   -- environments/
      -- dev/
         -- backend.tf
         -- main.tf
      -- stg/
         -- backend.tf
         -- main.tf
      -- prod/
         -- backend.tf
         -- main.tf
   -- modules/
      -- vpc_endpoint/
         -- main.tf
         -- variables.tf
         -- outputs.tf
         -- provider.tf
         -- README.md
      -- route_table/
         -- main.tf
         -- variables.tf
         -- outputs.tf
         -- provider.tf
         -- README.md
      -- ...other…
   -- docs/
      -- architecrture.drowio
      -- architecrture.png
```

## Implementation

### Endpoint Configuration
First, we describe the VPC Endpoint configuration in `modules/vpc_endpoint/main.tf`.

```tf
data "aws_ssm_parameter" "app_vpc_id" {
  name = "/vpc/id/app"
}

resource "aws_vpc_endpoint" "s3" {
  vpc_id            = data.aws_ssm_parameter.app_vpc_id.value
  service_name      = "com.amazonaws.ap-northeast-1.s3"
  route_table_ids   = [var.route_table_app_endpoint_id]
  vpc_endpoint_type = "Gateway"

  policy = <<POLICY
{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Action": "*",
      "Effect": "Allow",
      "Resource": "*",
      "Principal": "*"
    }
  ]
}
POLICY

  tags = {
    Name = "app-vpce-s3"
  }
}
```

- Retrieving VPC ID
For `vpc_id`, specify the ID of the VPC where the application accessing S3 is placed. In this case, the VPC ID is fetched from the SSM Parameter Store, as the corresponding VPC was manually created before the introduction of Terraform.

- Specifying the Route Table
`route_table_ids` specifies the ID of the route table to be associated with the created VPC Endpoint. This route table is already managed by Terraform, so its ID is read via a variable.

`modules/vpc_endpoint/variables.tf`
```tf
Copy code
variable "route_table_app_endpoint_id" {
  description = "The ID of the route table"
  type        = string
}
```

- Endpoint Policy
Endpoint policies control access to S3 via the endpoint. With policies, you can allow access only to specific buckets or specific actions.
Below is an example allowing only the GET action to a specific S3 bucket (`my-allowed-bucket`):

```tf
policy = <<POLICY
{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Action": "s3:GetObject",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::my-allowed-bucket/*",
      "Principal": "*"
    },
    {
      "Action": "s3:*",
      "Effect": "Deny",
      "Resource": "arn:aws:s3:::my-allowed-bucket/*",
      "Principal": "*",
      "Condition": {
        "StringNotEquals": {
          "s3:action": "s3:GetObject"
        }
      }
    }
  ]
}
POLICY
```

The `s3:GetObject` action for the `my-allowed-bucket` bucket is permitted, allowing reading of objects within this bucket. At the same time, all actions other than `s3:GetObject` for the same bucket are denied.

- Tagging
The tag `Name = "app-vpce-s3"` helps in identifying and filtering resources. Proper naming conventions and tagging are vital, especially in large environments or when managing multiple endpoints. This enables connecting to S3 through a private access route while avoiding costs and overhead associated with NAT Gateway.