AWS Prefix Lists for the Organization
Tags: aws, lambda, vpc, automation
Published: 2023-03-07 03:03:00

AWS Managed Prefix lists are a really powerful way of abstracting the details of CIDR Blocks into something meaningful for the humble cloud engineer.

Much like DNS does for IP addresses to hostnames, we can create a managed prefix for any service that has several CIDR blocks associated with it.

I've talked about this before [here](https://sjramblings.io/aws_managed_prefixes/), but with more and more stuff needing to talk to services hosted on the internet and with our implementation of AWS Network Firewall, it is time for a re-visit.

## The challenge

Most people are familiar with the AWS Public IP Address Ranges source ip-ranges.json, doco linked [here](https://aws.amazon.com/blogs/aws/aws-ip-ranges-json/), released in 2014. Can you believe it!

It provides all the Public IP ranges for all services across all regions. A very handy thing to have, and with it being hosted in a JSON file, we can automatically parse it with any automation.

But why don't AWS do this for us? I can't answer that today; however, a good example of such automation exists in the AWS samples GitHub [here](https://github.com/aws-samples/update-aws-ip-ranges).

The AWS Samples repo does all the heavy lifting with Lambda and creates the following:-

![AWS-Samples Workflow](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/flpj2v82spken7uv8kry.png)

But I'm feeling hacky and need some [Organization-level](https://docs.aws.amazon.com/vpc/latest/userguide/sharing-managed-prefix-lists.html#sharing-share) sharing, [Lambda PowerTools](https://awslabs.github.io/aws-lambda-powertools-python), [AppConfig](https://docs.aws.amazon.com/appconfig/latest/userguide/what-is-appconfig.html) and deployment through [SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html).
Let's get into it!

## Some Enhancements
### Lambda PowerTools

As the [homepage](https://awslabs.github.io/aws-lambda-powertools-python) displays, Lambda PowerTools is a suite of utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, idempotency, batching, [and more](https://awslabs.github.io/aws-lambda-powertools-python/2.9.1/#features).

One of the reasons for implementing this is to make the Cloudwatch logs nicer to parse. It also removes a few lines of code in Lambda without requiring major logging structure changes.

As I've rebuilt this into SAM Cli, this is just an addition to the `requirements.txt` file to include `aws-lambda-powertools`.

### AppConfig

If you look at the original AWS example, it requires a services.json to be defined to configure what regions and services to create the WAF IPsets and VPC Prefixes. It will look something like the following:-

```json
{
  "Services": [
    {
      "Name": "CODEBUILD",
      "Regions": [
        "ap-southeast-2"
      ],
      "PrefixList": {
        "Enable": true,
        "Summarize": true
      },
      "WafIPSet": {
        "Enable": true,
        "Summarize": true,
        "Scopes": [
          "REGIONAL"
        ]
      }
    }
  ]
}
```

This is in JSON format, so perfect for AppConfig.
As we have configured Lambda PowerTools, AppConfig is easily added as a Lambda Layer.

I've made several updates to the `template.yaml` file to define our AppConfig hosted configuration and deployment methods, adding environment variables and our Lambda layer.

```
Parameters:
  AppConfigAppName:
    Type: String
    Description: AppConfig Application Name
    Default: aws-ip-ranges
  AppConfigAppEnvironmentName:
    Type: String
    Description: AppConfig Application Environment Name
    Default: dev
  AppConfigName:
    Type: String
    Description: AppConfig Name
    Default: services
  AppConfigLayerArn:
    Type: String
    Description: Retrieve AWS AppConfig Lambda extension arn from `https://docs.aws.amazon.com/appconfig/latest/userguide/appconfig-integration-lambda-extensions-versions.html#appconfig-integration-lambda-extensions-enabling-x86-64`
    Default: arn:aws:lambda:ap-southeast-2:080788657173:layer:AWS-AppConfig-Extension:91
  AwsOrgArn:
    Type: String
    Description: The ARN of the AWS Organization used to share Prefix Lists
    Default: notset

Resources:
  SAMConfigApplication:
    Type: AWS::AppConfig::Application
    Properties:
      Name: !Ref AppConfigAppName
  
  Environment:
    Type: AWS::AppConfig::Environment
    Properties:
      Name: !Ref AppConfigAppEnvironmentName
      ApplicationId: !Ref SAMConfigApplication
  
  SAMConfigConfigurationProfile:
    Type: AWS::AppConfig::ConfigurationProfile
    Properties:
      ApplicationId: !Ref SAMConfigApplication
      Name: !Ref AppConfigName
      Type: 'AWS.Freeform'
      LocationUri: 'hosted'

  SAMConfigDeploymentStrategy:
    Type: AWS::AppConfig::DeploymentStrategy
    Properties:
      Name: "SAMConfigDeploymentStrategy"
      Description: "A deployment strategy to deploy the config immediately"
      DeploymentDurationInMinutes: 0
      FinalBakeTimeInMinutes: 0
      GrowthFactor: 100
      GrowthType: LINEAR
      ReplicateTo: NONE

  BasicHostedConfigurationVersion:
    Type: AWS::AppConfig::HostedConfigurationVersion
    Properties:
      ApplicationId: !Ref SAMConfigApplication
      ConfigurationProfileId: !Ref SAMConfigConfigurationProfile
      Description: 'AWS Service configuration for update-aws-ip-ranges'
      ContentType: 'application/json'
      Content: |
        {
          "Services": [
            {
              "Name": "CODEBUILD",
              "Regions": [
                "ap-southeast-2"
              ],
              "PrefixList": {
                "Enable": true,
                "Summarize": true
              },
              "WafIPSet": {
                "Enable": true,
                "Summarize": true,
                "Scopes": [
                  "REGIONAL"
                ]
              }
            }
          ]
        }
  AppConfigDeployment:
    Type: AWS::AppConfig::Deployment
    Properties:
      ApplicationId: !Ref SAMConfigApplication
      ConfigurationProfileId: !Ref SAMConfigConfigurationProfile
      ConfigurationVersion: !Ref BasicHostedConfigurationVersion
      DeploymentStrategyId: !Ref SAMConfigDeploymentStrategy
      EnvironmentId: !Ref Environment

  LambdaUpdateIPRanges:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: src/
      Handler: app.lambda_handler
      Runtime: python3.9
      Architectures:
        - x86_64
      Environment:
        Variables:
          APP_CONFIG_APP_NAME: !Ref AppConfigAppName
          APP_CONFIG_APP_ENV_NAME: !Ref AppConfigAppEnvironmentName
          APP_CONFIG_NAME: !Ref AppConfigName
          AWS_ORG_ARN: !Ref AwsOrgArn
          LOG_LEVEL: INFO
      Layers:
        - !Ref AppConfigLayerArn
```

In order to pull in the config, we add the following function and a few adjustments in the handler code.

```
def get_service_config():
    try:
        appconfig = f"http://localhost:2772/applications/{APP_CONFIG_APP_NAME}/environments/{APP_CONFIG_APP_ENV_NAME}/configurations/{APP_CONFIG_NAME}"
        with request.urlopen(appconfig) as response:  # nosec B310
            config = response.read()
        return config
    except Exception as error:
        logger.error("Error retrieving AppConfig configuration. Exiting")
        logger.exception(error)
        raise error

```

### Organization-level sharing

To enable all the users in our organization to benefit from our automation, we need to share the VPC Prefixes via Resource Access Manager (RAM).

We define an additional function to create the RAM share on creating the VPC Prefix List.

```python
def create_prefix_ram(client: Any, prefix_list_name: str, prefix_list_arn: str) -> None:
    """Create the VPC Prefix List RAM Share"""
    logger.info("create_prefix_ram start")
    logger.debug(f"Parameter client: {client}")
    logger.debug(f"Parameter prefix_list_name: {prefix_list_name}")
    logger.debug(f"Parameter prefix_list_arn: {prefix_list_arn}")

    logger.info(f'Creating RAM Share "{prefix_list_name}" with Arn "{prefix_list_arn}"')
    response = client.create_resource_share(
        name=prefix_list_name,
        resourceArns=[prefix_list_arn],
        principals=[AWS_ORG_ARN],
        allowExternalPrincipals=False,
        tags=[
            {"key": "Name", "value": prefix_list_name},
            {"key": "ManagedBy", "value": MANAGED_BY},
            {"key": "CreatedAt", "value": datetime.now(timezone.utc).isoformat()},
            {"key": "UpdatedAt", "value": "Not yet"},
        ],
    )

    logger.info(f'Created VPC Prefix List RAM Share"{prefix_list_name}"')
    logger.debug(f"Response: {response}")
    logger.debug("Function return: None")
    logger.info("create_prefix_ram end")
```

>**NOTE:** All the code is available at https://github.com/sjramblings/update-aws-ip-ranges

## The End Result

With our updates, this is now our enhanced Lambda workflow


![New Enhanced Workflow](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wt41rs0ie4uoymntszx4.png)

1. Update to `arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged` will trigger the function
2. Lambda will execute under the IAM Role, retrieve its service config from AppConfig and write output to Cloudwatch
3. For enabled services, it will create/update the VPC Prefixes & WAF IPSets
4. VPC Prefixes will be shared with the Organization ID via RAM

A few screenshot of the results:-


![Resulting VPC Prefix List for CodeBuild
](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hwtq1t4vdcigieut29v5.png)

![RAM Prefix Share
](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eoj0u3o8e4pk5as2ro9v.png)

## Summary

The wonderful contributors to AWS-Samples had done a great job in getting us started. However, I needed to take it a bit further, adding the VPC Prefix Sharing and a few other tweaks.

As noted above, all code is available at https://github.com/sjramblings/update-aws-ip-ranges. I will also raise Pull Requests to backport some of this into the AWS Samples repo.

I hope this helps someone else!

Cheers