Cloudformation templates for Cloudfront automatic cache invalidation using Lambda within CodePipeline
Tags: website, aws, cloudformation, cloudfront
Published: 2024-01-03 17:30:02

---
title: Cloudformation templates for Cloudfront automatic cache invalidation using Lambda within CodePipeline
published: true
date: 2024-01-03 19:00:00 UTC
tags: Website,AWS,Cloudformation,Cloudfront
canonical_url: 
---

(original post: https://marcoaguzzi.it/2024/01/03/lambda-invalidation-cloudformation/)

In this post I’m going to show how I triggered an automatic cache invalidation for the Cloudfront distribution that is serving this website. As in the previous posts, all the resources will be provisioned via CloudFormation.  
At the end of the post the CLI commands to create and / or update the resources will be shown.

## The manual procedure

Once that the markdown file for a post is written and a local compilation / rendering has been made, the markdown source can be pushed on the git repo. That triggers the AWS Codepipeline that will download the source, render the markdown into html, and push the result to the S3 bucket served by Cloudfront.  
Since Cloudfront is serving the S3 bucket, caching is in place. Newly pushed content won’t be visible until the cache expires, which is not feasible. So, after a successful compilation and pushing to S3, I manually get to Cloudfront distribution invalidations and fire a new invalidation. This way I’m sure that subsequent requests to the website will get the newly updated content.  
In the images below the steps for manual invalidation are shown:

Go to CloudFront / Distributions, and search for “Invalidations” tab

 ![Cloudfront invalidation manual step 1](https://marcoaguzzi.it/img/cache-invalidation-cloudformation/cloudfront_invalidation_1.png)

Then selecting the last successful invalidation (shown below on the very left) and “copy to new” (upper right)

 ![Cloudfront invalidation manual step 2](https://marcoaguzzi.it/img/cache-invalidation-cloudformation/cloudfront_invalidation_2.png)

And then confirming the copy of the invalidation with the last path (the path /\* is fine since AWS charges per invalidation, regardless of how much deep it is)

 ![Cloudfront invalidation manual step 3](https://marcoaguzzi.it/img/cache-invalidation-cloudformation/cloudfront_invalidation_3.png)

The invalidation takes a few minutes to be completed, and then the website is good to go. This is a mundane and forgetful-prone task, so I’m better automating it.

## Automation setup

There is not an “invalidate cache” action that can be directly call from CodePipeline. A Lambda that actually creates the invalidation is needed and must be called as an action in the CodePipeline structure.  
Let’s see in details the two resources:

### The Lambda function

The Lambda function will leverage boto3 Python libraries to create the invalidation and notify the pipeline about the outcome (credits to the website in the reference section).  
Let’s see some highlights. At the end of this section the link to the gist with the full source is provided.

- 
_Read cloudformation ID from environment_

- 
_Caller notifications_

- 
_Invalidation specifications_

#Click to view the Gist with the Lambda Python code<script src="https://gist.github.com/maguzzi/cc8eace18a5f0931bf317f54ab6f5dae.js"></script>
### Lambda Cloudformation stack and how to reference it

The Lambda cloudformation stack is similar to the one presented for the 301 redirects and URL rewriting in edge locations ([here’s the post](/2023/12/10/redirect-301-with-lambda/)); there are a few differences, though (at the end of the paragraph there is the gist with the full code):

- In the lambda’s resources, an environment variable is declared and its value is read from the cloudformation stack that contains the cloudfront distribution (referenced in the python code as _CLOUDFRONT\_DISTRIBUTION\_ID_).  
To be able to read that from here, the cloudformation stack that contains the cloudfront distribution has to list the variable as an output and flag it to be exported:

_Cloudfront distribution environment variable reference_

```
Parameters:
CloudformationExportVar:
Type: String
...
Environment:
Variables:
CLOUDFRONT\_DISTRIBUTION\_ID:
Fn::ImportValue:
!Ref CloudformationExportVar

```


And here’s the export in the stack hosting the cloudfront distribution

_Exported value from cloudfront distribution resource_

 
```
"Outputs": {
"CloudFrontDistributionId": {
"Description": "ID of the CloudFront distribution",
"Value": { "Fn::GetAtt": ["CloudFrontDistribution", "Id"] },
"Export": {
"Name": { "Fn::Join": ["-", [{ "Fn::Sub": "${AWS::StackName}-CloudFrontDistributionId" }, { "Ref": "Stage" }]]}
 }
 }
 }

```


- There is no need for the edge permission (only _lambda.amazonaws.com_ is needed)

_services for the AssumeRole action_

```
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action: "sts:AssumeRole"

```

- Other than the _BasicExecutionRole_, three other permissions must be granted for the creation of the invalidation and the notification back to the CodePipeline
  - cloudfront:CreateInvalidation
  - codepipeline:PutJobFailureResult
  - codepipeline:PutJobSuccessResult

_Additional permissions for basic lambda role_

```
Policies:
- PolicyName: InvalidateCloudfrontDistributionPolicy
PolicyDocument: 
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- cloudfront:CreateInvalidation
- codepipeline:PutJobFailureResult
- codepipeline:PutJobSuccessResult
Resource: "\*"

```

That should put the lambda in place for the purpose.  
Here below you can view the full gists of the cloudformation stack of

- the lambda#Click to view the Gist<script src="https://gist.github.com/maguzzi/396a32defa3b4cf66d4286fa71203264.js"></script>
- the cloudfront distribution (which is the parent stack as shown in the older articles)#Click to view the Gist<script src="https://gist.github.com/maguzzi/47473c9561292301d8669aa903698181.js"></script>

### CodePipeline stage

The action can be added in CodePipeline as a new stage. From there the lambda can be referenced. Here’s how the new stage will look like after cloudformation template has been deployed (You can see that the action is referring to the lambda and still no runs shown):

 ![Codepipeline stage invalidation new](https://marcoaguzzi.it/img/cache-invalidation-cloudformation/cloudfront_invalidation_4.png)
### Codepipeline Cloudformation stack

Let’s start from the addition of the new stage in CloudFormation. We can see

- the parameter that will reference the exported value from the lambda stack. The parameter value contains the name of the exported variable, and will reference the lambda function name
- the action
- the updated permissions in order to allow the calling of the lambda function from the pipeline:

_New stage in Codepipeline_

```
"Parameters": {
...
"InvalidationLambdaExported": {
"Description": "Lambda function performing the cache invalidation",
"Type": "String"
 }
}
... stages ...
{
"Actions": [
 {
"ActionTypeId": {
"Category": "Invoke",
"Owner": "AWS",
"Provider": "Lambda",
"Version": "1"
 },
"Configuration": {
"FunctionName": {"Fn::ImportValue":{"Ref":"InvalidationLambdaExported"}}
 },
"Name": "InvalidateCloudFrontCacheAction"
 }
 ],
"Name": "InvalidateCloudFrontCacheStage"
}
...
"Policies": [
 {
"PolicyDocument": {
"Statement": [
 {
"Action": [
...
"lambda:invokeFunction"
 ],
"Effect": "Allow",
"Resource": "\*"
 }
 ],
"Version": "2012-10-17"
 },
"PolicyName": "MyCodePipelineRolePolicy"
 }
]


The full reference to the cloudformation template can be found in the gist below:

#Click to view the Gist cloudformation for the Cloudfront distribution<script src="https://gist.github.com/maguzzi/4899697488d40105dd51ce2c37c1e327.js"></script>

Below is how the new CodePipeline stage should turn out if everything was successful:

 ![Codepipeline stage invalidation succedeed](https://marcoaguzzi.it/img/cache-invalidation-cloudformation/cloudfront_invalidation_5.png)

And that should do for adding a new stage with a new action calling the lambda and invalidating the cache

## Cloudformation CLI commands

Here’s the AWS CLI commands (legit ones!) that have been fired in order to create and / or update the cloudformation stacks (and the lambda, of course):

_CloudFront stack update_

```
aws cloudformation package --template-file marcoaguzzi.json --s3-bucket cf-templates-e5ht2sji9no7-us-east-1 --output-template-file target\packaged-template.yaml
aws cloudformation deploy --template-file target\packaged-template.yaml --stack-name marcoaguzzi-website-prod --capabilities CAPABILITY\_NAMED\_IAM --parameter-overrides Stage=prod

```

It should appear the exported output variable:  
 ![Export from CloudFront](https://marcoaguzzi.it/img/cache-invalidation-cloudformation/cloudfront_output.png)

_New lambda creation, passing the export from above_

```
cd lambda-cloudfront
compress-Archive .\index.py .\lambda-cloudfront-invalidate-prod-20240101.zip
aws s3 cp lambda-cloudfront-invalidate-prod-20240101.zip s3://lambda-artifacts-bucket-maguzzi/
aws cloudformation create-stack --stack-name lambda-invalidate-cloudfront-prod --template-body file://lambda-invalidate.yaml --capabilities CAPABILITY\_NAMED\_IAM --parameters ParameterKey=ZipDate,ParameterValue=20240101 ParameterKey=Stage,ParameterValue=prod ParameterKey=CloudformationExportVar,ParameterValue=marcoaguzzi-website-prod-CloudFrontDistributionId-prod

```

The new stack is created and the lambda name is exported in outputs:  
 ![Export from Lambda](https://marcoaguzzi.it/img/cache-invalidation-cloudformation/lambda_output.png)

_CodePipeline update referencing the lambda exported variable in parameters_

```
aws cloudformation update-stack --stack-name marcoaguzzi-stack-codepipeline-prod --template-body file://marcoaguzzi-codepipeline.json --parameters file://parameters-codepipeline-prod.json --capabilities 
CAPABILITY\_IAM

```

And now the Codepipeline should have the last stage as shown in the pictures above, ready to invalidate the cache after the website deploy to S3 :-)

## References

- [AWS: Creating a CloudFront Invalidation in CodePipeline using Lambda Actions](https://medium.com/fullstackai/aws-creating-a-cloudfront-invalidation-in-codepipeline-using-lambda-actions-49c1fd3a3c31)
- [AWS Cloudformation user guide](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html)