Managing Rolling Updates in AWS Auto Scaling Groups with AWS CloudFormation Update Policy: Part 2
Tags: tutorial, aws, iac, devops
Published: 2023-07-13 00:35:44

This is the concluding part of this blog post. Visit [this](https://dev.to/obabawale/managing-rolling-updates-in-aws-autoscaling-groups-with-aws-cloudformation-update-policy-part-1-3jhj) post to see the foundational part.

In this post, we create a cloud infrastructure using two cross-linked CloudFormation stacks: the `network` stack deploys the networking infrastructure, while the `servers` stack deploys the servers which are managed by an Auto Scaling Group (ASG). The purpose of this post is to demonstrate the use of `UpdatePolicy` to guarantee a smooth roll out of changes to the CloudFormation Template. 

We will walkthrough this in sections as outlined below:
1. Description of Files and Resources used.
2. Stack update behavior without an Update policy.
3. Stack update behavior with an Update policy
4. Clean up the AWS environment.

## **1. Description of Files and Resources used**
* The files and commands used can be found in this GitHub [repo](https://github.com/Babawale/DevBlogPost/tree/main). The README.md file gives a full description.
   * The resources created by the templates include:
   * A Virtual Private Cloud (VPC) with four subnets in two 
     availability zones (AZs)
   * The subnets are a pair of Public and Private subnets.
   * Two NAT Gateways
   * A Load Balancer
   * Auto Scaling Group
   * Amazon Linux AMIs
AWS Cloud9 is used as the IDE. See this [post](https://dev.to/obabawale/getting-started-with-aws-cloud9-h02) to learn how to set up a Cloud9 environment and learn some of its benefits.

The only difference between the two template files are:
1. Updating the `UserData` of the Launch Template to install `cfn-signals` and send signals.
``` yaml
WebAppLaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties:
      LaunchTemplateName: !Sub ${AWS::StackName}-launch-template
      LaunchTemplateData:
          UserData:
            Fn::Base64: !Sub |
              #!/bin/bash -xe
              yum update -y
              yum upgrade -y
              yum install -y aws-cfn-bootstrap
              yum install -y httpd
              systemctl enable httpd
              systemctl start httpd
              echo "<html><head><title>Blog Post Demo Test Page</title></head><body><h1><center>"This is Version 2.0"</center></h1></body></html>"  > /var/www/html/index.html
              /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource AppAutoScaleGroup --region ${AWS::Region}

```
  2. Updating the ASG resource definition with an `UpdatePolicy`
``` yaml
  AppAutoScaleGroup: 
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      VPCZoneIdentifier:
      - Fn::ImportValue: 
          !Sub "${EnvironmentName}-PRIV-NETS"
      LaunchTemplate:
        LaunchTemplateId:
            Ref: WebAppLaunchTemplate
        Version: !GetAtt WebAppLaunchTemplate.LatestVersionNumber
      MinSize: '2'
      MaxSize: '4'
      DesiredCapacity: "2"
      TargetGroupARNs:
      - Ref: WebAppTargetGroup
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MinInstancesInService: '1'
        MaxBatchSize: '1'
        PauseTime: PT12M5S
        WaitOnResourceSignals: true
        SuspendProcesses:
          - HealthCheck
          - ReplaceUnhealthy
          - AZRebalance
          - AlarmNotification
          - ScheduledActions
```


## **2. Stack update behavior without an Update policy**
Next, we examine the behavior of a stack without a defined update policy.

**Step 1:** We create the network stack using the helper scripts from the Cloud9 terminal 
```./create.sh infra network-infra.yaml network-infra-params.json us-east-1```

Wait for the operation to complete before proceeding to the next step.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w02maky30rnvvgqjvgkf.png)


**Step 2:** We create the servers stack 
```./create.sh servers servers-without-update-policy.yaml servers-params.json us-east-1```

Wait for the operation to complete and assess the DNS of the load balancer under the outputs tab.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/27wsumigzd8ki30w8rpp.png)
Open the link in a new tab to see the web page as shown below.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ir2i01te0yog4sc1a3bf.png)
This shows Version 1.0 of our deployment.

**Step 3:** Change the version number in the `UserData` section of `servers-without-update-policy.yaml` 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9m2wlkldsmxgtsvine93.png)
And update the stack with 
```./update.sh servers servers-without-update-policy.yaml servers-params.json us-east-1```

Observe that the update does not initiate a stack update. This is because a change in the `UserData` section does not cause a stack to update. The `UserData` is run only once during creation. 
The only way to force this update is to manually delete the instances in the Auto Scaling Group, thus forcing the ASG to pick up the new configuration and run the `UserData` again.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r2qhedu0dru30rcmqrw3.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zs3he4o8ootpyve29921.png)
This may case a disruption and make the application unavailable as shown below.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q7wdihbucvsg3ittt063.png)

New instances are created by the ASG and the service will become available once again.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hhygaloi8ds804wmqxm7.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2s6x1i5lfkzu02mzcrg4.png)

## **3. Stack update behavior with an Update policy**
Now, let's examine how an update policy impacts the same deployment.
```yaml
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MinInstancesInService: '1'
        MaxBatchSize: '1'
        PauseTime: PT12M5S
        WaitOnResourceSignals: true
        SuspendProcesses:
          - HealthCheck
          - ReplaceUnhealthy
          - AZRebalance
          - AlarmNotification
          - ScheduledActions
```
The `SuspendProcesses` parameter prevents other process from interfering with the update process. The other parameters have been explained in the first post.

**Step 1:** Delete the `servers` stack `./delete.sh servers us-east-1`. Wait for completion before proceeding.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iuxjtcf2y01urr5o7fdk.png)
**Step 2:** Create a new `servers` stack using the `servers-with-update-policy.yaml` template file `./create.sh servers servers-with-update-policy.yaml servers-params.json us-east-1`

Wait for completion and visit the web page using the DNS name of the load balancer.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s7wd57sl1n9zm5cwnwgg.png)
Observe the Instance ID of the instances as shown here.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5ak3twp1prov812vysdl.png)
This shows Version 1.0 of our deployment.

**Step 3:** Change the version number in the `UserData` section of `servers-with-update-policy.yaml` to `2.0`. And update the stack with `./update.sh servers servers-with-update-policy.yaml servers-params.json us-east-1`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0m6au32lsukhcz12eozw.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/grg9nc42qy3tidg0gydh.png)
The images above show the update process of the `servers` stack. The "Status reason" column is worth noting. It shows:
1. A rolling update with a batch size of 1, and at least one instance in service during the update.
2. CloudFormation will wait 12 minutes and 5 secs to receive success signals from the instance before proceeding with the next batch update.

**Step 4:** Refreshing the page during the update shows switches between Version 1.0 and Version 2.0 until all the instances are replaced with Version 2.0.

The rolling update ensures that the application remains available the entire time as defined in the `UpdatePolicy`.


## **4. Clean up the AWS environment.**

Finally, remember to clean up the AWS environment by first deleting the `servers` stack,

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/boq6k080hqr07yrsnd4a.png)
and then `network` stack once the previous stack is deleted.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3gg01a15v1w6qzh1ci5a.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vmt062yn7ii9fqgxwbt0.png)

In conclusion, taking control of the update behavior of your infrastructure using AWS CloudFormation is necessary to ensure a predictable and expected update process at all times.





> Thank you for taking the time to read this blog post. If you find the article interesting, do like the article, and comment. Also connect with me on [LinkedIn](https://www.linkedin.com/in/obabawale).