CloudFormation stack creation using Python
Tags: python, aws, tutorial, devops
Published: 2021-08-12 04:25:34

CloudFormation stack can be created from AWS Console, AWS CLI or using many other ways. We can also automate the creation of the CloudFormation stack using AWS CLI, CodePipeline etc. In this section of my blog, I am going to introduce how to use an AWS SDK for Python to automate the CloudFormation Stack creation. Here I am assuming that you know the basic Python and have understanding of the AWS SDK for Python. I would suggest to test the below scripts with small stack first and further you can customise the Python script for your own requirements.

Follow the steps below to automate the Cloudformation stack creation.

1. Make sure you have the latest Python installed on your box where you are intending to run the Python Script. In case of Mac it comes with default installed Python. I have upgraded it to the latest version.
    ```
    rimpaljohal@MacBook-Pro Blog-Contents % Python --version
    Python 3.9.6

    ```
2. Get hold of the AWS SDK for Python and installed it on the box where you are going to execute the Python Script.

    [AWS SDK for Python (Boto3)] (https://aws.amazon.com/sdk-for-python/)

3. You need two files for automate creation of CloudFormation stack. 
     *CFNStackCreation.py* --> Your Python Script
     *Parameter.json*      --> Your Parameter file

4. Keep these two files in the same directory path from where you are going to execute the Python script. For example
     ```
     rimpaljohal@MacBook-Pro Rimpal % ls CFNStackCreation.py Parameter.json 
CFNStackCreation.py	Parameter.json
     ```
5. Python Script below for reference
     ```
 #-- Import modules
import sys
import os.path
import json
import time
import boto3

     #-- Functions
def check_status( cfn_client_ss, cfn_stack_name_ss ):
    stacks_ss = cfn_client_ss.describe_stacks(StackName=cfn_stack_name_ss)["Stacks"]
    stack_ss_val = stacks_ss[0]
    status_cur_ss = stack_ss_val["StackStatus"]
    print ("Current status of stack " + stack_ss_val["StackName"] + ": " + status_cur_ss)
    for ln_loop in range(1, 9999):
        if "IN_PROGRESS" in status_cur_ss:
            print ("\rWaiting for status update(" + str(ln_loop) + ")...",)
            time.sleep(5) # pause 5 seconds

            try:
                stacks_ss = cfn_client_ss.describe_stacks(StackName=cfn_stack_name_ss)["Stacks"]
            except:
                print (" ")
                print ("Stack " + stack_ss_val["StackName"] + " no longer exists")
                status_cur_ss = "STACK_DELETED"
                break

            stack_ss_val = stacks_ss[0]

            if stack_ss_val["StackStatus"] != status_cur_ss:
                status_cur_ss = stack_ss_val["StackStatus"]
                print (" ")
                print ("Updated status of stack " + stack_ss_val["StackName"] + ": " + status_cur_ss)
        else:
            break

    return status_cur_ss
#-- End Functions


     #-- Main program
def main(access_key_ss, secret_key_ss, param_file_ss):

    #-- Confirm parameters file exists
    if os.path.isfile(param_file_ss):
        json_data_ss=open(param_file_ss).read()
    else:
        print ("Parameters file: " + param_file_ss + " is invalid!")
        print (" ")
        sys.exit(3)

    print ("Parameters file: " + param_file_ss)
    parameters_data_ss = json.loads(json_data_ss)
    region_ss = parameters_data_ss["RegionId"]

    #-- Connect to AWS region specified in parameters file
    print ("Connecting to region: " + region_ss)
    cfn_client_ss = boto3.client('cloudformation', region_ss, aws_access_key_id=access_key_ss, aws_secret_access_key=secret_key_ss)

    #-- Store parameters from file into local variables
    cfn_stack_name_ss = parameters_data_ss["StackName"]
    print ("You are deploying stack: " + cfn_stack_name_ss)
    #-- Check if this stack name already exists
    stack_list_ss = cfn_client_ss.describe_stacks()["Stacks"]
    stack_exists_ss = False
    for stack_ss_cf in stack_list_ss:
        if cfn_stack_name_ss == stack_ss_cf["StackName"]:
            print ("Stack " + cfn_stack_name_ss + " already exists.")
            stack_exists_ss = True
    
    

    #-- If the stack already exists then delete it first
    if stack_exists_ss:
        user_response = input ("Do you want to delete or update the stack")
        print ("Calling Delete Stack API for " + cfn_stack_name_ss)
        cfn_client_ss.delete_stack(StackName=cfn_stack_name_ss)

        #-- Check the status of the stack deletion
        check_status(cfn_client_ss, cfn_stack_name_ss)

    print (" ")
    print ("Loading parameters from parameters file:")
    fetch_stack_parameters_ss = []
    for key_ss in parameters_data_ss.keys():
        if key_ss == "TemplateUrl":
            template_url_ss = parameters_data_ss["TemplateUrl"]
        elif key_ss == "StackName" or key_ss == "RegionId":
            # -- Do not send as parameters
            print (key_ss + " - "+ parameters_data_ss[key_ss] + " (not sent as parameter)")
        else:
            print (key_ss + " - "+ parameters_data_ss[key_ss])
            fetch_stack_parameters_ss.append({"ParameterKey": key_ss, "ParameterValue": parameters_data_ss[key_ss]})

    #-- Call CloudFormation API to create the stack   TemplateBody='', 
    print (" ")
    print ("Calling CREATE_STACK method to create: " + cfn_stack_name_ss)

    status_cur_ss = ""

    result_ss = cfn_client_ss.create_stack(StackName=cfn_stack_name_ss, DisableRollback=True, TemplateURL=template_url_ss, Parameters=fetch_stack_parameters_ss, Capabilities=["CAPABILITY_NAMED_IAM"])
    print ("Output from API call: ")
    print (result_ss)
    print (" ")

    #-- Check the status of the stack creation
    status_cur_ss = check_status( cfn_client_ss, cfn_stack_name_ss )

    if status_cur_ss == "CREATE_COMPLETE":
        print ("Stack " + cfn_stack_name_ss + " created successfully.")
    else:
        print ("Failed to create stack " + cfn_stack_name_ss)
        sys.exit(1)


    #-- Call Main program
if __name__ == "__main__":
    if len(sys.argv) < 4:
        print ("%s:  Error: %s\n" % (sys.argv[0], "Not enough command options given"))
        print ("Argument 1 (required): AWS Access Key ")
        print ("Argument 2 (required): AWS Secret Access Key ")
        print ("Argument 3 (required): Stack Parameters JSON file ")
        print (" ")
        sys.exit(3)
    else:
        access_key_ss = sys.argv[1]
        secret_key_ss = sys.argv[2]
        param_file_ss = sys.argv[3]

    main(access_key_ss, secret_key_ss, param_file_ss)
   
     ```
6. Parameter json file below for reference. This is the template parameter json file and one can customise it as per the cloudformation build requirement. I am calling master stack here through this parameter file. Master stack is having the nested stacks to create VPC, Subnets(private and public, Database Subnets), application ec2 instances with autoscaling and Load Balancer. 
      ```
     {
    "RegionId": "us-east-1",
    "TemplateUrl": "https://blogbucket.s3.amazonaws.com/masterstackv1.yml",
    "StackName": "nonprodstack",
    "EnvironmentType": "dev",
    "VpcCIDR": "10.192.0.0/16",
    "PublicSubnet1CIDR": "10.192.10.0/24",
    "PublicSubnet2CIDR": "10.192.11.0/24",
    "PublicSubnet3CIDR": "10.192.12.0/24",
    "PrivateSubnet1CIDR": "10.192.20.0/24",
    "PrivateSubnet2CIDR": "10.192.22.0/24",
    "PrivateSubnet3CIDR": "10.192.24.0/24",
    "DatabaseSubnet1CIDR": "10.192.21.0/24",
    "DatabaseSubnet2CIDR": "10.192.23.0/24",
    "DatabaseSubnet3CIDR": "10.192.25.0/24",
    "LatestAmiID": "/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2",
    "ALBAccessCIDR": "0.0.0.0/0",
    "ServerInstanceType": "t2.micro",
    "WebAsgMax": "4",
    "WebAsgMin": "3"
    }
     ```

7. Once the required two files(Python script and the Parameter file) are saved in one directory then open the terminal and navigate/go to the directory which is having two mentioned files.

8. Execute the python scripts as given below

      ```
       rimpaljohal@MacBook-Pro Rimpal % python CFNStackCreation.py AU************* SL*********************** Parameter.json
      ```

    Here you need to give AWS Access Key and AWS Secret Access Key of your AWS account.

9. As the script progress, you can see the progress on the Mac terminal.


Above defined steps will help you to deploy the CloudFormation stack using python script.

Happy Coding!!