Question: How do I use custom resources with Amazon S3 buckets in CloudFormation?

I want to use custom resources with Amazon Simple Storage Service (Amazon S3) buckets in AWS CloudFormation.

Short descriptionUse the CloudFormation template in the following resolution to use custom resources with an S3 bucket.Consider the following:The template allows you to create folders in S3 buckets. Amazon S3 has a flat structure, but supports the folder concept as a means of grouping objects.The template allows you to perform operations after creating an S3 bucket, including copying content, uploading content, and synchronizing two different buckets.You can modify the template with your own code.This template uses AWS Lambda-backed custom resources and assumes that you're familiar with Lambda best practices and troubleshooting issues.Note: In the following resolution, all the S3 bucket content is deleted when the CloudFormation stack is deleted. You can modify this behavior by modifying the Lambda code.

ResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, then make sure that you’re using the most recent AWS CLI version.Get the CloudFormation templateTo create folders in an S3 bucket using CloudFormation, save the following JSON or YAML template:JSON:{  "AWSTemplateFormatVersion": "2010-09-09",  "Description": "Working with custom resources and S3",  "Parameters": {    "S3BucketName": {      "Type": "String",      "Description": "S3 bucket to create.",      "AllowedPattern": "[a-zA-Z][a-zA-Z0-9_-]*"    },    "DirsToCreate": {      "Description": "Comma delimited list of directories to create.",      "Type": "CommaDelimitedList"    }  },  "Resources": {    "SampleS3Bucket": {      "Type": "AWS::S3::Bucket",      "Properties": {        "BucketName": {"Ref":"S3BucketName"}      }    },    "S3CustomResource": {      "Type": "Custom::S3CustomResource",      "DependsOn":"AWSLambdaExecutionRole",      "Properties": {        "ServiceToken": {"Fn::GetAtt": ["AWSLambdaFunction","Arn"]},        "the_bucket": {"Ref":"SampleS3Bucket"},        "dirs_to_create": {"Ref":"DirsToCreate"}      }    },    "AWSLambdaFunction": {      "Type": "AWS::Lambda::Function",      "Properties": {        "Description": "Work with S3 Buckets!",        "FunctionName": {"Fn::Sub":"${AWS::StackName}-${AWS::Region}-lambda"},        "Handler": "index.handler",        "Role": {"Fn::GetAtt": ["AWSLambdaExecutionRole","Arn"]},        "Timeout": 360,        "Runtime": "python3.9",        "Code": {          "ZipFile": "import boto3\r\nimport cfnresponse\r\ndef handler(event, context):\r\n    # Init ...\r\n    the_event = event['RequestType']\r\n    print(\"The event is: \", str(the_event))\r\n    response_data = {}\r\n    s_3 = boto3.client('s3')\r\n    # Retrieve parameters\r\n    the_bucket = event['ResourceProperties']['the_bucket']\r\n    dirs_to_create = event['ResourceProperties']['dirs_to_create']\r\n    try:\r\n        if the_event in ('Create', 'Update'):\r\n            print(\"Requested folders: \", str(dirs_to_create))\r\n            for dir_name in dirs_to_create:\r\n                print(\"Creating: \", str(dir_name))\r\n                s_3.put_object(Bucket=the_bucket,\r\n                                Key=(dir_name\r\n                                    + '\/'))\r\n        elif the_event == 'Delete':\r\n            print(\"Deleting S3 content...\")\r\n            b_operator = boto3.resource('s3')\r\n            b_operator.Bucket(str(the_bucket)).objects.all().delete()\r\n        # Everything OK... send the signal back\r\n        print(\"Operation successful!\")\r\n        cfnresponse.send(event,\r\n                        context,\r\n                        cfnresponse.SUCCESS,\r\n                        response_data)\r\n    except Exception as e:\r\n        print(\"Operation failed...\")\r\n        print(str(e))\r\n        response_data['Data'] = str(e)\r\n        cfnresponse.send(event,\r\n                        context,\r\n                        cfnresponse.FAILED,\r\n                        response_data)"        }      }    },    "AWSLambdaExecutionRole": {      "Type": "AWS::IAM::Role",      "Properties": {        "AssumeRolePolicyDocument": {          "Statement": [            {              "Action": [                "sts:AssumeRole"              ],              "Effect": "Allow",              "Principal": {                "Service": [                  "lambda.amazonaws.com"                ]              }            }          ],          "Version": "2012-10-17"        },        "Path": "/",        "Policies": [          {            "PolicyDocument": {              "Statement": [                {                  "Action": [                    "logs:CreateLogGroup",                    "logs:CreateLogStream",                    "logs:PutLogEvents"                  ],                  "Effect": "Allow",                  "Resource": "arn:aws:logs:*:*:*"                }              ],              "Version": "2012-10-17"            },            "PolicyName": {"Fn::Sub": "${AWS::StackName}-${AWS::Region}-AWSLambda-CW"}          },          {            "PolicyDocument": {              "Statement": [                {                  "Action": [                    "s3:PutObject",                    "s3:DeleteObject",                    "s3:List*"                  ],                  "Effect": "Allow",                  "Resource": [                    {"Fn::Sub": "arn:aws:s3:::${SampleS3Bucket}/*"},                    {"Fn::Sub": "arn:aws:s3:::${SampleS3Bucket}"}                  ]                }              ],              "Version": "2012-10-17"            },            "PolicyName": {"Fn::Sub":"${AWS::StackName}-${AWS::Region}-AWSLambda-S3"}          }        ],        "RoleName": {"Fn::Sub":"${AWS::StackName}-${AWS::Region}-AWSLambdaExecutionRole"}      }    }  }}YAML:Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.Permission is hereby granted, free of charge, to any person obtaining a copy of thissoftware and associated documentation files (the "Software"), to deal in the Softwarewithout restriction, including without limitation the rights to use, copy, modify,merge, publish, distribute, sublicense, and/or sell copies of the Software, and topermit persons to whom the Software is furnished to do so.THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR APARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHTHOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTIONOF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THESOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.AWSTemplateFormatVersion: 2010-09-09Description: Working with custom resources and S3Parameters:  S3BucketName:    Type: String    Description: "S3 bucket to create."    AllowedPattern: "[a-zA-Z][a-zA-Z0-9_-]*"  DirsToCreate:    Description: "Comma delimited list of directories to create."    Type: CommaDelimitedListResources:  SampleS3Bucket:    Type: AWS::S3::Bucket    Properties:      BucketName: !Ref S3BucketName  S3CustomResource:    Type: Custom::S3CustomResource    Properties:      ServiceToken: !GetAtt AWSLambdaFunction.Arn      the_bucket: !Ref SampleS3Bucket      dirs_to_create: !Ref DirsToCreate  AWSLambdaFunction:     Type: "AWS::Lambda::Function"     Properties:       Description: "Work with S3 Buckets!"       FunctionName: !Sub '${AWS::StackName}-${AWS::Region}-lambda'       Handler: index.handler       Role: !GetAtt AWSLambdaExecutionRole.Arn       Timeout: 360       Runtime: python3.9       Code:         ZipFile: |          import boto3          import cfnresponse          def handler(event, context):              # Init ...              the_event = event['RequestType']              print("The event is: ", str(the_event))              response_data = {}              s_3 = boto3.client('s3')              # Retrieve parameters              the_bucket = event['ResourceProperties']['the_bucket']              dirs_to_create = event['ResourceProperties']['dirs_to_create']              try:                  if the_event in ('Create', 'Update'):                      print("Requested folders: ", str(dirs_to_create))                      for dir_name in dirs_to_create:                          print("Creating: ", str(dir_name))                          s_3.put_object(Bucket=the_bucket,                                         Key=(dir_name                                              + '/'))                  elif the_event == 'Delete':                      print("Deleting S3 content...")                      b_operator = boto3.resource('s3')                      b_operator.Bucket(str(the_bucket)).objects.all().delete()                  # Everything OK... send the signal back                  print("Operation successful!")                  cfnresponse.send(event,                                   context,                                   cfnresponse.SUCCESS,                                   response_data)              except Exception as e:                  print("Operation failed...")                  print(str(e))                  response_data['Data'] = str(e)                  cfnresponse.send(event,                                   context,                                   cfnresponse.FAILED,                                   response_data)  AWSLambdaExecutionRole:     Type: AWS::IAM::Role     Properties:       AssumeRolePolicyDocument:         Statement:         - Action:           - sts:AssumeRole           Effect: Allow           Principal:             Service:             - lambda.amazonaws.com         Version: '2012-10-17'       Path: "/"       Policies:       - PolicyDocument:           Statement:           - Action:             - logs:CreateLogGroup             - logs:CreateLogStream             - logs:PutLogEvents             Effect: Allow             Resource: arn:aws:logs:*:*:*           Version: '2012-10-17'         PolicyName: !Sub ${AWS::StackName}-${AWS::Region}-AWSLambda-CW       - PolicyDocument:           Statement:           - Action:             - s3:PutObject             - s3:DeleteObject             - s3:List*             Effect: Allow             Resource:             - !Sub arn:aws:s3:::${SampleS3Bucket}/*             - !Sub arn:aws:s3:::${SampleS3Bucket}           Version: '2012-10-17'         PolicyName: !Sub ${AWS::StackName}-${AWS::Region}-AWSLambda-S3       RoleName: !Sub ${AWS::StackName}-${AWS::Region}-AWSLambdaExecutionRoleDeploy your CloudFormation templateYou can deploy your CloudFormation template using either the CloudFormation console or AWS CLI.CloudFormation console:1.    Open the CloudFormation console.2.    Choose Create stack, and then choose With new resources (standard).3.    In the Specify template section, choose Upload a template file.4.    Choose Choose file, select the template that you downloaded in step 1, and then choose Next.5.    In the Parameters section, for S3BucketName, choose your S3 bucket.6.    For DirsToCreate, enter a comma-delimited list of folders and subfolders that you want to create.Note: For example, you can enter dir_1,dir_2/sub_dir_2,dir_3 as a list.7.    Complete the rest of the steps in the setup wizard, and then choose Create stack.AWS CLI:1.    Give your downloaded template the following name: custom-resource-lmabda-s3.template2.    Open a command line in your operating system, and then go to the folder where the template is located.3.    Run the following command:aws cloudformation create-stack \                   --timeout-in-minutes 10 \                   --disable-rollback \                   --stack-name testing-custom-resource-s3 \                   --template-body file://custom-resource-lmabda-s3.template \                   --capabilities CAPABILITY_NAMED_IAM \                   --parameters \                   ParameterKey=DirsToCreate,ParameterValue="dir_1\,dir_2/sub_dir_2\,dir_3" \                   ParameterKey=S3BucketName,ParameterValue="test-bucket-custom-resource"Note: Update parameters with your values.