Deploy to AWS cloudformation and pass stack outputs to next steps
Asked by Neethu Philip:

Is it possible to use Bitbucket pipes to create cloudformation stacks on AWS and pass the results for processing in next steps ? This was possible in bamboo with Tasks for AWS.


Answers:
author: Igor Stoyanov
body: [@Neethu Philip](/t5/user/viewprofilepage/user-id/4175086) , [@Steffen Opel _Utoolity_](/t5/user/viewprofilepage/user-id/781567) , [@Sam Ware](/t5/user/viewprofilepage/user-id/5160854) hi.  
We released a [new version 0.16.0 of the pipe](https://bitbucket.org/atlassian/aws-cloudformation-deploy/src/0.16.0/), where we added support to store cloudformation outputs into artifacts.  
Example:

```
script:  - pipe: atlassian/aws-cloudformation-deploy:0.16.0    variables:      AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID      AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY      AWS_DEFAULT_REGION: 'us-east-1'      STACK_NAME: 'my-stack-name'      TEMPLATE: 'https://s3.amazonaws.com/cfn-deploy-pipe/cfn-template.json'      CAPABILITIES: ['CAPABILITY_IAM', 'CAPABILITY_AUTO_EXPAND']      WAIT: 'true'      WAIT_INTERVAL: 60      OUTPUTS_FILENAME: 'outputs'artifacts:  - outputs.json
```

outputs.json sample data:

```
{

   "StackId": "arn:aws:cloudformation:us-east-1:1234567890:stack/pipes-test/927c79f0-0140-11ee-82ee-0ada86db685b",
   "Outputs": [
     {
       "OutputKey": "TableName",
       "OutputValue": "pipes-test-myDynamoDBTable",
       "Description": "Table name of the newly created DynamoDB table"
     }
  ]
}
```

Regards, Igor

comments: author: Darren Mansell
body: Hi [@Igor Stoyanov](/t5/user/viewprofilepage/user-id/4550492)

Would you be able to show an example of how the OutputValue would be used in a subsequent pipe?

I'm writing the outputs as an artifact as shown above and then I need the value to pass to another pipe (atlassian/aws-cloudfront-invalidate) but it needs to be parsed e.g.  

```
$(cat outputs.json | jq -r '.Outputs[] | select(.OutputKey == "DistributionId") | .OutputValue')
```

I'm unable to run arbitrary code in the aws-cloudfront-invalidate pipe and if I do it as another command in the same step, then any env vars are lost.

Thanks, Darren


author: Igor Stoyanov
body: [@Darren Mansell](/t5/user/viewprofilepage/user-id/5368507) hi. I do not understand why any env vars are lost.  

Try example below and if you have issues with it, please describe them with details:

```
script:  - pipe: atlassian/aws-cloudformation-deploy:0.19.0    variables:      AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID      AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY      AWS_DEFAULT_REGION: 'us-east-1'      STACK_NAME: 'my-stack-name'      TEMPLATE: 'https://s3.amazonaws.com/cfn-deploy-pipe/cfn-template.json'      CAPABILITIES: ['CAPABILITY_IAM', 'CAPABILITY_AUTO_EXPAND']      WAIT: 'true'      WAIT_INTERVAL: 60      OUTPUTS_FILENAME: 'outputs'

 - DISTRIBUTION_ID=$(cat outputs.json | jq -r '.Outputs[] | select(.OutputKey == "TableName") | .OutputValue')

 - pipe: atlassian/aws-cloudfront-invalidate:0.9.0
   variables:
   AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
   AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
   AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
   DISTRIBUTION_ID: $DISTRIBUTION_ID
```

Regards, Igor


author: Darren Mansell
body: [@Igor Stoyanov](/t5/user/viewprofilepage/user-id/4550492) thank you, this did work. I didn't know it could be structured like this and I'm happily passing variables between pipes in the same step.

Cheers

Darren


author: Igor Stoyanov
body: [@Neethu Philip](/t5/user/viewprofilepage/user-id/4175086) hi. Thanks for your question.   
Currently, this is not supported in the pipe. We will investigate this and notify you if this feature will become available.  

Regards, Igor

comments: author: Sam Ware
body: Is there a place to upvote the feature?


author: Igor Stoyanov
body: [@Sam Ware](/t5/user/viewprofilepage/user-id/5160854) hi. We added this feature as a task to our backlog, so the process when feature goes from tickets to our tasks passed. Now you should wait when this feature will become available if our team will find an option of how to implement this.  

Regards, Igor


author: Igor Stoyanov
body: Hi [@Neethu Philip](/t5/user/viewprofilepage/user-id/4175086), [@Sam Ware](/t5/user/viewprofilepage/user-id/5160854) .  
From my current investigation, the only response we have according to [create-stack](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudformation/client/create_stack.html), [update-stack](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudformation/client/update_stack.html) is:

```
{
  'StackId': 'string'
}?
```

Is this what you want to be passed as results into the next steps?  

Regards, Igor

comments: author: Steffen Opel _Utoolity_
body: Hi [@Igor Stoyanov](/t5/user/viewprofilepage/user-id/4550492)

The [Tasks for AWS (Bamboo)](https://marketplace.atlassian.com/apps/1211585/tasks-for-aws-bamboo) app [@Neethu Philip](/t5/user/viewprofilepage/user-id/4175086) is referring to supports [Injecting task configuration via Bamboo variables](https://utoolity.atlassian.net/wiki/spaces/TAWS/pages/19464201/Injecting+task+configuration+via+Bamboo+variables), and *"variables are automatically defined from AWS resources affected by a task"*.

So the ask here is not about the raw API response for [create-stack](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudformation/client/create_stack.html), [update-stack](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudformation/client/update_stack.html), rather about the outcome of this API operation, which depends on the resources provisioned by the CloudFormation template, and specifically which [Outputs](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html) are defined within that template for reuse (so this is up to the template author):
> The optional `Outputs` section declares output values that you can [import into other stacks](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html) (to [create cross-stack references](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/walkthrough-crossstackref.html)), return in response (to describe stack calls), or [view on the AWS CloudFormation console](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-console-view-stack-data-resources.html). For example, you can output the S3 bucket name for a stack to make the bucket easier to find.

Accordingly, you would need to use the [describe_stacks](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudformation/client/describe_stacks.html) API to await the stack operation to succeed, and then pass on all values in the *Outputs* array to the next pipeline steps in a suitable fashion.

You can see an example within the *Variables* section of the [CloudFormation task](https://utoolity.atlassian.net/wiki/spaces/TAWS/pages/19464198/Using+the+AWS+CloudFormation+Stack+task+in+Bamboo#Variables) (abbreviated here) - the stack at hand provisioned an SQS queue and has outputs for the SQS queue URL and ARN:
> Stack 'SampleStack-TAWS-IT230-CFN2-24' generated 2 outputs:  
> ... SampleStack-TAWS-IT230-CFN2-24.outputs: QueueURL;QueueARN  
> ... SampleStack-TAWS-IT230-CFN2-24.outputs.QueueURL: <https://sqs.us-east-1.amazonaws.com/309600995652/Bamboo-24>  
> ... SampleStack-TAWS-IT230-CFN2-24.outputs.QueueARN: arn:aws:sqs:us-east-1:309600995652:Bamboo-24

Cheers,  
Steffen


Tags: aws

bitbucket-cloud

bitbucket-pipelines

cloud

cloudformation