How to use OpenID Connect with multiple AWS accounts
Asked by Aaron Durant:

I have multiple AWS accounts for different stages (i.e. One for development, one for staging, and one for production)

In each account, I have created an OpenID Connect Identity Provider, using the URL and audience found in my Organisations BitBucket account, and an associated IAM Role. There are the same in every AWS account, except the name of the IAM Role changes to reflect the stage.

I have a BitBucket pipeline in a repository to deploy the application to AWS. I have set Deployment variables in the repository settings for \`CDK_DEFAULT_ACCOUNT\` and \`AWS_ROLE_ARN\` to reflect the different accounts I want to deploy to for each stage.

```
branches:
  staging:
    - stage:
        name: Build and Deploy Staging
        deployment: Staging
        steps:
          - step: *install_dependencies
          - step: *test
          - step: *build
          - step: *deploy
  development:
    - stage:
        name: Build and Deploy Development
        deployment: Development
        steps:
          - step: *install_dependencies
          - step: *test
          - step: *build
          - step: *deploy
```

When I run the pipeline on the \`development\` branch, it runs correctly and deploys to AWS.  
When I run the pipeline on the \`staging\` branch, I get

```
Building Assets Failed: Error: Need to perform AWS calls for account XXXXXXXXXX, but no credentials have been configured
```

despite having the same variables defined.

Is there a limitation on OpenID Connect and multiple AWS accounts, or some extra configuration needed? Or simply something set up incorrectly in my pipeline?


Answers:
author: Aaron Durant
body: I've managed to solve the problem - the issue was that the IAM Role associated with the OpenID Connect Provider in the staging account had permission for \`sts:AssumeRole\`, instead of the required \`sts:AssumeRoleWithWebIdentity\`

I added -vv and --debug flags to the cdk command, which provided more details on the authentication error

comments: author: Igor Stoyanov
body: [@Aaron Durant](/t5/user/viewprofilepage/user-id/5142744) Great! You could mark your answer as Accepted, so the other users could see that your case has solution.  

Regards, Igor


author: Igor Stoyanov
body: [@Aaron Durant](/t5/user/viewprofilepage/user-id/5142744) hi. Check [this guide](https://support.atlassian.com/bitbucket-cloud/docs/deploy-on-aws-using-bitbucket-pipelines-openid-connect/).

```
 - step:
     oidc: true
     script:
       - export AWS_REGION=us-west-2
       - export AWS_ROLE_ARN=arn:aws:iam::XXXXXXXXXXXX:role/oidc-demo
       - export AWS_WEB_IDENTITY_TOKEN_FILE=$(pwd)/web-identity-token
       - echo $BITBUCKET_STEP_OIDC_TOKEN > $(pwd)/web-identity-token
       - aws s3 cp s3://bucket/XXXXXX ./XXXXXXX
```

Regards, Igor

comments: author: Aaron Durant
body: Sorry, I should have posted my deploy step as well  

```
- step: &deploy
    name: Deploy
    caches:
      - node
    oidc: true
    script:
      - npm install
      - export AWS_WEB_IDENTITY_TOKEN_FILE=$(pwd)/web-identity-token
      - echo $BITBUCKET_STEP_OIDC_TOKEN > $(pwd)/web-identity-token
      - npx cdk deploy
```

\`AWS_REGION\` and \`AWS_ROLE_ARN\` are defined in pipeline/deployment variables. Do I have to explicitly export them inside the script?


author: Igor Stoyanov
body: [@Aaron Durant](/t5/user/viewprofilepage/user-id/5142744) variables defined in pipeline/deployment variables should be available without export. You could test this case with \`echo \<any unsecured variable\>\` command.

Check the [page for bitbucket variables and secrets](https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/)

Regards, Igor


author: Aaron Durant
body: Thanks Igor.   
I have printed all the variables involved for each Deployment, and verified they are correct, but the AWS credentials are still not recognised for the second account.  

Is there anything else I can try?

Thank you


author: Igor Stoyanov
body: [@Aaron Durant](/t5/user/viewprofilepage/user-id/5142744) hi. Please check the same for \`staging\` branch, where you investigated the error.

Regards, Igor


author: Igor Stoyanov
body: Sorry, i read Development instead of Deployment.


author: Aaron Durant
body: I have verified the variables on both the \`staging\` and \`development\` branches, and in both cases they are correct


author: Igor Stoyanov
body: [@Aaron Durant](/t5/user/viewprofilepage/user-id/5142744) i think you should dive into this:  
<https://github.com/aws/aws-cdk/issues/20935>,

especially:
> the CDK CLI does not have the capability to automatically pick the correct profile for the configured stack environment. The CLI needs some set of credentials to start with, and if you do not provide the `--profile` then it will assume that you are using the `default` profile.
>
> If you have [bootstrapped](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html#getting_started_bootstrap) your accounts then there will be a set of IAM roles that have been created in those accounts. When you run a CDK command, the CLI will use your configured credentials to assume the bootstrapped roles in each account.
>
> So for example if you run `npx cdk synth --profile=dev --all` it will use the credentials from your `dev` profile and will first assume a specific role in the target account.


Tags: aws

bitbucket-cloud

cloud

openid