Package: cdk-pipelines-github
CDK Pipelines for GitHub WorkflowsThe APIs in this module are experimental and under active development.
They are subject to non-backward compatible changes or removal in any future version. These are
not subject to theSemantic Versioningmodel and breaking changes will be
announced in the release notes. This means that while you may use them, you may need to update
your source code when upgrading to a newer version of this package.A construct library for painless Continuous Delivery of CDK applications,
deployed viaGitHub Workflows.The CDK already has a CI/CD solution,CDK Pipelines,
which creates an AWS CodePipeline that deploys CDK applications. This module
serves the same surface area, except that it is implemented with GitHub
Workflows.Table of ContentsCDK Pipelines for GitHub WorkflowsTable of ContentsUsageInitial SetupAWS CredentialsGitHub Action RoleGitHubActionRoleConstructGitHub SecretsRunners with Preconfigured CredentialsUsing Docker in the PipelineAuthenticating to Docker registriesRunner TypesGitHub Hosted RunnerSelf Hosted RunnerEscape HatchesAdditional FeaturesGitHub Action StepConfigure GitHub EnvironmentWaves for Parallel BuildsManual Approval StepPipeline YAML CommentsCommon Configuration for Docker Asset Publishing StepsTutorialNot supported yetContributingLicenseUsageAssuming you have aStagecalledMyStagethat includes CDK stacks for your app and you want to deploy it
to two AWS environments (BETA_ENVandPROD_ENV):fromaws_cdk.pipelinesimportShellStepapp=App()pipeline=GitHubWorkflow(app,"Pipeline",synth=ShellStep("Build",commands=["yarn install","yarn build"]),aws_creds=AwsCredentials.from_open_id_connect(git_hub_action_role_arn="arn:aws:iam::<account-id>:role/GitHubActionRole"))# Build the stagesbeta_stage=MyStage(app,"Beta",env=BETA_ENV)prod_stage=MyStage(app,"Prod",env=PROD_ENV)# Add the stages for sequential build - earlier stages failing will stop later ones:pipeline.add_stage(beta_stage)pipeline.add_stage(prod_stage)# OR add the stages for parallel building of multiple stages with a Wave:wave=pipeline.add_wave("Wave")wave.add_stage(beta_stage)wave.add_stage(prod_stage)app.synth()When you runcdk synth, adeploy.ymlworkflow will be created under.github/workflowsin your repo. This workflow will deploy your application
based on the definition of the pipeline. In the example above, it will deploy
the two stages in sequence, and within each stage, it will deploy all the
stacks according to their dependency order and maximum parallelism. If your app
uses assets, assets will be published to the relevant destination environment.ThePipelineclass fromcdk-pipelines-githubis derived from the base CDK
Pipelines class, so most features should be supported out of the box. See theCDK Pipelinesdocumentation for more details.To express GitHub-specifc details, such as those outlined inAdditional Features, you have a few options:Use aGitHubStageinstead ofStage(or make aGitHubStagesubclass instead of aStagesubclass) - this adds theGitHubCommonPropsto theStagepropertiesWith this you can usepipeline.addStage(myGitHubStage)orwave.addStage(myGitHubStage)and the properties of the
stage will be usedUsing aStage(or subclass thereof) or aGitHubStage(or subclass thereof) you can callpipeline.addStageWithGitHubOptions(stage, stageOptions)orwave.addStageWithGitHubOptions(stage, stageOptions)In this case you're providing the same options along with the stage instead of embedded in the stage.Note that properties of aGitHubStageadded withaddStageWithGitHubOptions()will override the options provided toaddStageWithGitHubOptions()NOTES:Environments must be bootstrapped separately usingcdk bootstrap. SeeCDK
Environment
Bootstrappingfor details.Initial SetupAssuming you have your CDK app checked out on your local machine, here are the suggested steps
to develop your GitHub Workflow.Set up AWS Credentials your local environment. It is highly recommended to authenticate via an OpenId
Connect IAM Role. You can set one up using theGithubActionRoleclass provided
in this module. For more information (and alternatives), seeAWS Credentials.When you've updated your pipeline and are ready to deploy, runcdk synth. This creates a workflow file
in.github/workflows/deploy.yml.When you are ready to test your pipeline, commit your code changes as well as thedeploy.ymlfile to
GitHub. GitHub will automatically try to run the workflow found under.github/workflows/deploy.yml.You will be able to see the result of the run on theActionstab in your repository:For an in-depth run-through on creating your own GitHub Workflow, see theTutorialsection.AWS CredentialsThere are two ways to supply AWS credentials to the workflow:GitHub Action IAM Role (recommended).Long-lived AWS Credentials stored in GitHub Secrets.The GitHub Action IAM Role authenticates via the GitHub OpenID Connect provider
and is recommended, but it requires preparing your AWS account beforehand. This
approach allows your Workflow to exchange short-lived tokens directly from AWS.
With OIDC, benefits include:No cloud secrets.Authentication and authorization management.Rotating credentials.You can read morehere.GitHub Action RoleAuthenticating via OpenId Connect means you do not need to store long-lived
credentials as GitHub Secrets. With OIDC, you provide a pre-provisioned IAM
role with optional role session name to your GitHub Workflow via theawsCreds.fromOpenIdConnectAPI:fromaws_cdk.pipelinesimportShellStepapp=App()pipeline=GitHubWorkflow(app,"Pipeline",synth=ShellStep("Build",commands=["yarn install","yarn build"]),aws_creds=AwsCredentials.from_open_id_connect(git_hub_action_role_arn="arn:aws:iam::<account-id>:role/GitHubActionRole",role_session_name="optional-role-session-name"))There are two ways to create this IAM role:Use theGitHubActionRoleconstruct (recommended and described below).Manually set up the role (Guide).GitHubActionRoleConstructBecause this construct involves creating an IAM role in your account, it must
be created separate to your GitHub Workflow and deployed via a normalcdk deploywith your local AWS credentials. Upon successful deployment, the
arn of your newly created IAM role will be exposed as aCfnOutput.To utilize this construct, create a separate CDK stack with the following code
andcdk deploy:classMyGitHubActionRole(Stack):def__init__(self,scope,id,*,description=None,env=None,stackName=None,tags=None,synthesizer=None,terminationProtection=None,analyticsReporting=None,crossRegionReferences=None,permissionsBoundary=None):super().__init__(scope,id,description=description,env=env,stackName=stackName,tags=tags,synthesizer=synthesizer,terminationProtection=terminationProtection,analyticsReporting=analyticsReporting,crossRegionReferences=crossRegionReferences,permissionsBoundary=permissionsBoundary)provider=GitHubActionRole(self,"github-action-role",repos=["myUser/myRepo"])app=App()MyGitHubActionRole(app,"MyGitHubActionRole")app.synth()Specifying areposarray grants GitHub full access to the specified repositories.
To restrict access to specific git branch, tag, or otherGitHub OIDC subject claim,
specify asubjectClaimsarray instead of areposarray.classMyGitHubActionRole(Stack):def__init__(self,scope,id,*,description=None,env=None,stackName=None,tags=None,synthesizer=None,terminationProtection=None,analyticsReporting=None,crossRegionReferences=None,permissionsBoundary=None):super().__init__(scope,id,description=description,env=env,stackName=stackName,tags=tags,synthesizer=synthesizer,terminationProtection=terminationProtection,analyticsReporting=analyticsReporting,crossRegionReferences=crossRegionReferences,permissionsBoundary=permissionsBoundary)provider=GitHubActionRole(self,"github-action-role",subject_claims=["repo:owner/repo1:ref:refs/heads/main","repo:owner/repo1:environment:prod"])app=App()MyGitHubActionRole(app,"MyGitHubActionRole")app.synth()Note: If you have previously created the GitHub identity provider with urlhttps://token.actions.githubusercontent.com, the above example will fail
because you can only have one such provider defined per account. In this
case, you must provide the already created provider into yourGithubActionRoleconstruct via theproviderproperty.Make sure the audience for the provider issts.amazonaws.comin this case.classMyGitHubActionRole(Stack):def__init__(self,scope,id,*,description=None,env=None,stackName=None,tags=None,synthesizer=None,terminationProtection=None,analyticsReporting=None,crossRegionReferences=None,permissionsBoundary=None):super().__init__(scope,id,description=description,env=env,stackName=stackName,tags=tags,synthesizer=synthesizer,terminationProtection=terminationProtection,analyticsReporting=analyticsReporting,crossRegionReferences=crossRegionReferences,permissionsBoundary=permissionsBoundary)provider=GitHubActionRole(self,"github-action-role",repos=["myUser/myRepo"],provider=GitHubActionRole.existing_git_hub_actions_provider(self))GitHub SecretsAuthenticating via this approach means that you will be manually creating AWS
credentials and duplicating them in GitHub secrets. The workflow expects the
GitHub repository to include secrets with AWS credentials underAWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEY. You can override these defaults
by supplying theawsCreds.fromGitHubSecretsAPI to the workflow:fromaws_cdk.pipelinesimportShellStepapp=App()pipeline=GitHubWorkflow(app,"Pipeline",synth=ShellStep("Build",commands=["yarn install","yarn build"]),aws_creds=AwsCredentials.from_git_hub_secrets(access_key_id="MY_ID",# GitHub will look for the access key id under the secret `MY_ID`secret_access_key="MY_KEY"))Runners with Preconfigured CredentialsIf your runners provide credentials themselves, you can configureawsCredsto
skip passing credentials:fromaws_cdk.pipelinesimportShellStepapp=App()pipeline=GitHubWorkflow(app,"Pipeline",synth=ShellStep("Build",commands=["yarn install","yarn build"]),aws_creds=AwsCredentials.runner_has_preconfigured_creds())Using Docker in the PipelineYou can use Docker in GitHub Workflows in a similar fashion to CDK Pipelines.
For a full discussion on how to use Docker in CDK Pipelines, seeUsing Docker in the Pipeline.Just like CDK Pipelines, you may need to authenticate to Docker registries to
avoid being throttled.Authenticating to Docker registriesYou can specify credentials to use for authenticating to Docker registries as
part of the Workflow definition. This can be useful if any Docker image assets —
in the pipeline or any of the application stages — require authentication, either
due to being in a different environment (e.g., ECR repo) or to avoid throttling
(e.g., DockerHub).fromaws_cdk.pipelinesimportShellStepapp=App()pipeline=GitHubWorkflow(app,"Pipeline",synth=ShellStep("Build",commands=["yarn install","yarn build"]),docker_credentials=[# Authenticate to ECRDockerCredential.ecr("<account-id>.dkr.ecr.<aws-region>.amazonaws.com"),# Authenticate to GHCRDockerCredential.ghcr(),# Authenticate to DockerHubDockerCredential.docker_hub(# These properties are defaults; feel free to omitusername_key="DOCKERHUB_USERNAME",personal_access_token_key="DOCKERHUB_TOKEN"),# Authenticate to Custom RegistriesDockerCredential.custom_registry("custom-registry",username_key="CUSTOM_USERNAME",password_key="CUSTOM_PASSWORD")])Runner TypesYou can choose to run the workflow in either a GitHub hosted orself-hostedrunner.GitHub Hosted RunnerThe default isRunner.UBUNTU_LATEST. You can override this as shown below:fromaws_cdk.pipelinesimportShellStepapp=App()pipeline=GitHubWorkflow(app,"Pipeline",synth=ShellStep("Build",commands=["yarn install","yarn build"]),runner=Runner.WINDOWS_LATEST)Self Hosted RunnerThe following example shows how to configure the workflow to run on a self-hosted runner. Note that you do not need to pass inself-hostedexplicitly as a label.fromaws_cdk.pipelinesimportShellStepapp=App()pipeline=GitHubWorkflow(app,"Pipeline",synth=ShellStep("Build",commands=["yarn install","yarn build"]),runner=Runner.self_hosted(["label1","label2"]))Escape HatchesYou can override thedeploy.ymlworkflow file post-synthesis however you like.fromaws_cdk.pipelinesimportShellStepapp=App()pipeline=GitHubWorkflow(app,"Pipeline",synth=ShellStep("Build",commands=["yarn install","yarn build"]))deploy_workflow=pipeline.workflow_file# add `on: workflow_call: {}` to deploy.ymldeploy_workflow.patch(JsonPatch.add("/on/workflow_call",{}))# remove `on: workflow_dispatch` from deploy.ymldeploy_workflow.patch(JsonPatch.remove("/on/workflow_dispatch"))Additional FeaturesBelow is a compilation of additional features available for GitHub Workflows.GitHub Action StepIf you want to call a GitHub Action in a step, you can utilize theGitHubActionStep.GitHubActionStepextendsStepand can be used anywhere aSteptype is allowed.ThejobStepsarray is placed into the pipeline job at the relevantjobs.<job_id>.stepsasdocumented here.In this example,fromaws_cdk.pipelinesimportShellStepapp=App()pipeline=GitHubWorkflow(app,"Pipeline",synth=ShellStep("Build",commands=["yarn install","yarn build"]))# "Beta" stage with a pre-check that uses code from the repo and an actionstage=MyStage(app,"Beta",env=BETA_ENV)pipeline.add_stage(stage,pre=[GitHubActionStep("PreBetaDeployAction",job_steps=[JobStep(name="Checkout",uses="actions/checkout@v3"),JobStep(name="pre beta-deploy action",uses="my-pre-deploy-action@1.0.0"),JobStep(name="pre beta-deploy check",run="npm run preDeployCheck")])])app.synth()Configure GitHub EnvironmentYou can run your GitHub Workflow in selectGitHub Environments.
Via the GitHub UI, you can configure environments with protection rules and secrets, and reference
those environments in your CDK app. A workflow that references an environment must follow any
protection rules for the environment before running or accessing the environment's secrets.Assuming (just like in the mainexample) you have aStagecalledMyStagethat includes CDK stacks for your app and you want to deploy it
to two AWS environments (BETA_ENVandPROD_ENV) as well as GitHub Environmentsbetaandprod:fromaws_cdk.pipelinesimportShellStepapp=App()pipeline=GitHubWorkflow(app,"Pipeline",synth=ShellStep("Build",commands=["yarn install","yarn build"]),aws_creds=AwsCredentials.from_open_id_connect(git_hub_action_role_arn="arn:aws:iam::<account-id>:role/GitHubActionRole"))pipeline.add_stage_with_git_hub_options(Stage(self,"Beta",env=BETA_ENV),git_hub_environment=GitHubEnvironment(name="beta"))pipeline.add_stage_with_git_hub_options(MyStage(self,"Prod",env=PROD_ENV),git_hub_environment=GitHubEnvironment(name="prod"))app.synth()Waves for Parallel BuildsYou can add a Wave to a pipeline, where each stage of a wave will build in parallel.Note: Thepipeline.addWave()call will return aWaveobject that is actually aGitHubWaveobject, but
due to JSII rules the return type ofaddWave()cannot be changed. If you need to usewave.addStageWithGitHubOptions()then you should callpipeline.addGitHubWave()instead, or you can
useGitHubStages to carry the GitHub properties.When deploying to multiple accounts or otherwise deploying mostly-unrelated stacks, using waves can be a huge win.Here's a relatively large (but real) example,withouta wave:You can see how dependencies get chained unnecessarily, where thecUrlstep should be the final step (a test) for an account:Here's the exact same stages deploying the same stacks to the same accounts, butwitha wave:And the dependency chains are reduced to only what is actually needed, with thecUrlcalls as the final stage for each account:For additional information and a code example seehere.Manual Approval StepOne use case for using GitHub Environments with your CDK Pipeline is to create a
manual approval step for specific environments via Environment protection rules.
From the GitHub UI, you can specify up to 5 required reviewers that must approve
before the deployment can proceed:For more information and a tutorial for how to set this up, see thisdiscussion.Pipeline YAML CommentsAn "AUTOMATICALLY GENERATED FILE..." comment will by default be added to the top
of the pipeline YAML. This can be overriden as desired to add additional context
to the pipeline YAML.declare const pipeline: GitHubWorkflow;

pipeline.workflowFile.commentAtTop = `AUTOGENERATED FILE, DO NOT EDIT DIRECTLY!

Deployed stacks from this pipeline:
${STACK_NAMES.map((s)=>`- ${s}\n`)}`;This will generate the normaldeploy.ymlfile, but with the additional comments:# AUTOGENERATED FILE, DO NOT EDIT DIRECTLY!# Deployed stacks from this pipeline:# - APIStack# - AuroraStackname:deployon:push:branches:< the rest of the pipeline YAML contents>Common Configuration for Docker Asset Publishing StepsYou can provide common job configuration for all of the docker asset publishing
jobs using thedockerAssetJobSettingsproperty. You can use this to:Set additionalpermissionsat the job levelRun additional steps prior to the docker build/push stepBelow is an example of example of configuration an additionalpermissionwhich
allows the job to authenticate against GitHub packages. It also shows
configuration additionalsetupSteps, in this case setup steps to configure
dockerbuildxandQEMUto enable building images for arm64 architecture.fromaws_cdk.pipelinesimportShellStepapp=App()pipeline=GitHubWorkflow(app,"Pipeline",synth=ShellStep("Build",commands=["yarn install","yarn build"]),docker_asset_job_settings=DockerAssetJobSettings(permissions=JobPermissions(packages=JobPermission.READ),setup_steps=[JobStep(name="Setup Docker QEMU",uses="docker/setup-qemu-action@v3"),JobStep(name="Setup Docker buildx",uses="docker/setup-buildx-action@v3")]),aws_creds=AwsCredentials.from_open_id_connect(git_hub_action_role_arn="arn:aws:iam::<account-id>:role/GitHubActionRole"))app.synth()TutorialYou can find an example usage intest/example-app.tswhich includes a simple CDK app and a pipeline.You can find a repository that uses this example here:eladb/test-app-cdkpipeline.To run the example, clone this repository and install dependencies:cd~/projects# or some other playground spacegitclonehttps://github.com/cdklabs/cdk-pipelines-githubcdcdk-pipelines-github
yarnNow, create a new GitHub repository and clone it as well:cd~/projects
gitclonehttps://github.com/myaccount/my-test-repositoryYou'll need to set up AWS credentials in your environment. Note that this tutorial uses
long-lived GitHub secrets as credentials for simplicity, but it is recommended to set up
a GitHub OIDC role instead.exportAWS_ACCESS_KEY_ID=xxxxexportAWS_SECRET_ACCESS_KEY=xxxxxBootstrap your environments:exportCDK_NEW_BOOTSTRAP=1npxcdkbootstrapaws://ACCOUNTID/us-east-1
npxcdkbootstrapaws://ACCOUNTID/eu-west-2Now, run themanual-test.shscript when your working directory is the new repository:cd~/projects/my-test-repository
~/projects/cdk-piplines/github/test/manual-test.shThis will produce acdk.outdirectory and a.github/workflows/deploy.ymlfile.Commit and push these files to your repo and you should see the deployment
workflow in action. Make sure your GitHub repository hasAWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEYsecrets that can access the same account that you
synthesized against.In this tutorial, you are supposed to commitcdk.out(i.e. the code is pre-synthed).
Do not do this in your app; you should always synth during the synth step of the GitHub
workflow. In the example app this is achieved through thepreSynthed: trueoption.
It is for example purposes only and is not something you should do in your app.fromaws_cdk.pipelinesimportShellSteppipeline=GitHubWorkflow(App(),"Pipeline",synth=ShellStep("Build",commands=["echo\"nothing to do (cdk.out is committed)\""]),# only the example app should do this. your app should synth in the synth step.pre_synthed=True)Not supported yetMost features that exist in CDK Pipelines are supported. However, as the CDK Pipelines
feature are expands, the feature set for GitHub Workflows may lag behind. If you see a
feature that you feel should be supported by GitHub Workflows, please open a GitHub issue
to track it.ContributingSeeCONTRIBUTINGfor more information.LicenseThis project is licensed under the Apache-2.0 License.