Package: weco-deploy
weco-deployweco-deploy helps us deploy applications as Docker images within Amazon ECS.This tool only makes sense in the context of how we tag and deploy images, so this README explains our broad approach and how we use weco-deploy.How we tag our Docker imagesWe package applications as Docker images.
Images are automatically pushed on every merge to a main branch, and pushed to an ECR repository.Within our ECR repository, we have three types of tag.
Here's a list of images, to serve as an example:Here's how we tag images:Every image has a tag startingref; this is the Git commit has that was used to build a given image.
This helps us match an image to its source code.Thelatesttag points to the last image that was pushed to this repository.
It helps us know what the newest version of our code is.This tag is updated by our CI/CD pipeline.Theenvtags point to the image being used in a particular environment.
For example, we can see here there are images with anenv.stageand anenv.prodtag – these are the images used in our staging and prod environment, respectively.These are floating tags set by weco-deploy.How we use tags to deploy imagesWithin our ECS task definitions, we point to anenvtag as the Docker image to use.{
  "containerDefinitions": [
    {
      "image": "{ECR repository prefix}/our_app:env.stage",
      ...
    }
  ],
  ...
}weco-deploy gives us tools for updating these floatingenvtags.
This is easiest to explain with some example scenarios:Example #1: We've pushed some new code, and we want to deploy it to our staging environment.First our CI/CD pipeline builds new Docker images, publishes them to an ECR repository, and tags them withlatest.
Then we ask weco-deploy to update theenv.stagetag to point to the images that are currently taggedlatest.
Finally, weco-deploy tells ECS to redeploy any services using that tag, which causes them to pull the new Docker image.Example #2: We've tested the code in staging, we're satisfied it works, and we want to deploy it to production.First we ask weco-deploy to update theenv.prodtag to point to the images that are currently taggedenv.stage.
Then, weco-deploy tells ECS to redeploy any services using that tag, which causes them to pull the new Docker image.weco-deploy also tracks all the deployments, so we can see what code was deployed when.TerminologyThese are the terms which are used by weco-deploy, which have a fairly specific meaning in this context:projectA collection of images/services that are deployed together.
    Different projects are managed independently, e.g. we can deploy new images to the catalogue API without affecting the storage service.releaseA set of specific versions of Docker images that can be deployed together.
    e.g. The set of images `bag_verifier:ref.123`, `bag_replicator:ref.456` and `bag_unpacker:ref.789`.deploymentThe act of applying a release to an environment.
    i.e. telling ECS to use a particular set of images.A release is just an abstract concept; a deployment is making it concrete by deploying the images.
    Note that one release might be used in multiple deployments: for example, deploying a release to a staging environment and then a production environment..wellcome_project fileA YAML file that is used to configure weco-deploy: in particular, listing each project, the Docker images it contains, and what ECS services need to be redeployed when the image tags are updated.
    The easiest way to understand is tolook at an example.Installationweco-deploy is published as a package to PyPI, so you can install it with pip:$pip3installweco-deployUsageMost of our use of weco-deploy is automated in Buildkite, so manual use is rare.The most useful subcommands are:$ # create a new release, but don't deploy it
$ weco-deploy prepare

$ # deploy a previously created release
$ weco-deploy deploy

$ # create a new release and deploy it straight to ECS
$ weco-deploy release-deployYou can select a project/label using command-line flags, or if not the tool will prompt you for the required inputs.