Package: iam-docker-run
Run Docker containers within the context of an AWS IAM Role, and other
development workflow helpers.MotivationThe goal is to run our application on our laptops in development in as
similar environment as possible to the production environment when the
application runs in ECS or EKS, which would run under the task IAM role
with permissions specific to that task.A shortcut sometimes taken by developers is to execute code locally with
their personal IAM user which often has very high and broad privileges.
Worse, those long lived credentials sometimes find themselves checked
into source control as part of a docker-compose.yml file, etc.
IAM-Docker-Run allows you to run your containers locally within the
context of the IAM role you’ve created for your application. The
credentials themselves are temporary, they are stored in a file in your
system’s temp path, and never wind up in source control.IAM-Docker-Run generates AWS temporary credentials and builds a lengthlydocker runcommand line statement, echoing it as it executes it so
you have transparency into the command it is running.This is a development workflow tool, not designed to run production
containers.A related effort isIAM-Starterwhich starts
a local process (outside of Docker) in the context of an AWS IAM role.Installationpipinstalliam-docker-runBasic Example UsageSay you are developing a console application using AWS resources and are
in your project’s root directory and want to execute your application
using your latest source code and the IAM role created for your project.iam-docker-run\--imagemycompany/myservice:latest\--rolerole-myservice-taskYou can alternatively specify a local AWS profile, then the container
will run with the credentials given by that role. This profile would
have to exist locally in your~/.aws/configfile, which can be
created withaws configure--profilemyprofile.iam-docker-run\--imagemycompany/myservice:latest\--profilemyprofileOr you can specify a role and a profile. In this case the profile
provides the credentials necessary to assume the role.iam-docker-run\--imagemycompany/myservice:latest\--rolerole-myservice-task\--profilemyprofileArguments and More ExamplesFull argument listFor a full list of arguments, runiam-docker-run-h.Full EntrypointThe Docker syntax for overriding an entrypoint with anything more than
one word can seem couterintuitive. With the Docker syntax, the
entrypoint can only be the first command and all arguments to that are
separated out on the cmd, so if you want to runpython myapp.py--myargtest123, thenpythonis your entrypoint
and the rest go on your cmd, to produce a docker run statement like:dockerrun--entrypointpythonmycompany/myimagemyapp.py--myargtest123To make things easier, iam-docker-run provides the--full-entrypointargument, so you can use it like this:iam-docker-run\--imagemycompany/myimage\--full-entrypoint"python myapp.py --myarg test123"ShellIf you want to debug something in the container, just add a--shellargument and it will override the entrypoint with/bin/bash. If you
wish to use an alternate shell, you can override this with the following
enrivonment variable:exportIAM_DOCKER_RUN_SHELL_COMMAND="/bin/sh"It is especially convenient to use this command to add to the end of any
existing set of arguments. It will override both the default ENTRYPOINT
defined in the Dockerfile as well as the--full-entrypointargument.# for example, --shell will take precedence over --full-entrypointiam-docker-run\--imagemycompany/myimage\--full-entrypoint"python myapp.py --myarg test123"\--shell# let me jump in real quick without modifying the rest of my argsCustom environment variables fileIf you have environment variables you want passed to Docker viadocker run--env-file, with iam-docker-run you would use--custom-env-file. The reason for this is that iam-docker-run is
already using a file to pass into Docker with the environment variables
for the AWS temporary credentials, so if you have environment variables
to add to that, specify a--custom-env-fileand that will be
concatenated to the env file created by iam-docker-run.Default behavior is to look for a file callediam-docker-run.env. If
this file is not found it is silently ignored. This is helpful if you
have an environment variable such asAWS_ENV=devwhich you want
loaded each time without specifying this argument. Hopefully the rest of
your variables are loaded into the environment from a remote
configuration store such as AWS SSM Parameter Store. If you need help
with this seessm-starter.Custom environment argumentsAdditionally you can pass environment variables by-eor--envvar, which is passthrough to thedocker-eargument. These
are additive with the custom environment variables file.Foreground / backgroundAs the main use case is a development workflow, by default the container
runs in the foreground. To run in the background, specify--detached, which maps to thedocker run-dcommand. To interact
with the terminal, specify--interactive, which maps todocker run-it.Source code volume mount by arguments (developer workflow)The--host-source-pathand--container-source-patharguments are
designed to make it easy to mount your source code into the container
when using Docker in a developer workflow where you make changes in your
IDE on your host computer and want that source code immediately inserted
into the container. The--host-source-pathargument can be relative.
In prior versions of IAM-Docker-Run the source code mount was automatic
and required the--no-volumeargument to prevent mounting it. This
automatic mount behavior has been removed however these arguments will
remain for backward compatibility.iam-docker-run\--imagemycompany/myservice\--rolerole-myservice-task\--host-source-path./mysource\--container-source-path/myappAdditional volume mountsYou can mount additional volumes by-vor--volume, which is
passthrough to thedocker-vargument. These are additive with the
source code volume mount (if specified) and the docker in docker mount.Overcoming SELinux with volume mountsIf you are running SELinux and experience permission denied issues when
mounting volumes, specify the--selinuxargument, which will alter
the dockr run volume mount argument so that the volume is readable.Enable Docker in DockerIf you want to enable Docker in Docker, you can mount the Docker socket
by adding the--mount-dockerargument. If you then install Docker in
the container with the below script and use the Docker CLI from within
the container.# install the docker clientcurl-fsSLget.docker.com-oget-docker.shshget-docker.shAdding a portmapYou can use--portmapor-p, which is a direct match to thedocker run-pargument, for example:iam-docker-run\--imagemycompany/myservice\--rolerole-myservice-task\--portmap30000:3000The--portmap30000:3000argument in this example would take a HTTP
server listening in the container on port 3000 and maps it to port 30000
on your laptop.Note that you can use multiple portmaps as follows:iam-docker-run\--imagemycompany/myservice\--rolerole-myservice-task\-p4430:443\-p8080:80RegionIf--regionis provided that will take precidence, otherwise
iam-docker-run will look for your region in AWS_REGION or
AWS_DEFAULT_REGION environment variables. If none are provided it will
default to us-east-1.Container Name TempfileIAM-Docker-Run generates a random container name if the –name arg is not
supplied. If this container name is needed for anything downstream such
as the code debugging inside the container feature of VSCode, the
container name needs to be discoverable. IAM-Docker-Run enables this by
generating a file which contains the name of the container and writes it
in a pre-determined location.The location of this file follows the:/temp/<lastdirectory name ofpwd>/_container_name.txtYou can override the first part of the prefix with the following
environment variable:exportIAM_DOCKER_RUN_CONTAINER_NAME_PATH_PREFIX=/tmp/somewhere/elseOr you can disable this entirely by setting:exportIAM_DOCKER_RUN_DISABLE_CONTAINER_NAME_TEMPFILE=trueShortcutAn alternate way to invoke iam-docker-run on the command line is to use
the aliasidr. Just less typing.idr--imagebusybox--rolemyroleExample CI workflowThe second use case for iam-docker-run is for running tests from
continuous integration.iam-docker-run\--imagemycompany/myimage\--rolerole-myservice-task\--full-entrypoint"/bin/bash /tests/run-integration-test.sh"\--profilejenkinsVerbose debuggingTo turn on verbose output for debugging, set the--verboseargument.Temporary Credentials Expire Within 1 HourA goal of this project was to be as easy as possible for developers to
use and to allow the greatest portability. To that end, the temporary
AWS credentials are generated just once before the container starts,
rather than requiring a more complex setup where an additional container
would run all the time and regenerate credentials. When the temp
credentials expire (the STS max of 1 hour), the application will start
experiencing expired credential exceptions. For this among other reasons
is why you would not use this tool in any environment other than local
development or in your build/CI/CD workflow where usage periods are
short and the container can be restarted easily and often.Note: While the STS temporary credentials maximum was recently raised to
12 hours, if you are already in the context of an IAM role which is then
assuming another role, the limit in this case remains to be 1 hour.TestingRun the automated script cli tests:pipinstall--usernosescripttestpythonsetup.pyinstall--userexportAWS_REGION=us-east-1# set AWS_PROFILE to a valid profile name which can assume rolesexportAWS_PROFILE=devnosetests-v--exe-w./testTesting the use case of a role being supplied without a profile, using
the credentials in the environment, is difficult to test an a generic
automated way. For now, the following manual steps can test this
condition.# set ROLE_ARN_FOR_LOCAL_CREDS to a role which can list s3 bucketsexportROLE_ARN_FOR_LOCAL_CREDS=arn:aws:iam::123456789012:role/my-role# set AWS_PROFILE to a valid profile name which can assume the ROLE_ARNexportAWS_PROFILE=devexportROLE_NAME_FOR_CONTAINER=role-ops-developersawsstsassume-role\--role-arn$ROLE_ARN_FOR_LOCAL_CREDS\--role-session-nametesting\--profile$AWS_PROFILE# put credentials in the environmentexportAWS_ACCESS_KEY_ID=fromaboveexportAWS_SECRET_ACCESS_KEY=fromaboveexportAWS_SESSION_TOKEN=fromaboveiam-docker-run\--role$ROLE_NAME_FOR_CONTAINER\--imagemesosphere/aws-cli:latest--full-entrypoint"aws s3 ls"# command should succeed with a listing of s3 bucketsPublishing Updates to PyPiFor the maintainer - to publish an updated version of Iam-Docker-Run,
increment the version number in iam_docker_run.py and run the following:dockerbuild-f./Dockerfile.buildenv-tbilltrust/iam-docker-run:build.dockerrun--rm-it--entrypointmakebilltrust/iam-docker-run:buildpublishAt the prompts, enter the username and password to the Billtrust
pypi.org repo.LicenseMIT LicenseCopyright (c) 2018 Factor Systems Inc.Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the
“Software”), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.