Package: appendlayer
Append layerThis standalone utility appends a tarball to an existing image in a
container registry – without having to pull down the image locally.It supports any registry that implements theOCI Distribution
Spec.WhyThe basic use-case for this utility is when you have a base image that
is already available in a container registry, and you simply need to
add one or more files, then push the result back to the same registry.In this case, you can do no better in terms of network transfer than
this utility. It does the minimum amount of work in order to get the
job done.With theDocker
ADDcommand,
you'd have to download the existing image and start up a build process
in order to run theADDcommand.FROMapache/airflow:2.3.2ADDyour_tar_file.tar.gz/opt/airflow/dagThe resulting image would be exactly the same, but there is no special
optimization in Docker that would avoid downloading the base image
(although theoretically, it could be done but it would require bigger
changes in the data model in order to support lazy referencing of
layer data).Incidentally, the script was designed exactly withApache
Airflowin mind.Note that Buildkitwill have support for
thisfrom v0.10 but
it's not clear how this will be supported in a Dockerfile.InstallationInstall the tool using pip:$pipinstallappendlayerThis makes available "appendlayer" as a script in your environment.Alternatively, download theappendlayer.pyscript
and run it using Python directly:$pythonappendlayer.pyThe script has no external dependencies, using only what's included already with Python.UsagePipe in the layer contents using a tarball and provide the repository (orimage) name and the old and new tags:$echo"Hello world">test.txt
$tarcvf-test.txt|appendlayer<host><repository><old-tag><new-tag>Alternatively, qualify source and destination using image syntax:$tarcvf-test.txt|appendlayer<host><old-repository>:<old-tag><new-repository>:<new-tag>Or even across different hosts:$tarcvf-test.txt|appendlayer<old-host>/<old-repository>:<old-tag><new-host>/<new-repository>:<new-tag>For Azure Container Registry (ACR) for example, thehostis
typically<registry-name>.azurecr.io.AuthenticationThe script uses OAuth2 to authorize requests to the container
registry.This is configured using either theACCESS_TOKENorREFRESH_TOKENenvironment variable, or by extracting authentication details from the Docker
configuration file (located based on theDOCKER_CONFIGenvironment
variable).For example, forAzure Container
Registry,
to authorize to a specific container registry.$exportREFRESH_TOKEN=$(\azacrlogin-t--name<registry-name>.azurecr.io\--expose-token--outputtsv--queryaccessToken)To authorize across multiple registries, use an access token:$exportACCESS_TOKEN=$(\azaccountget-access-token--queryaccessToken--outputtsv)Changes2.2 (2022-09-30)Add support for stored Docker credentials with "https"-based URL
instead of hostname as the authentication entry.2.1 (2022-09-30)Add support for username/password credentials in stored Docker
credentials in addition to identity token.2.0 (2022-09-30)Add support for extracting refresh token from stored Docker
credentials.Fix bug where two different repositories would not correctly get
authorized for the destination registry.Added support for specifying different source and destination
repositories. Missing blobs will be copied if necessary.1.0 (2022-01-29)Initial release.