Docker-registry

We run our own Docker registry at docker-registry.wikimedia.org. Internally the domain docker-registry.discovery.wmnet is also used. The registry is used by our k8s cluster, CI, and local development.
It is highly available (docker_registry_ha Puppet module) and backed by Swift. Although we run it active/passive because of the swift replication lag.
The docker-registry nodes consist of the docker registry itself as well as an nginx reverse-proxy in front to handle authentication as well as local caching.
== Browsing ==
Visit https://docker-registry.wikimedia.org/ to see a list of images and their tags. The listing is updated on a hourly timer and is done by the registry-homepage-builder.py script in Puppet.
== Downloading images ==
Despite the name, the docker-registry is usable by any OCI container tool, including podman. Nearly all images may be publicly downloaded, examined, run, etc. The only exception is images under the restricted/ namespace, which contain non-disclosed security patches and require specific credentials to fetch.
Kubernetes nodes use Dragonfly to pull images.
== Uploading images ==
For services we recommend using the Deployment pipeline which is Blubber.
For other docker images, like infrastructure images, we manage them using docker-pkg, see: Kubernetes/Images#Image_building
Hosts that want to upload images must be individually listed in Puppet hiera.
== Access control ==
The upstream docker-registry software provides no access control, so it is implemented at the nginx level, which restricts GET/POST/etc. requests accordingly. As of 2021-03-18, the following accounts exist:
* ci-restricted: Can pull and push any image (including "restricted/"). Used by releases servers that build the restricted MediaWiki production image.
* ci-build: Can pull and push any non-restricted image. Used by contint servers via docker-pkg and the deployment pipeline.
* prod-build: Can pull and push any non-restricted image. Used by build2001.codfw.wmnet via docker-pkg and build-base-images.
* kubernetes: Can pull any image (including "restricted/"). Used by k8s nodes to pull images, including the restricted MediaWiki production image.
**See Kubernetes/Clusters/New#Access to restricted docker images for more details.
The passwords are all deployed using the private puppet repo. In case rotation is needed (e.g. compromise), grepping for <name>_user_password should find all uses (switch hyphens to underscores).
=== jwt-authorizer ===
docker-registry also supports authorization using JSON Web Tokens. A dedicated daemon is running which handles jwt validation. See Docker-registry/jwt-authorizer for more information.
== Deleting images ==
To delete an image entirely, you may use the tool docker-registryctl on the current build host. It will do it's best to remove the tags/image from the registry, despite the circumstances. Note: the domain used here is important. discovery.wmnet has to be used, so you will have to adjust it if you copy and paste from the browser UI.
elukey@build2001:~$ sudo -i docker-registryctl delete-tags docker-registry.discovery.wmnet/wikimedia/machinelearning-liftwing-inference-services
We're about to delete the following tags for image docker-registry.discovery.wmnet/wikimedia/machinelearning-liftwing-inference-services:
2021-07-28-175322-production
stable
Ok to proceed? (y/n)y
docker-registry.discovery.wmnet/wikimedia/machinelearning-liftwing-inference-services:2021-07-28-175322-production[DONE]
docker-registry.discovery.wmnet/wikimedia/machinelearning-liftwing-inference-services:stable[GONE]
== httpbb ==
There is a (not comprehensive) Httpbb test case for the docker registry:
sudo httpbb /srv/deployment/httpbb-tests/docker-registry/test_docker-registry.yaml --hosts 'registry2003.codfw.wmnet'
== See also ==
* /Runbook (draft)
*Service Level Objective (SLO)
* Docker
* Upstream Git repository
*User:JMeybohm/Docker-Registry-Stresstest
Category:Containers
Category:Docker