Project: containerd (Runtime / Container Runtime)
File: hosts.md

New and additional registry hosts config support has been implemented in containerd v1.5 for the `ctr` client (the containerd tool for admins/developers), containerd image service clients, and CRI clients such as `kubectl` and `crictl`. Configuring registries, for these clients, will be done by specifying (optionally) a `hosts.toml` file for each desired registry host in a configuration directory. Note: Updates under this directory do not require restarting the containerd daemon. All configured registry hosts are expected to comply with the . Registries which are non-compliant or implement non-standard behavior are not guaranteed to be supported and may break unexpectedly between releases. Currently supported OCI Distribution version: When pulling a container image via `ctr` using the `--hosts-dir` option tells `ctr` to find and use the host configuration files located in the specified path: ``` ctr images pull --hosts-dir "/etc/containerd/certs.d" myregistry.io:5000/image_name:tag ``` _The old CRI config pattern for specifying registry.mirrors and registry.configs has been DEPRECATED. You should now point your registry `configpath` to the path where your `hosts.toml` files are located. Modify your `config.toml` (default location: `/etc/containerd/config.toml`) as follows: Before containerd 2.0 ```toml version = 2 [plugins."io.containerd.grpc.v1.cri".registry] config_path = "/etc/containerd/certs.d" ``` In containerd 2.0 ``` version = 3 [plugins."io.containerd.cri.v1.images".registry] config_path = "/etc/containerd/certs.d" ``` If no hosts.toml configuration exists in the host directory, it will fallback to check certificate files based on (".crt" files for CA certificates and ".cert"/".key" files for client certificates). A registry host is the location where container images and artifacts are sourced. These registry hosts may be local or remote and are typically accessed via http/https using the . A registry mirror is not a registry host but these mirrors can also be used to pull content. Registry hosts are typically referred to by their internet domain names, aka. registry host names. For example, docker.io, quay.io, gcr.io, and ghcr.io. A registry host namespace is, for the purpose of containerd registry configuration, a path to the `hosts.toml` file specified by the registry host name, or ip address, and an optional port identifier. When making a pull request for an image the format is typically as follows: ``` pull <image_name>[:tag|@DIGEST] ``` The registry host namespace portion is ``. Example tree for docker.io: ``` $ tree /etc/containerd/certs.d /etc/containerd/certs.d docker.io hosts.toml ``` Optionally the `_default` registry host namespace can be used as a fallback, if no other namespace matches. The `/v2` portion of the pull request format shown above refers to the version of the distribution api. If not included in the pull request, `/v2` is added by default for all clients compliant to the distribution specification linked above. If a host is configured that's different to the registry host namespace (e.g. a mirror), then containerd will append the registry host namespace to requests as a query parameter called `ns`. For example when pulling `imagename:tagname` from a private registry named `myregistry.io` over port 5000: ``` pull myregistry.io:5000/imagename:tagname ``` The pull will resolve to `https://myregistry.io:5000/v2/imagename/manifests/tagname`. The same pull with a host configuration for `mymirror.io` will resolve to `https://mymirror.io/v2/imagename/manifests/tagname?ns=myregistry.io:5000`. When performing image operations via `ctr` use the --help option to get a list of options you can set for specifying credentials: ``` ctr i pull --help

OPTIONS: --skip-verify, -k skip SSL certificate validation --plain-http allow connections using plain HTTP --user value, -u value user[:password] Registry user and password --refresh value refresh token for authorization server --hosts-dir value Custom hosts configuration directory --tlscacert value path to TLS root CA --tlscert value path to TLS client certificate --tlskey value path to TLS client key --http-dump dump all HTTP request/responses when interacting with container registry --http-trace enable HTTP tracing for registry interactions --snapshotter value snapshotter name. Empty value stands for the default value. [$CONTAINERD_SNAPSHOTTER] --label value labels to attach to the image --platform value Pull content from a specific platform --all-platforms pull content and metadata from all platforms --all-metadata Pull metadata for all platforms --print-chainid Print the resulting image's chain ID --max-concurrent-downloads value Set the max concurrent downloads for each pull (default: 0) ``` Although we have deprecated the old CRI config pattern for specifying registry.mirrors and registry.configs you can still specify your credentials via . Additionally, the containerd CRI plugin implements/supports the authentication parameters passed in through CRI pull image service requests. For example, when containerd is the container runtime implementation for `Kubernetes`, the containerd CRI plugin receives authentication credentials from kubelet as retrieved from Here is a simple example for a default registry hosts configuration. Set `config_path = "/etc/containerd/certs.d"` in your config.toml for containerd. Make a directory tree at the config path that includes `docker.io` as a directory representing the host namespace to be configured. Then add a `hosts.toml` file in the `docker.io` to configure the host namespace. It should look like this: ``` $ tree /etc/containerd/certs.d /etc/containerd/certs.d docker.io hosts.toml $ cat /etc/containerd/certs.d/docker.io/hosts.toml server = "https://docker.io" [host."https://registry-1.docker.io"] capabilities = ["pull", "resolve"] ``` ``` server = "https://registry-1.docker.io" # Exclude this to not use upstream [host."https://public-mirror.example.com"] capabilities = ["pull"] # Requires less trust, won't resolve tag to digest from this host [host."https://docker-mirror.internal"] capabilities = ["pull", "resolve"] ca = "docker-mirror.crt" # Or absolute path /etc/containerd/certs.d/docker.io/docker-mirror.crt ``` This is an example of using a mirror regardless of the intended registry. The upstream registry will automatically be used after all defined hosts have been tried. ``` $ tree /etc/containerd/certs.d /etc/containerd/certs.d _default hosts.toml $ cat /etc/containerd/certs.d/_default/hosts.toml [host."https://registry.example.com"] capabilities = ["pull", "resolve"] ``` If you wish to ensure only the mirror is utilised and the upstream not consulted, set the mirror as the `server` instead of a host. You may still specify additional hosts if you'd like to use other mirrors first. ``` $ cat /etc/containerd/certs.d/_default/hosts.toml server = "https://registry.example.com" ``` To bypass the TLS verification for a private registry at `192.168.31.250:5000` Create a path and `hosts.toml` text at the path "/etc/containerd/certs.d/docker.io/hosts.toml" with following or similar contents: ```toml server = "https://registry-1.docker.io" [host."http://192.168.31.250:5000"] capabilities = ["pull", "resolve", "push"] skip_verify = true ``` For each registry host namespace directory in your registry `config_path` you may include a `hosts.toml` configuration file. The following root level toml fields apply to the registry host namespace: Note: All paths specified in the `hosts.toml` file may be absolute or relative to the `hosts.toml` file. `server` specifies the default server for this registry host namespace. When `host`(s) are specified, the hosts will be tried first in the order listed. If all `host`(s) are tried then `server` will be used as a fallback. If `server` is not specified then the image's registry host namespace will automatically be used. ``` server = "https://docker.io" ``` `capabilities` is an optional setting for specifying what operations a host is capable of performing. Include only the values that

``` capabilities = ["pull", "resolve", "push"] ``` capabilities (or Host capabilities) represent the capabilities of the registry host. This also represents the set of operations for which the registry host may be trusted to perform. For example, pushing is a capability which should only be performed on an upstream source, not a mirror. Resolving (the process of converting a name into a digest) must be considered a trusted operation and only done by a host which is trusted (or more preferably by secure process which can prove the provenance of the mapping). A public mirror should never be trusted to do a resolve action. | Registry Type | Pull | Resolve | Push | ||||| | Public Registry | yes | yes | yes | | Private Registry | yes | yes | yes | | Public Mirror | yes | no | no | | Private Mirror | yes | yes | no | `ca` (Certificate Authority Certification) can be set to a path or an array of paths each pointing to a ca file for use in authenticating with the registry namespace. ``` ca = "/etc/certs/mirror.pem" ``` or ``` ca = ["/etc/certs/test-1-ca.pem", "/etc/certs/special.pem"] ``` `client` certificates are configured as follows a path: ``` client = "/etc/certs/client.pem" ``` an array of paths: ``` client = ["/etc/certs/client-1.pem", "/etc/certs/client-2.pem"] ``` an array of pairs of paths: ``` client = [["/etc/certs/client.cert", "/etc/certs/client.key"],["/etc/certs/client.pem", ""]] ``` `skip_verify` skips verifications of the registry's certificate chain and host name when set to `true`. This should only be used for testing or in combination with other method of verifying connections. (Defaults to `false`) ``` skip_verify = false ``` `[header]` contains some number of keys where each key is to one of a string or an array of strings as follows: ``` [header] x-custom-1 = "custom header" ``` or ``` [header] x-custom-1 = ["custom header part a","part b"] ``` or ``` [header] x-custom-1 = "custom header", x-custom-1-2 = "another custom header" ``` `override_path` is used to indicate the host's API root endpoint is defined in the URL path rather than by the API specification. This may be used with non-compliant OCI registries which are missing the `/v2` prefix. (Defaults to `false`) ``` override_path = true ``` `[host]."https://namespace"` and `[host]."http://namespace"` entries in the `hosts.toml` configuration are registry namespaces used in lieu of the default registry host namespace. These hosts are sometimes called mirrors because they may contain a copy of the container images and artifacts you are attempting to retrieve from the default registry. Each `host`/`mirror` namespace is also configured in much the same way as the default registry namespace. Notably the `server` is not specified in the `host` description because it is specified in the namespace. Here are a few rough examples configuring host mirror namespaces for this registry host namespace: ``` [host."https://mirror.registry"] capabilities = ["pull"] ca = "/etc/certs/mirror.pem" skip_verify = false [host."https://mirror.registry".header] x-custom-2 = ["value1", "value2"] [host."https://mirror-bak.registry/us"] capabilities = ["pull"] skip_verify = true [host."http://mirror.registry"] capabilities = ["pull"] [host."https://test-1.registry"] capabilities = ["pull", "resolve", "push"] ca = ["/etc/certs/test-1-ca.pem", "/etc/certs/special.pem"] client = [["/etc/certs/client.cert", "/etc/certs/client.key"],["/etc/certs/client.pem", ""]] [host."https://test-2.registry"] client = "/etc/certs/client.pem" [host."https://test-3.registry"] client = ["/etc/certs/client-1.pem", "/etc/certs/client-2.pem"] [host."https://non-compliant-mirror.registry/v2/upstream"] capabilities = ["pull"] override_path = true ``` Note: Recursion is not supported in the specification of host mirror namespaces in the hosts.toml file. Thus the following is not allowed/supported: ``` [host."http://mirror.registry"] capabilities = ["pull"] [host."http://double-mirror.registry"] capabilities = ["pull"] ```