Chapter 2. Obtaining and modifying container images A containerized overcloud requires access to a registry with the required container images. This chapter provides information on how to prepare the registry and your undercloud and overcloud configuration to use container images for Red Hat OpenStack Platform. 2.1. Preparing container images The overcloud installation requires an environment file to determine where to obtain container images and how to store them. Generate and customize this environment file that you can use to prepare your container images. Note If you need to configure specific container image versions for your overcloud, you must pin the images to a specific version. For more information, see Pinning container images for the overcloud . Procedure Log in to your undercloud host as the stack user. Generate the default container image preparation file: This command includes the following additional options: --local-push-destination sets the registry on the undercloud as the location for container images. This means that director pulls the necessary images from the Red Hat Container Catalog and pushes them to the registry on the undercloud. Director uses this registry as the container image source. To pull directly from the Red Hat Container Catalog, omit this option. --output-env-file is an environment file name. The contents of this file include the parameters for preparing your container images. In this case, the name of the file is containers-prepare-parameter.yaml . Note You can use the same containers-prepare-parameter.yaml file to define a container image source for both the undercloud and the overcloud. Modify the containers-prepare-parameter.yaml to suit your requirements. 2.2. Container image preparation parameters The default file for preparing your containers ( containers-prepare-parameter.yaml ) contains the ContainerImagePrepare heat parameter. This parameter defines a list of strategies for preparing a set of images: Each strategy accepts a set of sub-parameters that defines which images to use and what to do with the images. The following table contains information about the sub-parameters that you can use with each ContainerImagePrepare strategy: Parameter Description excludes List of regular expressions to exclude image names from a strategy. includes List of regular expressions to include in a strategy. At least one image name must match an existing image. All excludes are ignored if includes is specified. modify_append_tag String to append to the tag for the destination image. For example, if you pull an image with the tag 16.2.3-5.161 and set the modify_append_tag to -hotfix , the director tags the final image as 16.2.3-5.161-hotfix. modify_only_with_labels A dictionary of image labels that filter the images that you want to modify. If an image matches the labels defined, the director includes the image in the modification process. modify_role String of ansible role names to run during upload but before pushing the image to the destination registry. modify_vars Dictionary of variables to pass to modify_role . push_destination Defines the namespace of the registry that you want to push images to during the upload process. If set to true , the push_destination is set to the undercloud registry namespace using the hostname, which is the recommended method. If set to false , the push to a local registry does not occur and nodes pull images directly from the source. If set to a custom value, director pushes images to an external local registry. If you set this parameter to false in production environments while pulling images directly from Red Hat Container Catalog, all overcloud nodes will simultaneously pull the images from the Red Hat Container Catalog over your external connection, which can cause bandwidth issues. Only use false to pull directly from a Red Hat Satellite Server hosting the container images. If the push_destination parameter is set to false or is not defined and the remote registry requires authentication, set the ContainerImageRegistryLogin parameter to true and include the credentials with the ContainerImageRegistryCredentials parameter. pull_source The source registry from where to pull the original container images. set A dictionary of key: value definitions that define where to obtain the initial images. tag_from_label Use the value of specified container image metadata labels to create a tag for every image and pull that tagged image. For example, if you set tag_from_label: {version}-{release} , director uses the version and release labels to construct a new tag. For one container, version might be set to 16.2.3 and release might be set to 5.161 , which results in the tag 16.2.3-5.161. Director uses this parameter only if you have not defined tag in the set dictionary. Important When you push images to the undercloud, use push_destination: true instead of push_destination: UNDERCLOUD_IP:PORT . The push_destination: true method provides a level of consistency across both IPv4 and IPv6 addresses. The set parameter accepts a set of key: value definitions: Key Description ceph_image The name of the Ceph Storage container image. ceph_namespace The namespace of the Ceph Storage container image. ceph_tag The tag of the Ceph Storage container image. ceph_alertmanager_image ceph_alertmanager_namespace ceph_alertmanager_tag The name, namespace, and tag of the Ceph Storage Alert Manager container image. ceph_grafana_image ceph_grafana_namespace ceph_grafana_tag The name, namespace, and tag of the Ceph Storage Grafana container image. ceph_node_exporter_image ceph_node_exporter_namespace ceph_node_exporter_tag The name, namespace, and tag of the Ceph Storage Node Exporter container image. ceph_prometheus_image ceph_prometheus_namespace ceph_prometheus_tag The name, namespace, and tag of the Ceph Storage Prometheus container image. name_prefix A prefix for each OpenStack service image. name_suffix A suffix for each OpenStack service image. namespace The namespace for each OpenStack service image. neutron_driver The driver to use to determine which OpenStack Networking (neutron) container to use. Use a null value to set to the standard neutron-server container. Set to ovn to use OVN-based containers. tag Sets a specific tag for all images from the source. If not defined, director uses the Red Hat OpenStack Platform version number as the default value. This parameter takes precedence over the tag_from_label value. Note The container images use multi-stream tags based on the Red Hat OpenStack Platform version. This means that there is no longer a latest tag. 2.3. Guidelines for container image tagging The Red Hat Container Registry uses a specific version format to tag all Red Hat OpenStack Platform container images. This format follows the label metadata for each container, which is version-release . version Corresponds to a major and minor version of Red Hat OpenStack Platform. These versions act as streams that contain one or more releases. release Corresponds to a release of a specific container image version within a version stream. For example, if the latest version of Red Hat OpenStack Platform is 16.2.3 and the release for the container image is 5.161 , then the resulting tag for the container image is 16.2.3-5.161. The Red Hat Container Registry also uses a set of major and minor version tags that link to the latest release for that container image version. For example, both 16.2 and 16.2.3 link to the latest release in the 16.2.3 container stream. If a new minor release of 16.2 occurs, the 16.2 tag links to the latest release for the new minor release stream while the 16.2.3 tag continues to link to the latest release within the 16.2.3 stream. The ContainerImagePrepare parameter contains two sub-parameters that you can use to determine which container image to download. These sub-parameters are the tag parameter within the set dictionary, and the tag_from_label parameter. Use the following guidelines to determine whether to use tag or tag_from_label . The default value for tag is the major version for your OpenStack Platform version. For this version it is 16.2. This always corresponds to the latest minor version and release. To change to a specific minor version for OpenStack Platform container images, set the tag to a minor version. For example, to change to 16.2.2, set tag to 16.2.2. When you set tag , director always downloads the latest container image release for the version set in tag during installation and updates. If you do not set tag , director uses the value of tag_from_label in conjunction with the latest major version. The tag_from_label parameter generates the tag from the label metadata of the latest container image release it inspects from the Red Hat Container Registry. For example, the labels for a certain container might use the following version and release metadata: The default value for tag_from_label is {version}-{release} , which corresponds to the version and release metadata labels for each container image. For example, if a container image has 16.2.3 set for version and 5.161 set for release , the resulting tag for the container image is 16.2.3-5.161. The tag parameter always takes precedence over the tag_from_label parameter. To use tag_from_label , omit the tag parameter from your container preparation configuration. A key difference between tag and tag_from_label is that director uses tag to pull an image only based on major or minor version tags, which the Red Hat Container Registry links to the latest image release within a version stream, while director uses tag_from_label to perform a metadata inspection of each container image so that director generates a tag and pulls the corresponding image. 2.4. Obtaining container images from private registries The registry.redhat.io registry requires authentication to access and pull images. To authenticate with registry.redhat.io and other private registries, include the ContainerImageRegistryCredentials and ContainerImageRegistryLogin parameters in your containers-prepare-parameter.yaml file. ContainerImageRegistryCredentials Some container image registries require authentication to access images. In this situation, use the ContainerImageRegistryCredentials parameter in your containers-prepare-parameter.yaml environment file. The ContainerImageRegistryCredentials parameter uses a set of keys based on the private registry URL. Each private registry URL uses its own key and value pair to define the username (key) and password (value). This provides a method to specify credentials for multiple private registries. In the example, replace my_username and my_password with your authentication credentials. Instead of using your individual user credentials, Red Hat recommends creating a registry service account and using those credentials to access registry.redhat.io content. To specify authentication details for multiple registries, set multiple key-pair values for each registry in ContainerImageRegistryCredentials : Important The default ContainerImagePrepare parameter pulls container images from registry.redhat.io , which requires authentication. For more information, see Red Hat Container Registry Authentication . ContainerImageRegistryLogin The ContainerImageRegistryLogin parameter is used to control whether an overcloud node system needs to log in to the remote registry to fetch the container images. This situation occurs when you want the overcloud nodes to pull images directly, rather than use the undercloud to host images. You must set ContainerImageRegistryLogin to true if push_destination is set to false or not used for a given strategy. However, if the overcloud nodes do not have network connectivity to the registry hosts defined in ContainerImageRegistryCredentials and you set ContainerImageRegistryLogin to true , the deployment might fail when trying to perform a login. If the overcloud nodes do not have network connectivity to the registry hosts defined in the ContainerImageRegistryCredentials , set push_destination to true and ContainerImageRegistryLogin to false so that the overcloud nodes pull images from the undercloud. 2.5. Layering image preparation entries The value of the ContainerImagePrepare parameter is a YAML list. This means that you can specify multiple entries. The following example demonstrates two entries where director uses the latest version of all images except for the nova-api image, which uses the version tagged with 16.2.1-hotfix : The includes and excludes parameters use regular expressions to control image filtering for each entry. The images that match the includes strategy take precedence over excludes matches. The image name must match the includes or excludes regular expression value to be considered a match. A similar technique is used if your Block Storage (cinder) driver requires a vendor supplied cinder-volume image known as a plugin. If your Block Storage driver requires a plugin, see Deploying a vendor plugin in the Advanced Overcloud Customization guide. 2.6. Modifying images during preparation It is possible to modify images during image preparation, and then immediately deploy the overcloud with modified images. Note Red Hat OpenStack Platform (RHOSP) director supports modifying images during preparation for RHOSP containers, not for Ceph containers. Scenarios for modifying images include: As part of a continuous integration pipeline where images are modified with the changes being tested before deployment. As part of a development workflow where local changes must be deployed for testing and development. When changes must be deployed but are not available through an image build pipeline. For example, adding proprietary add-ons or emergency fixes. To modify an image during preparation, invoke an Ansible role on each image that you want to modify. The role takes a source image, makes the requested changes, and tags the result. The prepare command can push the image to the destination registry and set the heat parameters to refer to the modified image. The Ansible role tripleo-modify-image conforms with the required role interface and provides the behaviour necessary for the modify use cases. Control the modification with the modify-specific keys in the ContainerImagePrepare parameter: modify_role specifies the Ansible role to invoke for each image to modify. modify_append_tag appends a string to the end of the source image tag. This makes it obvious that the resulting image has been modified. Use this parameter to skip modification if the push_destination registry already contains the modified image. Change modify_append_tag whenever you modify the image. modify_vars is a dictionary of Ansible variables to pass to the role. To select a use case that the tripleo-modify-image role handles, set the tasks_from variable to the required file in that role. While developing and testing the ContainerImagePrepare entries that modify images, run the image prepare command without any additional options to confirm that the image is modified as you expect: Important To use the openstack tripleo container image prepare command, your undercloud must contain a running image-serve registry. As a result, you cannot run this command before a new undercloud installation because the image-serve registry will not be installed. You can run this command after a successful undercloud installation. 2.7. Updating existing packages on container images Note Red Hat OpenStack Platform (RHOSP) director supports updating existing packages on container images for RHOSP containers, not for Ceph containers. Procedure The following example ContainerImagePrepare entry updates in all packages on the container images by using the dnf repository configuration of the undercloud host: 2.8. Installing additional RPM files to container images You can install a directory of RPM files in your container images. This is useful for installing hotfixes, local package builds, or any package that is not available through a package repository. Note Red Hat OpenStack Platform (RHOSP) director supports installing additional RPM files to container images for RHOSP containers, not for Ceph containers. Note When you modify container images in existing deployments, you must then perform a minor update to apply the changes to your overcloud. For more information, see Keeping Red Hat OpenStack Platform Updated . Procedure The following example ContainerImagePrepare entry installs some hotfix packages on only the nova-compute image: 2.9. Modifying container images with a custom Dockerfile You can specify a directory that contains a Dockerfile to make the required changes. When you invoke the tripleo-modify-image role, the role generates a Dockerfile.modified file that changes the FROM directive and adds extra LABEL directives. Note Red Hat OpenStack Platform (RHOSP) director supports modifying container images with a custom Dockerfile for RHOSP containers, not for Ceph containers. Procedure The following example runs the custom Dockerfile on the nova-compute image: The following example shows the /home/stack/nova-custom/Dockerfile file. After you run any USER root directives, you must switch back to the original image default user: 2.10. Preparing a Satellite server for container images Red Hat Satellite 6 offers registry synchronization capabilities. This provides a method to pull multiple images into a Satellite server and manage them as part of an application life cycle. The Satellite also acts as a registry for other container-enabled systems to use. For more information about managing container images, see Managing Container Images in the Red Hat Satellite 6 Content Management Guide . The examples in this procedure use the hammer command line tool for Red Hat Satellite 6 and an example organization called ACME . Substitute this organization for your own Satellite 6 organization. Note This procedure requires authentication credentials to access container images from registry.redhat.io . Instead of using your individual user credentials, Red Hat recommends creating a registry service account and using those credentials to access registry.redhat.io content. For more information, see "Red Hat Container Registry Authentication" . Procedure Create a list of all container images: If you plan to install Ceph and enable the Ceph Dashboard, you need the following ose-prometheus containers: Copy the satellite_images file to a system that contains the Satellite 6 hammer tool. Alternatively, use the instructions in the Hammer CLI Guide to install the hammer tool to the undercloud. Run the following hammer command to create a new product ( OSP Containers ) in your Satellite organization: This custom product will contain your images. Add the overcloud container images from the satellite_images file: Add the Ceph Storage 4 container image: Note If you want to install the Ceph dashboard, include --name rhceph-4-dashboard-rhel8 in the hammer repository create command: Synchronize the container images: Wait for the Satellite server to complete synchronization. Note Depending on your configuration, hammer might ask for your Satellite server username and password. You can configure hammer to automatically login using a configuration file. For more information, see the Authentication section in the Hammer CLI Guide . If your Satellite 6 server uses content views, create a new content view version to incorporate the images and promote it along environments in your application life cycle. This largely depends on how you structure your application lifecycle. For example, if you have an environment called production in your lifecycle and you want the container images to be available in that environment, create a content view that includes the container images and promote that content view to the production environment. For more information, see Managing Content Views . Check the available tags for the base image: This command displays tags for the OpenStack Platform container images within a content view for a particular environment. Return to the undercloud and generate a default environment file that prepares images using your Satellite server as a source. Run the following example command to generate the environment file: --output-env-file is an environment file name. The contents of this file include the parameters for preparing your container images for the undercloud. In this case, the name of the file is containers-prepare-parameter.yaml . Edit the containers-prepare-parameter.yaml file and modify the following parameters: push_destination - Set this to true or false depending on your chosen container image management strategy. If you set this parameter to false , the overcloud nodes pull images directly from the Satellite. If you set this parameter to true , the director pulls the images from the Satellite to the undercloud registry and the overcloud pulls the images from the undercloud registry. namespace - The URL and port of the registry on the Satellite server. The default registry port on Red Hat Satellite is 443. name_prefix - The prefix is based on a Satellite 6 convention. This differs depending on whether you use content views: If you use content views, the structure is [org]-[environment]-[content view]-[product]- . For example: acme-production-myosp16-osp_containers- . If you do not use content views, the structure is [org]-[product]- . For example: acme-osp_containers- . ceph_namespace , ceph_image , ceph_tag - If you use Ceph Storage, include these additional parameters to define the Ceph Storage container image location. Note that ceph_image now includes a Satellite-specific prefix. This prefix is the same value as the name_prefix option. The following example environment file contains Satellite-specific parameters: Note To use a specific container image version stored on your Red Hat Satellite Server, set the tag key-value pair to the specific version in the set dictionary. For example, to use the 16.2.2 image stream, set tag: 16.2.2 in the set dictionary. You must define the containers-prepare-parameter.yaml environment file in the undercloud.conf configuration file, otherwise the undercloud uses the default values:

Commands:
sudo openstack tripleo container image prepare default --local-push-destination --output-env-file containers-prepare-parameter.yaml
parameter_defaults: ContainerImagePrepare: - (strategy one) - (strategy two) - (strategy three)
parameter_defaults: ContainerImagePrepare: - set: tag: 16.2
parameter_defaults: ContainerImagePrepare: - set: tag: 16.2.2
parameter_defaults: ContainerImagePrepare: - set: # tag: 16.2 tag_from_label: '{version}-{release}'
"Labels": { "release": "5.161", "version": "16.2.3", }
parameter_defaults: ContainerImagePrepare: - push_destination: true set: namespace: registry.redhat.io/ ContainerImageRegistryCredentials: registry.redhat.io: my_username: my_password
parameter_defaults: ContainerImagePrepare: - push_destination: true set: namespace: registry.redhat.io/ - push_destination: true set: namespace: registry.internalsite.com/ ContainerImageRegistryCredentials: registry.redhat.io: myuser: 'p@55w0rd!' registry.internalsite.com: myuser2: '0th3rp@55w0rd!' '192.0.2.1:8787': myuser3: '@n0th3rp@55w0rd!'
parameter_defaults: ContainerImagePrepare: - push_destination: false set: namespace: registry.redhat.io/ ContainerImageRegistryCredentials: registry.redhat.io: myuser: 'p@55w0rd!' ContainerImageRegistryLogin: true
parameter_defaults: ContainerImagePrepare: - push_destination: true set: namespace: registry.redhat.io/ ContainerImageRegistryCredentials: registry.redhat.io: myuser: 'p@55w0rd!' ContainerImageRegistryLogin: false
parameter_defaults: ContainerImagePrepare: - tag_from_label: "{version}-{release}" push_destination: true excludes: - nova-api set: namespace: registry.redhat.io/rhosp-rhel8 name_prefix: openstack- name_suffix: '' tag:16.2 - push_destination: true includes: - nova-api set: namespace: registry.redhat.io/rhosp-rhel8 tag: 16.2.1-hotfix
sudo openstack tripleo container image prepare -e ~/containers-prepare-parameter.yaml
ContainerImagePrepare: - push_destination: true modify_role: tripleo-modify-image modify_append_tag: "-updated" modify_vars: tasks_from: yum_update.yml compare_host_packages: true yum_repos_dir_path: /etc/yum.repos.d
ContainerImagePrepare: - push_destination: true includes: - nova-compute modify_role: tripleo-modify-image modify_append_tag: "-hotfix" modify_vars: tasks_from: rpm_install.yml rpms_path: /home/stack/nova-hotfix-pkgs
ContainerImagePrepare: - push_destination: true includes: - nova-compute modify_role: tripleo-modify-image modify_append_tag: "-hotfix" modify_vars: tasks_from: modify_image.yml modify_dir_path: /home/stack/nova-custom
FROM registry.redhat.io/rhosp-rhel8/openstack-nova-compute:latest USER "root" COPY customize.sh /tmp/ RUN /tmp/customize.sh USER "nova"
sudo podman search --limit 1000 "registry.redhat.io/rhosp-rhel8/openstack" --format="{{ .Name }}" | sort > satellite_images sudo podman search --limit 1000 "registry.redhat.io/rhceph" | grep rhceph-4-dashboard-rhel8 sudo podman search --limit 1000 "registry.redhat.io/rhceph" | grep rhceph-4-rhel8 sudo podman search --limit 1000 "registry.redhat.io/openshift" | grep ose-prometheus
registry.redhat.io/openshift4/ose-prometheus-node-exporter:v4.6 registry.redhat.io/openshift4/ose-prometheus:v4.6 registry.redhat.io/openshift4/ose-prometheus-alertmanager:v4.6
hammer product create --organization "ACME" --name "OSP Containers"
while read IMAGE; do IMAGE_NAME=USD(echo USDIMAGE | cut -d"/" -f3 | sed "s/openstack-//g") ; IMAGE_NOURL=USD(echo USDIMAGE | sed "s/registry.redhat.io\///g") ; hammer repository create --organization "ACME" --product "OSP Containers" --content-type docker --url https://registry.redhat.io --docker-upstream-name USDIMAGE_NOURL --upstream-username USERNAME --upstream-password PASSWORD --name USDIMAGE_NAME ; done < satellite_images
hammer repository create --organization "ACME" --product "OSP Containers" --content-type docker --url https://registry.redhat.io --docker-upstream-name rhceph/rhceph-4-rhel8 --upstream-username USERNAME --upstream-password PASSWORD --name rhceph-4-rhel8
hammer repository create --organization "ACME" --product "OSP Containers" --content-type docker --url https://registry.redhat.io --docker-upstream-name rhceph/rhceph-4-dashboard-rhel8 --upstream-username USERNAME --upstream-password PASSWORD --name rhceph-4-dashboard-rhel8
hammer product synchronize --organization "ACME" --name "OSP Containers"
hammer docker tag list --repository "base" --organization "ACME" --lifecycle-environment "production" --product "OSP Containers"
sudo openstack tripleo container image prepare default --output-env-file containers-prepare-parameter.yaml
parameter_defaults: ContainerImagePrepare: - push_destination: false set: ceph_image: acme-production-myosp16_1-osp_containers-rhceph-4 ceph_namespace: satellite.example.com:443 ceph_tag: latest name_prefix: acme-production-myosp16_1-osp_containers- name_suffix: '' namespace: satellite.example.com:443 neutron_driver: null tag: '16.2'
container_images_file = /home/stack/containers-prepare-parameter.yaml