Project: KubeVirt (App Definition and Development / Application Definition & Image Build)
File: build-the-builder.md

The KubeVirt build system starts with a builder helper container, where the entire toolchain and its dependencies are stored, in order to ensure consistency between builds. Yet, there are sometimes CVEs that must be remedied with an update of the build toolchain, and from time to time, KubeVirt must be enabled for a new system architecture, e.g. `ppc64le` or `s390x`. The KubeVirt official build stores its builder container and built images in a protected quay.io registry. If there is no builder image for the architecture of the machine you are trying to build KubeVirt on and for, `make cluster-up` will fail in its attempt to pull a builder image. You will need to build your own builder image, and tweak the build scripts to refer to it. This document will provide the steps to: prepare the local environment to build the builder container build the builder container push the builder image to a custom registry how to update the rpm dependencies with bazeldnf invoke the new test builder image to build new kubevirt operator images push the new kubevirt operator images to a custom registry install podman or docker On Ubuntu, ``` $ apt get install -y podman ``` or ``` $ apt get install -y docker ``` On RHEL or CENTOS ``` $ dnf install -y podman ``` You will first need to create read and write access to an appropriate namespace in a container registry or or are popular registries to use for this purpose. Then set up a repository or namespace to hold the container you are building, e.g. `builder` or, where the namespace needs to be more unique, `kubevirt-builder`. Bazel binary distribution images for version 5.3.0 for the Arm and x86 architectures are available from https://github.com/bazelbuild/bazel/releases/tag/5.3.0 This, the dependent libraries and build script configuration will constitute the containerized KubeVirt build toolchain. For architectures other than arm and x86, e.g. IBM ppc64le and s390x, you need to build bazel from scratch on your build-the-builder box. Currently, step 1.2 'Build Bazel v5.3.0'. The Dockerfile you'll be building with that bazel installation expects to find the bazel executable in `kubevirt/hack/builder/bazel`, so copy it there,

``` $ cp /usr/local/bin/bazel ~/kubevirt/hack/builder/bazel ``` Note that the default system architecture is amd64, so if you do not set ARCH as an environment variable, you will build a docker container with x86 object format binaries in it. ``` export ARCH="<your-system-architecture>" export DOCKER_PREFIX="<your-registry-URL>/<your-kubevirt-namespace>" export DOCKER_IMAGE="builder" export VERSION="<your-docker-tag>" export KUBEVIRTBUILDERIMAGE="${DOCKERPREFIX}/${DOCKERIMAGE}/${VERSION}" ``` where <your-system-architecture> is either `amd64` or `arm64` or `s390x` or `ppc64le` for x86, ARM, s390x or Power systems, respectively. `make builder-build` invokes the build of the bazel builder container. This tags your image with namespace you set up in your environment variables. Now you can ``` $ make builder-publish ``` which, among many other things, pushes the image up to the container registry of choice. Note that you will be pushing these changes to your test repository, in order to use this builder image to build kubevirt test operators in further steps. edit your local copy of this section of the file `hack/rpm-deps.sh`: ``` LIBVIRTVERSION=${LIBVIRTVERSION:-0:9.5.0-5.el9} QEMUVERSION=${QEMUVERSION:-17:8.0.0-13.el9} SEABIOSVERSION=${SEABIOSVERSION:-0:1.16.1-1.el9} EDK2VERSION=${EDK2VERSION:-0:20221207gitfff6d81270b5-9} LIBGUESTFSVERSION=${LIBGUESTFSVERSION:-1:1.48.4-4.el9} GUESTFSTOOLSVERSION=${GUESTFSTOOLSVERSION:-0:1.48.2-8.el9} PASSTVERSION=${PASSTVERSION:-0:0^20230818.g0af928e-4.el9} VIRTIOFSDVERSION=${VIRTIOFSDVERSION:-0:1.5.0-1.el9} SWTPMVERSION=${SWTPMVERSION:-0:0.8.0-1.el9} SINGLEARCH=${SINGLEARCH:-""} BASESYSTEM=${BASESYSTEM:-"centos-stream-release"} ``` Update the version numbers of the rpms in accordance with the latest versions observable in https://mirror.stream.centos.org/9-stream/AppStream/$ARCH/os/Packages/ If you do not have a version that can be looked up in the CentOS rpm collection, the build of the bazel helper container for KubeVirt will fail. Once you get it working, you can consider incorporating your local versions of the RPM dependencies in a PR, but careful to check for CVEs and other issues in the dependencies you specified. If you are building the builder on Power, x86 or arm architecture you can get a pre-built binary of the `bazeldnf` utility from https://github.com/rmohr/bazeldnf/releases If you are building the builder container natively on s390x, you will need to compile it from source, which is also provided at https://github.com/rmohr/bazeldnf/releases Now run a script which you can name `./fix-rpm-deps.sh` to update these versions throughout the installation with bazeldnf: ``` set -ex BASESYSTEM=${BASESYSTEM:-"centos-stream-release"} bazeldnf_repos="--repofile rpm/repo.yaml" centos_main=" acl curl-minimal vim-minimal " centos_extra=" coreutils-single glibc-minimal-langpack libcurl-minimal " sandboxroot_main=" findutils gcc glibc-static python3 sssd-client " bazeldnf fetch \ ${bazeldnf_repos} bazeldnf rpmtree \ --public --nobest \ --name sandboxroot_s390x --arch s390x \ --basesystem ${BASESYSTEM} \ ${bazeldnf_repos} \ $centos_main \ $centos_extra \ $sandboxroot_main SINGLE_ARCH=<your-system-architecture> make rpm-deps ``` where <your-system-architecture> is `amd64`, `arm64`, `ppc64le` or `s390x` Because in this instance, you are using your test registry and potentially an architecture other than x86, before running `make && make bazel-push-images && make manifests` as in , set the necessary environment variables for these make targets: ``` export BUILD_ARCH="<your-system-architecture" export DOCKER_PREFIX="<your-registry-URL>/<your-kubevirt-namespace>" export QUAY_REPOSITORY="kubevirt" export PACKAGE_NAME="kubevirt-operatorhub" ```