Chapter 5. Installing RHEL AI on Google Cloud Platform (GCP) (Technology preview) For installing and deploying Red Hat Enterprise Linux AI on Google Cloud Platform, you must first convert the RHEL AI image into an GCP image. You can then launch an instance using the GCP image and deploy RHEL AI on a Google Cloud Platform machine. Important Installing Red Hat Enterprise Linux AI on GCP is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . 5.1. Converting the RHEL AI image into a Google Cloud Platform image. To create a bootable image in Google Cloud Platform you must configure your Google Cloud Platform account, create an Google Cloud Storage bucket, and create an Google Cloud Platform image using the RHEL AI raw image. Prerequisites You installed the Google Cloud Platform CLI on your specific machine. For more information about installing the GCP CLI, see Install the Google Cloud Platform CLI on Linux . You must be on a Red Hat Enterprise Linux version 9.2 - 9.4 system Your machine must have an additional 100 GB of disk space. Procedure Log in to Google Cloud Platform with the following command: USD gcloud auth login Example output of the login. USD gcloud auth login Your browser has been opened to visit: https://accounts.google.com/o/oauth2/auth?XXXXXXXXXXXXXXXXXXXX You are now logged in as [user@example.com]. Your current project is [your-project]. You can change this setting by running: USD gcloud config set project PROJECT_ID You need to set up some Google Cloud Platform configurations and create your GCP Storage Container before creating the GCP image. Configure Google Cloud Platform CLI to use your project. USD gcloud_project=your-gcloud-project USD gcloud config set project USDgcloud_project Create an environment variable defining the region where you want to operate. USD gcloud_region=us-central1 Create a Google Cloud Platform Storage Container. USD gcloud_bucket=name-for-your-bucket USD gsutil mb -l USDgcloud_region gs://USDgcloud_bucket Red Hat currently does not provide RHEL AI Google Cloud Platform images. You need to create a GCP disk image using RHEL AI bootc image as base. Create this Containerfile file, using the appropriate version of RHEL AI in the FROM line. FROM registry.redhat.io/rhelai1/bootc-nvidia-rhel9:1.2 RUN eval USD(grep VERSION_ID /etc/os-release) \ && echo -e "[google-compute-engine]\nname=Google Compute Engine\nbaseurl=https://packages.cloud.google.com/yum/repos/google-compute-engine-elUSD{VERSION_ID/.*}-x86_64-stable\nenabled=1\ngpgcheck=1\nrepo_gpgcheck=0\ngpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg\n https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg" > /etc/yum.repos.d/google-cloud.repo \ && dnf install -y --nobest \ acpid \ cloud-init \ google-compute-engine \ google-osconfig-agent \ langpacks-en \ rng-tools \ timedatex \ tuned \ vim \ && curl -sSo /tmp/add-google-cloud-ops-agent-repo.sh https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh \ && bash /tmp/add-google-cloud-ops-agent-repo.sh --also-install --remove-repo \ && rm /tmp/add-google-cloud-ops-agent-repo.sh \ && mkdir -p /var/lib/rpm-state \ && dnf remove -y irqbalance microcode_ctl \ && rmdir /var/lib/rpm-state \ && rm -f /etc/yum.repos.d/google-cloud.repo \ && sed -i -e '/^pool /c\server metadata.google.internal iburst' /etc/chrony.conf \ && echo -e 'PermitRootLogin no\nPasswordAuthentication no\nClientAliveInterval 420' >> /etc/ssh/sshd_config \ && echo -e '[InstanceSetup]\nset_boto_config = false' > /etc/default/instance_configs.cfg \ && echo 'blacklist floppy' > /etc/modprobe.d/blacklist_floppy.conf \ && echo -e '[install]\nkargs = ["net.ifnames=0", "biosdevname=0", "scsi_mod.use_blk_mq=Y", "console=ttyS0,38400n8d", "cloud-init=disabled"]' > /usr/lib/bootc/install/05-cloud-kargs.toml Build the bootc image, in the same directory that holds the Containerfile , by running the following commands: USD GCP_BOOTC_IMAGE=quay.io/yourquayusername/bootc-nvidia-rhel9-gcp USD podman build --file Containerfile --tag USD{GCP_BOOTC_IMAGE} . Note Ensure you are running the podman build command from a RHEL enabled system. If you are not on a RHEL system, building the Containerfile file will fail. Create a config.toml file that will be used in disk image generation. [customizations.kernel] name = "gcp" append = "net.ifnames=0 biosdevname=0 scsi_mod.use_blk_mq=Y console=ttyS0,38400n8d cloud-init=disabled" Build the disk image using bootc-image-builder by running the following commands: USD mkdir -p build/store build/output USD podman run --rm -ti --privileged --pull newer \ -v /var/lib/containers/storage:/var/lib/containers/storage \ -v ./build/store:/store -v ./build/output:/output \ -v ./config.toml:/config.toml \ quay.io/centos-bootc/bootc-image-builder \ --config /config.toml \ --chown 0:0 \ --local \ --type raw \ --target-arch x86_64 \ USD{GCP_BOOTC_IMAGE} Set the name you want to use as the RHEL AI Google Cloud Platform image. USD image_name=rhel-ai-1-2 Create a tar.gz file containing the RAW file you created. USD raw_file=<path-to-raw-file> USD tar cf rhelai_gcp.tar.gz --transform "s|USDraw_file|disk.raw|" --use-compress-program=pigz "USDraw_file" Note You can use gzip instead of pigz . Upload the tar.gz file to the Google Cloud Platform Storage Container by running the following command: USD gsutil cp rhelai_gcp.tar.gz "gs://USD{gcloud_bucket}/USDimage_name.tar.gz" Create an Google Cloud Platform image from the tar.gz file you just uploaded with the following command: USD gcloud compute images create \ "USDimage_name" \ --source-uri="gs://USD{gcloud_bucket}/USDimage_name.tar.gz" \ --family "rhel-ai" \ --guest-os-features=GVNIC 5.2. Deploying your instance on Google Cloud Platform using the CLI You can launch an instance with your new RHEL AI Google Cloud Platform image from the Google Cloud Platform web console or the CLI. You can use whichever method of deployment you want to launch your instance. The following procedure displays how you can use the CLI to launch an Google Cloud Platform instance with the custom Google Cloud Platform image If you choose to use the CLI as a deployment option, there are several configurations you have to create, as shown in "Prerequisites". Prerequisites You created your RHEL AI Google Cloud Platform image. For more information, see "Converting the RHEL AI image to a Google Cloud Platform image". You installed the Google Cloud Platform CLI on your specific machine, see Install the Google Cloud Platform CLI on Linux . Procedure Log in to your Google Cloud Platform account by running the following command: USD gcloud auth login Before launching your Google Cloud Platform instance on the CLI, you need to create several configuration variables for your instance. You need to select the instance profile that you want to use for the deployment. List all the profiles in the desired region by running the following command: USD gcloud compute machine-types list --zones=<zone> Make a note of your preferred machine type, you will need it for your instance deployment. You can now start creating your Google Cloud Platform instance. Populate environment variables for when you create the instance. name=my-rhelai-instance zone=us-central1-a machine_type=a3-highgpu-8g accelerator="type=nvidia-h100-80gb,count=8" image=my-custom-rhelai-image disk_size=1024 subnet=default Configure the zone to be used. USD gcloud config set compute/zone USDzone You can now launch your instance, by running the following command: USD gcloud compute instances create \ USD{name} \ --machine-type USD{machine_type} \ --image USDimage \ --zone USDzone \ --subnet USDsubnet \ --boot-disk-size USD{disk_size} \ --boot-disk-device-name USD{name} \ --accelerator=USDaccelerator Verification To verify that your Red Hat Enterprise Linux AI tools are installed correctly, run the ilab command: USD ilab Example output USD ilab Usage: ilab [OPTIONS] COMMAND [ARGS]... CLI for interacting with InstructLab. If this is your first time running ilab, it's best to start with `ilab config init` to create the environment. Options: --config PATH Path to a configuration file. [default: /home/auser/.config/instructlab/config.yaml] -v, --verbose Enable debug logging (repeat for even more verbosity) --version Show the version and exit. --help Show this message and exit. Commands: config Command Group for Interacting with the Config of InstructLab. data Command Group for Interacting with the Data generated by... model Command Group for Interacting with the Models in InstructLab. system Command group for all system-related command calls taxonomy Command Group for Interacting with the Taxonomy of InstructLab. Aliases: chat model chat convert model convert diff taxonomy diff download model download evaluate model evaluate generate data generate init config init list model list serve model serve sysinfo system info test model test train model train

Commands:
gcloud auth login
gcloud auth login Your browser has been opened to visit: https://accounts.google.com/o/oauth2/auth?XXXXXXXXXXXXXXXXXXXX You are now logged in as [user@example.com]. Your current project is [your-project]. You can change this setting by running: USD gcloud config set project PROJECT_ID
gcloud_project=your-gcloud-project gcloud config set project USDgcloud_project
gcloud_region=us-central1
gcloud_bucket=name-for-your-bucket gsutil mb -l USDgcloud_region gs://USDgcloud_bucket
FROM registry.redhat.io/rhelai1/bootc-nvidia-rhel9:1.2 RUN eval USD(grep VERSION_ID /etc/os-release) && echo -e "[google-compute-engine]\nname=Google Compute Engine\nbaseurl=https://packages.cloud.google.com/yum/repos/google-compute-engine-elUSD{VERSION_ID/.*}-x86_64-stable\nenabled=1\ngpgcheck=1\nrepo_gpgcheck=0\ngpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg\n https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg" > /etc/yum.repos.d/google-cloud.repo && dnf install -y --nobest acpid cloud-init google-compute-engine google-osconfig-agent langpacks-en rng-tools timedatex tuned vim && curl -sSo /tmp/add-google-cloud-ops-agent-repo.sh https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh && bash /tmp/add-google-cloud-ops-agent-repo.sh --also-install --remove-repo && rm /tmp/add-google-cloud-ops-agent-repo.sh && mkdir -p /var/lib/rpm-state && dnf remove -y irqbalance microcode_ctl && rmdir /var/lib/rpm-state && rm -f /etc/yum.repos.d/google-cloud.repo && sed -i -e '/^pool /c\server metadata.google.internal iburst' /etc/chrony.conf && echo -e 'PermitRootLogin no\nPasswordAuthentication no\nClientAliveInterval 420' >> /etc/ssh/sshd_config && echo -e '[InstanceSetup]\nset_boto_config = false' > /etc/default/instance_configs.cfg && echo 'blacklist floppy' > /etc/modprobe.d/blacklist_floppy.conf && echo -e '[install]\nkargs = ["net.ifnames=0", "biosdevname=0", "scsi_mod.use_blk_mq=Y", "console=ttyS0,38400n8d", "cloud-init=disabled"]' > /usr/lib/bootc/install/05-cloud-kargs.toml
GCP_BOOTC_IMAGE=quay.io/yourquayusername/bootc-nvidia-rhel9-gcp podman build --file Containerfile --tag USD{GCP_BOOTC_IMAGE} .
[customizations.kernel] name = "gcp" append = "net.ifnames=0 biosdevname=0 scsi_mod.use_blk_mq=Y console=ttyS0,38400n8d cloud-init=disabled"
mkdir -p build/store build/output podman run --rm -ti --privileged --pull newer -v /var/lib/containers/storage:/var/lib/containers/storage -v ./build/store:/store -v ./build/output:/output -v ./config.toml:/config.toml quay.io/centos-bootc/bootc-image-builder --config /config.toml --chown 0:0 --local --type raw --target-arch x86_64 USD{GCP_BOOTC_IMAGE}
image_name=rhel-ai-1-2
raw_file=<path-to-raw-file> tar cf rhelai_gcp.tar.gz --transform "s|USDraw_file|disk.raw|" --use-compress-program=pigz "USDraw_file"
gsutil cp rhelai_gcp.tar.gz "gs://USD{gcloud_bucket}/USDimage_name.tar.gz"
gcloud compute images create "USDimage_name" --source-uri="gs://USD{gcloud_bucket}/USDimage_name.tar.gz" --family "rhel-ai" --guest-os-features=GVNIC
gcloud auth login
gcloud compute machine-types list --zones=<zone>
name=my-rhelai-instance zone=us-central1-a machine_type=a3-highgpu-8g accelerator="type=nvidia-h100-80gb,count=8" image=my-custom-rhelai-image disk_size=1024 subnet=default
gcloud config set compute/zone USDzone
gcloud compute instances create USD{name} --machine-type USD{machine_type} --image USDimage --zone USDzone --subnet USDsubnet --boot-disk-size USD{disk_size} --boot-disk-device-name USD{name} --accelerator=USDaccelerator
ilab
ilab Usage: ilab [OPTIONS] COMMAND [ARGS] CLI for interacting with InstructLab. If this is your first time running ilab, it's best to start with `ilab config init` to create the environment. Options: --config PATH Path to a configuration file. [default: /home/auser/.config/instructlab/config.yaml] -v, --verbose Enable debug logging (repeat for even more verbosity) --version Show the version and exit. --help Show this message and exit. Commands: config Command Group for Interacting with the Config of InstructLab. data Command Group for Interacting with the Data generated by model Command Group for Interacting with the Models in InstructLab. system Command group for all system-related command calls taxonomy Command Group for Interacting with the Taxonomy of InstructLab. Aliases: chat model chat convert model convert diff taxonomy diff download model download evaluate model evaluate generate data generate init config init list model list serve model serve sysinfo system info test model test train model train