Chapter 4. Installing RHEL AI on IBM cloud For installing and deploying Red Hat Enterprise Linux AI on IBM Cloud, you must first convert the RHEL AI image into an IBM Cloud image. You can then launch an instance using the IBM Cloud image and deploy RHEL AI on an IBM Cloud machine. Important On Red Hat Enterprise Linux AI version 1.1 currently only supports inference serving on IBM Cloud 4.1. Converting the RHEL AI image into a IBM Cloud image. To create a bootable image in IBM Cloud you must configure your IBM Cloud accounts, set up a Cloud Object Storage (COS) bucket, and create a IBM Cloud image using the RHEL AI image. Prerequisites You installed the IBM CLI on your specific machine, see Installing the stand-alone IBM Cloud CLI . Procedure Log in to IBM Cloud with the following command: USD ibmcloud login When prompted, select your desired account to log in to. Example output of the login USD ibmcloud login API endpoint: https://cloud.ibm.com Region: us-east Get a one-time code from https://identity-1.eu-central.iam.cloud.ibm.com/identity/passcode to proceed. Open the URL in the default browser? [Y/n] > One-time code > Authenticating... OK Select an account: 1. <account-name> 2. <account-name-2> API endpoint: https://cloud.ibm.com Region: us-east User: <user-name> Account: <selected-account> Resource group: No resource group targeted, use 'ibmcloud target -g RESOURCE_GROUP' You need to set up various IBM Cloud configurations and create your COS bucket before generating a QCOW2 image. You can install the necessary IBM Cloud plugins by running the following command: USD ibmcloud plugin install cloud-object-storage infrastructure-service Set your preferred resource group, the following example command sets the resource group named Default . USD ibmcloud target -g Default Set your preferred region, the following example command sets the us-east region. USD ibmcloud target -r us-east You need to select a deployment plan for your service instance. Ensure you check the properties and pricing on the IBM cloud website. You can list the available deployment plans by running the following command: USD ibmcloud catalog service cloud-object-storage --output json | jq -r '.[].children[] | select(.children != null) | .children[].name' The following example command uses the premium-global-deployment plan and puts it in the environment variable cos_deploy_plan : USD cos_deploy_plan=premium-global-deployment Create a Cloud Object Storage (COS) service instance and save the name in an environment variable named cos_si_name and create the cloud-object-storage and by running the following commands: USD cos_si_name=THE_NAME_OF_YOUR_SERVICE_INSTANCE USD ibmcloud resource service-instance-create USD{cos_si_name} cloud-object-storage standard global -d USD{cos_deploy_plan} Get the Cloud Resource Name (CRN) for your Cloud Object Storage (COS) bucket in a variable named cos_crn by running the following commands: USD cos_crn=USD(ibmcloud resource service-instance USD{cos_si_name} --output json| jq -r '.[] | select(.crn | contains("cloud-object-storage")) | .crn') USD ibmcloud cos config crn --crn USD{cos_crn} --force Create your Cloud Object Storage (COS) bucket named as the environment variable bucket_name with the following commands: USD bucket_name=NAME_OF_MY_BUCKET USD ibmcloud cos bucket-create --bucket USD{bucket_name} Allow the infrastructure service to read the buckets that are in the service instance USD{cos_si_guid} variable by running the following commands: USD cos_si_guid=USD(ibmcloud resource service-instance USD{cos_si_name} --output json| jq -r '.[] | select(.crn | contains("cloud-object-storage")) | .guid') USD ibmcloud iam authorization-policy-create is cloud-object-storage Reader --source-resource-type image --target-service-instance-id USD{cos_si_guid} Now that your S3 bucket is set up, you need to download the RAW image from Red Hat Enterprise Linux AI download page Copy the RAW image link and add it to the following command: USD curl -Lo disk.qcow2 "PASTE_HERE_THE_LINK_OF_THE_QCOW2_FILE" Set the name you want to use as the RHEL AI IBM Cloud image USD image_name=rhel-ai-20240703v0 Upload the QCOW2 image to the Cloud Object Storage (COS) bucket by running the following command: USD ibmcloud cos upload --bucket USD{bucket_name} --key USD{image_name}.qcow2 --file disk.qcow2 --region <region> Convert the QCOW2 you just uploaded to an IBM Cloud image with the following commands: USD ibmcloud is image-create USD{image_name} --file cos://<region>/USD{bucket_name}/USD{image_name}.qcow2 --os-name red-ai-9-amd64-nvidia-byol Once the job launches, set the IBM Cloud image configurations into a variable called image_id by running the following command: USD image_id=USD(ibmcloud is images --visibility private --output json | jq -r '.[] | select(.name=="'USDimage_name'") | .id') You can view the progress of the job with the following command: USD while ibmcloud is image --output json USD{image_id} | jq -r .status | grep -xq pending; do sleep 1; done You can view the information of the newly created image with the following command: USD ibmcloud is image USD{image_id} 4.2. Deploying your instance on IBM Cloud using the CLI You can launch an instance with your new RHEL AI IBM Cloud image from the IBM Cloud web console or the CLI. You can use whichever method of deployment you want to launch your instance. The following procedure displays how you can use the CLI to launch an IBM Cloud instance with the custom IBM Cloud image If you choose to use the CLI as a deployment option, there are several configurations you have to create, as shown in "Prerequisites". Prerequisites You created your RHEL AI IBM Cloud image. For more information, see "Converting the RHEL AI image to an IBM Cloud image". You installed the IBM CLI on your specific machine, see Installing the stand-alone IBM Cloud CLI . You configured your Virtual private cloud (VPC). You created a subnet for your instance. Procedure Log in to your IBM Cloud account and select the Account, Region and Resource Group by running the following command: USD ibmcloud login -c <ACCOUNT_ID> -r <REGION> -g <RESOURCE_GROUP> Before launching your IBM Cloud instance on the CLI, you need to create several configuration variables for your instance. Install the infrastructure-service plugin for IBM Cloud by running the following command USD ibmcloud plugin install infrastructure-service You need to create an SSH public key for your IBM Cloud account. IBM Cloud supports RSA and ed25519 keys. The following example command uses the ed25519 key types and names it ibmcloud . USD ssh-keygen -f ibmcloud -t ed25519 You can now upload the public key to your IBM Cloud account by following the example command. USD ibmcloud is key-create my-ssh-key @ibmcloud.pub --key-type ed25519 You need to create a Floating IP for your IBM Cloud instance by following the example command. Ensure you change the region to your preferred zone. USD ibmcloud is floating-ip-reserve my-public-ip --zone <region> You need to select the instance profile that you want to use for the deployment. List all the profiles by running the following command: USD ibmcloud is instance-profiles Make a note of your preferred instance profile, you will need it for your instance deployment. You can now start creating your IBM Cloud instance. Populate environment variables for when you create the instance. name=my-rhelai-instance vpc=my-vpc-in-us-east zone=us-east-1 subnet=my-subnet-in-us-east-1 instance_profile=gx3-64x320x4l4 image=my-custom-rhelai-image sshkey=my-ssh-key floating_ip=my-public-ip disk_size=250 You can now launch your instance, by running the following command: USD ibmcloud is instance-create \ USDname \ USDvpc \ USDzone \ USDinstance_profile \ USDsubnet \ --image USDimage \ --keys USDsshkey \ --boot-volume '{"name": "'USD{name}'-boot", "volume": {"name": "'USD{name}'-boot", "capacity": 'USD{disk_size}', "profile": {"name": "general-purpose"}}}' \ --allow-ip-spoofing false Link the Floating IP to the instance by running the following command: USD ibmcloud is floating-ip-update USDfloating_ip --nic primary --in USDname User account The default user account in the RHEL AI AMI is cloud-user . It has all permissions via sudo without password. Verification To verify that your Red Hat Enterprise Linux AI tools are installed correctly, run the ilab command: USD ilab Example output USD ilab Usage: ilab [OPTIONS] COMMAND [ARGS]... CLI for interacting with InstructLab. If this is your first time running ilab, it's best to start with `ilab config init` to create the environment. Options: --config PATH Path to a configuration file. [default: /home/auser/.config/instructlab/config.yaml] -v, --verbose Enable debug logging (repeat for even more verbosity) --version Show the version and exit. --help Show this message and exit. Commands: config Command Group for Interacting with the Config of InstructLab. data Command Group for Interacting with the Data generated by... model Command Group for Interacting with the Models in InstructLab. system Command group for all system-related command calls taxonomy Command Group for Interacting with the Taxonomy of InstructLab. Aliases: chat model chat convert model convert diff taxonomy diff download model download evaluate model evaluate generate data generate init config init list model model_list serve model serve sysinfo system info test model test train model train 4.3. Adding more storage to your IBM Cloud instance In [ibm-c], there is a size restriction of 250 GB of storage in the main IBM Cloud disk. RHEL AI might require more storage for models and generation data. You can add more storage by attaching an extra disk to your instance and using it to hold data for RHEL AI. Prerequisites You have a IBM Cloud RHEL AI instance. Procedure Create an environment variable called name that has the name of your instance by running the following command: USD name=my-rhelai-instance Set the size of the new volume by running the following command: USD data_volume_size=1000 Create and attach the instance volume by running the following command: USD ibmcloud is instance-volume-attachment-add data USD{name} \ --new-volume-name USD{name}-data \ --profile general-purpose \ --capacity USD{data_volume_size} You can list all the disks with the following command: USD lsblk Create a disk variable with the content of the disk path your using. The following example command uses the /dev/vdb path. USD disk=/dev/vdb Create a partition on your disk by running the following command: USD sgdisk -n 1:0:0 USDdisk Format and label the partition by running the following command: USD mkfs.xfs -L ilab-data USD{disk}1 You can configure your system to auto mount to your preferred directory. The following example command uses the /mnt directory. USD echo LABEL=ilab-data /mnt xfs defaults 0 0 >> /etc/fstab Reload the systemd service to acknowledge the new configuration on mounts by running the following command: USD systemctl daemon-reload Mount the disk with the following command: USD mount -a Grant write permissions to all users in the new file system by running the following command: USD chmod 1777 /mnt/ 4.4. Adding a data storage directory to your instance By default RHEL AI holds configuration data in the USDHOME directory. You can change this default to a different directory for holding InstructLab data. Prerequisites You have a Red Hat Enterprise Linux AI instance You added an extra storage disk to your instance Procedure You can configure the ILAB_HOME environment variable by writing it to the USDHOME/.bash_profile file by running the following commands: USD echo 'export ILAB_HOME=/mnt' >> USDHOME/.bash_profile You can make that change effective by reloading the USDHOME/.bash_profile file with the following command: USD source USDHOME/.bash_profile

Commands:
ibmcloud login
ibmcloud login API endpoint: https://cloud.ibm.com Region: us-east Get a one-time code from https://identity-1.eu-central.iam.cloud.ibm.com/identity/passcode to proceed. Open the URL in the default browser? [Y/n] > One-time code > Authenticating OK Select an account: 1. <account-name> 2. <account-name-2> API endpoint: https://cloud.ibm.com Region: us-east User: <user-name> Account: <selected-account> Resource group: No resource group targeted, use 'ibmcloud target -g RESOURCE_GROUP'
ibmcloud plugin install cloud-object-storage infrastructure-service
ibmcloud target -g Default
ibmcloud target -r us-east
ibmcloud catalog service cloud-object-storage --output json | jq -r '.[].children[] | select(.children != null) | .children[].name'
cos_deploy_plan=premium-global-deployment
cos_si_name=THE_NAME_OF_YOUR_SERVICE_INSTANCE
ibmcloud resource service-instance-create USD{cos_si_name} cloud-object-storage standard global -d USD{cos_deploy_plan}
cos_crn=USD(ibmcloud resource service-instance USD{cos_si_name} --output json| jq -r '.[] | select(.crn | contains("cloud-object-storage")) | .crn')
ibmcloud cos config crn --crn USD{cos_crn} --force
bucket_name=NAME_OF_MY_BUCKET
ibmcloud cos bucket-create --bucket USD{bucket_name}
cos_si_guid=USD(ibmcloud resource service-instance USD{cos_si_name} --output json| jq -r '.[] | select(.crn | contains("cloud-object-storage")) | .guid')
ibmcloud iam authorization-policy-create is cloud-object-storage Reader --source-resource-type image --target-service-instance-id USD{cos_si_guid}
curl -Lo disk.qcow2 "PASTE_HERE_THE_LINK_OF_THE_QCOW2_FILE"
image_name=rhel-ai-20240703v0
ibmcloud cos upload --bucket USD{bucket_name} --key USD{image_name}.qcow2 --file disk.qcow2 --region <region>
ibmcloud is image-create USD{image_name} --file cos://<region>/USD{bucket_name}/USD{image_name}.qcow2 --os-name red-ai-9-amd64-nvidia-byol
image_id=USD(ibmcloud is images --visibility private --output json | jq -r '.[] | select(.name=="'USDimage_name'") | .id')
while ibmcloud is image --output json USD{image_id} | jq -r .status | grep -xq pending; do sleep 1; done
ibmcloud is image USD{image_id}
ibmcloud login -c <ACCOUNT_ID> -r <REGION> -g <RESOURCE_GROUP>
ibmcloud plugin install infrastructure-service
ssh-keygen -f ibmcloud -t ed25519
ibmcloud is key-create my-ssh-key @ibmcloud.pub --key-type ed25519
ibmcloud is floating-ip-reserve my-public-ip --zone <region>
ibmcloud is instance-profiles
name=my-rhelai-instance vpc=my-vpc-in-us-east zone=us-east-1 subnet=my-subnet-in-us-east-1 instance_profile=gx3-64x320x4l4 image=my-custom-rhelai-image sshkey=my-ssh-key floating_ip=my-public-ip disk_size=250
ibmcloud is instance-create USDname USDvpc USDzone USDinstance_profile USDsubnet --image USDimage --keys USDsshkey --boot-volume '{"name": "'USD{name}'-boot", "volume": {"name": "'USD{name}'-boot", "capacity": 'USD{disk_size}', "profile": {"name": "general-purpose"}}}' --allow-ip-spoofing false
ibmcloud is floating-ip-update USDfloating_ip --nic primary --in USDname
ilab
ilab Usage: ilab [OPTIONS] COMMAND [ARGS] CLI for interacting with InstructLab. If this is your first time running ilab, it's best to start with `ilab config init` to create the environment. Options: --config PATH Path to a configuration file. [default: /home/auser/.config/instructlab/config.yaml] -v, --verbose Enable debug logging (repeat for even more verbosity) --version Show the version and exit. --help Show this message and exit. Commands: config Command Group for Interacting with the Config of InstructLab. data Command Group for Interacting with the Data generated by model Command Group for Interacting with the Models in InstructLab. system Command group for all system-related command calls taxonomy Command Group for Interacting with the Taxonomy of InstructLab. Aliases: chat model chat convert model convert diff taxonomy diff download model download evaluate model evaluate generate data generate init config init list model model_list serve model serve sysinfo system info test model test train model train
name=my-rhelai-instance
data_volume_size=1000
ibmcloud is instance-volume-attachment-add data USD{name} --new-volume-name USD{name}-data --profile general-purpose --capacity USD{data_volume_size}
lsblk
disk=/dev/vdb
sgdisk -n 1:0:0 USDdisk
mkfs.xfs -L ilab-data USD{disk}1
echo LABEL=ilab-data /mnt xfs defaults 0 0 >> /etc/fstab
systemctl daemon-reload
mount -a
chmod 1777 /mnt/
echo 'export ILAB_HOME=/mnt' >> USDHOME/.bash_profile
source USDHOME/.bash_profile