Chapter 3. Installing RHEL AI on AWS There are multiple ways you can install and deploy Red Hat Enterprise Linux AI on AWS. You can purchase RHEL AI from the AWS marketplace . You can download the RHEL AI RAW file on the RHEL AI download page and convert it to an AWS image. For installing and deploying RHEL AI using the RAW file, you must first convert the RHEL AI image into an Amazon Machine Image (AMI). 3.1. Converting the RHEL AI image to an AWS AMI Before deploying RHEL AI on an AWS machine, you must set up a S3 bucket and convert the RHEL AI image to a AWS AMI. In the following process, you create the following resources: An S3 bucket with the RHEL AI image AWS EC2 snapshots An AWS AMI An AWS instance Prerequisites You have an Access Key ID configured in the AWS IAM account manager . Procedure Install the AWS command-line tool by following the AWS documentation You need to create a S3 bucket and set the permissions to allow image file conversion to AWS snapshots. Create the necessary environment variables by running the following commands: USD export BUCKET=<custom_bucket_name> USD export RAW_AMI=nvidia-bootc.ami USD export AMI_NAME="rhel-ai" USD export DEFAULT_VOLUME_SIZE=1000 Note On AWS, the DEFAULT_VOLUME_SIZE is measured GBs. You can create an S3 bucket by running the following command: USD aws s3 mb s3://USDBUCKET You must create a trust-policy.json file with the necessary configurations for generating a S3 role for your bucket: USD printf '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "vmie.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals":{ "sts:Externalid": "vmimport" } } } ] }' > trust-policy.json Create an S3 role for your bucket that you can name. In the following example command, vmiport is the name of the role. USD aws iam create-role --role-name vmimport --assume-role-policy-document file://trust-policy.json You must create a role-policy.json file with the necessary configurations for generating a policy for your bucket: USD printf '{ "Version":"2012-10-17", "Statement":[ { "Effect":"Allow", "Action":[ "s3:GetBucketLocation", "s3:GetObject", "s3:ListBucket" ], "Resource":[ "arn:aws:s3:::%s", "arn:aws:s3:::%s/*" ] }, { "Effect":"Allow", "Action":[ "ec2:ModifySnapshotAttribute", "ec2:CopySnapshot", "ec2:RegisterImage", "ec2:Describe*" ], "Resource":"*" } ] }' USDBUCKET USDBUCKET > role-policy.json Create a policy for your bucket by running the following command: USD aws iam put-role-policy --role-name vmimport --policy-name vmimport-USDBUCKET --policy-document file://role-policy.json Now that your S3 bucket is set up, you need to download the RAW image from Red Hat Enterprise Linux AI download page Copy the RAW image link and add it to the following command: USD curl -Lo disk.raw <link-to-raw-file> Upload the image to the S3 bucket with the following command: USD aws s3 cp disk.raw s3://USDBUCKET/USDRAW_AMI Convert the image to a snapshot and store it in the task_id variable name by running the following commands: USD printf '{ "Description": "my-image", "Format": "raw", "UserBucket": { "S3Bucket": "%s", "S3Key": "%s" } }' USDBUCKET USDRAW_AMI > containers.json USD task_id=USD(aws ec2 import-snapshot --disk-container file://containers.json | jq -r .ImportTaskId) You can check the progress of the disk image to snapshot conversion job with the following command: USD aws ec2 describe-import-snapshot-tasks --filters Name=task-state,Values=active Once the conversion job is complete, you can get the snapshot ID and store it in a variable called snapshot_id by running the following command: USD snapshot_id=USD(aws ec2 describe-import-snapshot-tasks | jq -r '.ImportSnapshotTasks[] | select(.ImportTaskId=="'USD{task_id}'") | .SnapshotTaskDetail.SnapshotId') Add a tag name to the snapshot, so it's easier to identify, by running the following command: USD aws ec2 create-tags --resources USDsnapshot_id --tags Key=Name,Value="USDAMI_NAME" Register an AMI from the snapshot with the following command: USD ami_id=USD(aws ec2 register-image \ --name "USDAMI_NAME" \ --description "USDAMI_NAME" \ --architecture x86_64 \ --root-device-name /dev/sda1 \ --block-device-mappings "DeviceName=/dev/sda1,Ebs={VolumeSize=USD{DEFAULT_VOLUME_SIZE},SnapshotId=USD{snapshot_id}}" \ --virtualization-type hvm \ --ena-support \ | jq -r .ImageId) You can add another tag name to identify the AMI by running the following command: USD aws ec2 create-tags --resources USDami_id --tags Key=Name,Value="USDAMI_NAME" 3.2. Deploying your instance on AWS using the CLI You can launch the AWS instance with your new RHEL AI AMI from the AWS web console or the CLI. You can use whichever method of deployment you want to launch your instance. The following procedure displays how you can use the CLI to launch your AWS instance with the custom AMI. If you choose to use the CLI as a deployment option, there are several configurations you have to create, as shown in "Prerequisites". Prerequisites You created your RHEL AI AMI. For more information, see "Converting the RHEL AI image to an AWS AMI". You have the AWS command-line tool installed and is properly configured with your aws_access_key_id and aws_secret_access_key. You configured your Virtual Private Cloud (VPC). You created a subnet for your instance. You created a SSH key-pair. You created a security group on AWS. Procedure For various parameters, you need to gather the ID of the variable. To access the image ID, run the following command: USD aws ec2 describe-images --owners self To access the security group ID, run the following command: USD aws ec2 describe-security-groups To access the subnet ID, run the following command: USD aws ec2 describe-subnets Populate environment variables for when you create the instance USD instance_name=rhel-ai-instance USD ami=<ami-id> USD instance_type=<instance-type-size> USD key_name=<key-pair-name> USD security_group=<sg-id> USD disk_size=<size-of-disk> Create your instance using the variables by running the following command: USD aws ec2 run-instances \ --image-id USDami \ --instance-type USDinstance_type \ --key-name USDkey_name \ --security-group-ids USDsecurity_group \ --subnet-id USDsubnet \ --block-device-mappings DeviceName=/dev/sda1,Ebs='{VolumeSize='USDdisk_size'}' \ --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value='USDinstance_name'}]' User account The default user account in the RHEL AI AMI is cloud-user . It has all permissions via sudo without password. Verification To verify that your Red Hat Enterprise Linux AI tools are installed correctly, you need to run the ilab command: USD ilab Example output USD ilab Usage: ilab [OPTIONS] COMMAND [ARGS]... CLI for interacting with InstructLab. If this is your first time running ilab, it's best to start with `ilab config init` to create the environment. Options: --config PATH Path to a configuration file. [default: /home/<user>/.config/instructlab/config.yaml] -v, --verbose Enable debug logging (repeat for even more verbosity) --version Show the version and exit. --help Show this message and exit. Commands: config Command Group for Interacting with the Config of InstructLab. data Command Group for Interacting with the Data generated by... model Command Group for Interacting with the Models in InstructLab. system Command group for all system-related command calls taxonomy Command Group for Interacting with the Taxonomy of InstructLab. Aliases: chat model chat generate data generate serve model serve train model train

Commands:
export BUCKET=<custom_bucket_name> export RAW_AMI=nvidia-bootc.ami export AMI_NAME="rhel-ai" export DEFAULT_VOLUME_SIZE=1000
aws s3 mb s3://USDBUCKET
printf '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "vmie.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals":{ "sts:Externalid": "vmimport" } } } ] }' > trust-policy.json
aws iam create-role --role-name vmimport --assume-role-policy-document file://trust-policy.json
printf '{ "Version":"2012-10-17", "Statement":[ { "Effect":"Allow", "Action":[ "s3:GetBucketLocation", "s3:GetObject", "s3:ListBucket" ], "Resource":[ "arn:aws:s3:::%s", "arn:aws:s3:::%s/*" ] }, { "Effect":"Allow", "Action":[ "ec2:ModifySnapshotAttribute", "ec2:CopySnapshot", "ec2:RegisterImage", "ec2:Describe*" ], "Resource":"*" } ] }' USDBUCKET USDBUCKET > role-policy.json
aws iam put-role-policy --role-name vmimport --policy-name vmimport-USDBUCKET --policy-document file://role-policy.json
curl -Lo disk.raw <link-to-raw-file>
aws s3 cp disk.raw s3://USDBUCKET/USDRAW_AMI
printf '{ "Description": "my-image", "Format": "raw", "UserBucket": { "S3Bucket": "%s", "S3Key": "%s" } }' USDBUCKET USDRAW_AMI > containers.json
task_id=USD(aws ec2 import-snapshot --disk-container file://containers.json | jq -r .ImportTaskId)
aws ec2 describe-import-snapshot-tasks --filters Name=task-state,Values=active
snapshot_id=USD(aws ec2 describe-import-snapshot-tasks | jq -r '.ImportSnapshotTasks[] | select(.ImportTaskId=="'USD{task_id}'") | .SnapshotTaskDetail.SnapshotId')
aws ec2 create-tags --resources USDsnapshot_id --tags Key=Name,Value="USDAMI_NAME"
ami_id=USD(aws ec2 register-image --name "USDAMI_NAME" --description "USDAMI_NAME" --architecture x86_64 --root-device-name /dev/sda1 --block-device-mappings "DeviceName=/dev/sda1,Ebs={VolumeSize=USD{DEFAULT_VOLUME_SIZE},SnapshotId=USD{snapshot_id}}" --virtualization-type hvm --ena-support | jq -r .ImageId)
aws ec2 create-tags --resources USDami_id --tags Key=Name,Value="USDAMI_NAME"
aws ec2 describe-images --owners self
aws ec2 describe-security-groups
aws ec2 describe-subnets
instance_name=rhel-ai-instance ami=<ami-id> instance_type=<instance-type-size> key_name=<key-pair-name> security_group=<sg-id> disk_size=<size-of-disk>
aws ec2 run-instances --image-id USDami --instance-type USDinstance_type --key-name USDkey_name --security-group-ids USDsecurity_group --subnet-id USDsubnet --block-device-mappings DeviceName=/dev/sda1,Ebs='{VolumeSize='USDdisk_size'}' --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value='USDinstance_name'}]'
ilab
ilab Usage: ilab [OPTIONS] COMMAND [ARGS] CLI for interacting with InstructLab. If this is your first time running ilab, it's best to start with `ilab config init` to create the environment. Options: --config PATH Path to a configuration file. [default: /home/<user>/.config/instructlab/config.yaml] -v, --verbose Enable debug logging (repeat for even more verbosity) --version Show the version and exit. --help Show this message and exit. Commands: config Command Group for Interacting with the Config of InstructLab. data Command Group for Interacting with the Data generated by model Command Group for Interacting with the Models in InstructLab. system Command group for all system-related command calls taxonomy Command Group for Interacting with the Taxonomy of InstructLab. Aliases: chat model chat generate data generate serve model serve train model train