Chapter 2. Initializing InstructLab You must initialize the InstructLab environments to begin working with the Red Hat Enterprise Linux AI models. 2.1. Creating your RHEL AI environment You can start interacting with LLM models and the RHEL AI tooling by initializing the InstructLab environment. Prerequisites You installed RHEL AI with the bootable container image. You have root user access on your machine. Procedure Optional: To set up training profiles, you need to know the GPU accelerators in your machine. You can view your system information by running the following command: USD ilab system info Initialize InstructLab by running the following command: USD ilab config init The CLI prompts you to setup your config.yaml . Example output Follow the CLI prompts to set up your training hardware configurations. This updates your config.yaml file and adds the proper train configurations for training an LLM model. Type the number of the YAML file that matches your hardware specifications. Important These profiles only add the necessary configurations to the train section of your config.yaml file, therefore any profile can be selected for inference serving a model. Example output of selecting training profiles Example output of a completed ilab config init run. You selected: A100_H100_x8.yaml Initialization completed successfully, you're ready to start using `ilab`. Enjoy! Configuring your system's GPU for inference serving: This step is only required if you are using Red Hat Enterprise Linux AI exclusively for inference serving. Edit your config.yaml file by running the following command: USD ilab config edit In the evaluate section of the configurations file, edit the gpus: parameter and add the number of accelerators on your machine. evaluate: base_branch: null base_model: ~/.cache/instructlab/models/granite-7b-starter branch: null gpus: <num-gpus> In the vllm section of the serve field in the configuration file, edit the gpus: and vllm_args: ["--tensor-parallel-size"] parameters and add the number of accelerators on your machine. serve: backend: vllm chat_template: auto host_port: 127.0.0.1:8000 llama_cpp: gpu_layers: -1 llm_family: '' max_ctx_size: 4096 model_path: ~/.cache/instructlab/models/granite-7b-redhat-lab vllm: llm_family: '' vllm_args: ["--tensor-parallel-size", "<num-gpus>"] gpus: <num-gpus> If you want to use the skeleton taxonomy tree, which includes two skills and one knowledge qna.yaml file, you can clone the skeleton repository and place it in the taxonomy directory by running the following command: rm -rf ~/.local/share/instructlab/taxonomy/ ; git clone https://github.com/RedHatOfficial/rhelai-sample-taxonomy.git ~/.local/share/instructlab/taxonomy/ Directory structure of the InstructLab environment 1 ~/.cache/instructlab/models/ : Contains all downloaded large language models, including the saved output of ones you generate with RHEL AI. 2 ~/.local/share/instructlab/datasets/ : Contains data output from the SDG phase, built on modifications to the taxonomy repository. 3 ~/.local/share/instructlab/taxonomy/ : Contains the skill and knowledge data. 4 ~/.local/share/instructlab/phased/<phase1-or-phase2>/checkpoints/ : Contains the output of the multi-phase training process Verification You can view the full config.yaml file by running the following command USD ilab config show You can also manually edit the config.yaml file by running the following command: USD ilab config edit

Commands:
ilab system info
ilab config init
Welcome to InstructLab CLI. This guide will help you to setup your environment. Please provide the following values to initiate the environment [press Enter for defaults]: Generating `/home/<example-user>/.config/instructlab/config.yaml` and `/home/<example-user>/.local/share/instructlab/internal/train_configuration/profiles`
Please choose a train profile to use: [0] No profile (CPU-only) [1] A100_H100_x2.yaml [2] A100_H100_x4.yaml [3] A100_H100_x8.yaml [4] L40_x4.yaml [5] L40_x8.yaml [6] L4_x8.yaml Enter the number of your choice [hit enter for the default CPU-only profile] [0]:
You selected: A100_H100_x8.yaml Initialization completed successfully, you're ready to start using `ilab`. Enjoy!
ilab config edit
evaluate: base_branch: null base_model: ~/.cache/instructlab/models/granite-7b-starter branch: null gpus: <num-gpus>
serve: backend: vllm chat_template: auto host_port: 127.0.0.1:8000 llama_cpp: gpu_layers: -1 llm_family: '' max_ctx_size: 4096 model_path: ~/.cache/instructlab/models/granite-7b-redhat-lab vllm: llm_family: '' vllm_args: ["--tensor-parallel-size", "<num-gpus>"] gpus: <num-gpus>
rm -rf ~/.local/share/instructlab/taxonomy/ ; git clone https://github.com/RedHatOfficial/rhelai-sample-taxonomy.git ~/.local/share/instructlab/taxonomy/
├─ ~/.cache/instructlab/models/ 1 ├─ ~/.local/share/instructlab/datasets 2 ├─ ~/.local/share/instructlab/taxonomy 3 ├─ ~/.local/share/instructlab/phased/<phase1-or-phase2>/checkpoints/ 4
ilab config show
ilab config edit