Repository: instructlab/instructlab

Issue #2108: [Epic - Phase 2 Profiles] Introduce Profiles in `ilab`
Created: 2024-08-20T15:53:45Z
What is the current experience of ilab you've found issue with?
Using Instructlab with different hardware capabilities
How do you think the described experience should be?
Users should be able to set different configurations upon initialization of instructlab that allow them to properly use their hardware. These profiles should contain things like the following:

always use vllm or llamacpp based on GPU availability and vRAM
default to certain models based off of hardware availability for different tasks
set certain things like gpus for generate, evaluate, etc based on hardware
download all relevant models (via a prompt) for the user that are in the profile so all commands work out of the box (unless the models are on the system.

What is the benefit of the change you are proposing?

In ilab 0.18.0 the idea of a train-profile was introduced. These yaml files allow users to set different options for training based on their GPU configuration. This idea should be more broadly expanded across ilab in the following ways to alleviate user confusion about how to best utilize the newly introduced more intensive pipelines.
Additional context

The system can look like this:
ilab config init
Found /home/cdoern/.config/instructlab/config.yaml, do you still want to continue? [y/N]: y
Found /home/cdoern/.local/share/instructlab/internal/train_configuration/profiles, do you also want to reset existing profiles? [y/N]: y
Welcome to InstructLab CLI. This guide will help you to setup your environment.
Please provide the following values to initiate the environment [press Enter for defaults]:
Path to taxonomy repo [/home/cdoern/.local/share/instructlab/taxonomy]: 
Path to your model [/home/cdoern/.cache/instructlab/models/merlinite-7b-lab-Q4_K_M.gguf]: 
Generating `/home/cdoern/.config/instructlab/config.yaml` and `/home/cdoern/.local/share/instructlab/internal/train_configuration/profiles`...
Please choose a system profile to use:
[0] Choose for me
[1] Single Consumer GPU (8GB-16GB vRAM)
[2] Multi Consumer GPU (24+GB vRAM)
[3] Single Server GPU (24+GB vRAM)
[4] Multi Server GPU (48+GB vRAM)
[5] MacOS M-Series GPU (16GB+ vRAM)
Enter the number of your choice [hit enter for the default System profile] [0]: 1
Please choose which train profile matches your system configuration:
[0] Choose for me
[1] Nvidia RTX 4070
[2] Nvidia RTX 4080
[3] Nvidia RTX 4090

The first choice would allow the user to get some sane defaults for commands like data generate, model evaluate etc. And then the secondary prompt allows us to set proper effective_batch_size and other entries in the _train config.
This is a pretty lofty EP so I intend to use this issue as a tracker for sub-issues:

 Ensure Nvidia e2e full config compatibility before working on other support
 #2457
 #2458
 #2459
 #2506

Duplicate issue #1523: Create a cleaner mapping between the CLI flags and the training configs
Created: 2024-06-28T21:51:45Z
Currently, we flatten the CLI config for training so it can be merged with the CLI flag options, only to then be unflattened again and sent into the training interface data structures.
To get around this, we need to instead change it so that all of the CLI flags are manually mapped into the existing config and then sent into the training data structures.
A notable issue of the flattening and unflattening is that the training library has two different save_samples keys in different places.
There is the deepspeed.save_samples and save_samples which translate into the checkpoint save frequency of the hugging face and deepspeed format checkpoints respectively.
The CLI currently erases one of these because of the collision when flattening.

Closing comment (2024-08-26T18:27:39Z): Related to #2108