Repository: instructlab/instructlab

Issue:
[Phase 1 Profiles] HW auto-detect for CLI profiles
It would be nice to implement a way to auto-detect hardware, and suggest an appropriate default config profile for each of the components based on the hardware we detect. This will make for a smoother user experience that requires less work on the user's part to maximize the usage of their hardware and optimize performance (and get better output results.)

 
Feature Overview (aka. Goal Summary)
As a user, I want RHEL AI to detect the hardware I am working with so I do not have to choose/edit config files. 

Goals (aka. expected user outcomes)
This will make getting started with the process easier. Users will not have to look for relevant config files and reduces scope for errors.

Questions to Answer:
1. Will this be applicable for all hardware configurations (eg. RHEL AI suitable ones - 4xH100s and so on, as well as others, eg. 4xA10s)?
2. Will all the different components (eval, training, generate) be updated? 


Issues in this Epic:
- [x] #2159 


Discussion:
Subset of feature/enhancement #2108 
Came here to open this issue, glad to see it's already reported.

Running `config init` with ilab 1.18, it's frustrating to me to see the different profiles offered for multiple GPUs while sitting only on a single T4 GPU. Since the configuration is pretty complex, it's a big task right from the beginning to update the configuration file for someone new to InstructLab.

It would be much more helpful if ilab already detected my GPUs (especially for a single card) and adapted the config automatically. This especially applies to `serve` and `generate`, I understand that training is more complex.



Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 797ef64049..dc783f6a4b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -17,6 +17,10 @@
 * Intel Gaudi software has been updated to 1.17.1 with Python 3.11 and
   Torch 2.3.1 support.
 
+### Features
+
+* `ilab config init` now auto detects your hardware when running on Nvidia enabled systems and chooses the best train profile. It does this by checking first if your system directly matches one of our supported train profiles and then attempts to match the vRAM for each profile to the total vRAM on your system.
+
 ## v0.18.1
 
 ### Features
diff --git a/src/instructlab/config/init.py b/src/instructlab/config/init.py
index f22ba4fbee..863c1ce0f7 100644
--- a/src/instructlab/config/init.py
+++ b/src/instructlab/config/init.py
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: Apache-2.0
 
 # Standard
+from math import floor
 from os import listdir
 from os.path import dirname, exists, join
 import pathlib
@@ -14,7 +15,9 @@
 from instructlab import clickext, utils
 from instructlab.configuration import (
     DEFAULTS,
+    PROFILE_MAPPINGS,
     Config,
+    _train,
     ensure_storage_directories_exist,
     get_default_config,
     read_config,
@@ -22,6 +25,7 @@
     recreate_train_profiles,
     write_config,
 )
+from instructlab.utils import convert_bytes_to_proper_mag
 
 
 @click.command()
@@ -117,53 +121,74 @@ def init(
     if train_profile is not None:
         cfg.train = read_train_profile(train_profile)
     elif interactive:
-        train_profile_filenames = sorted(listdir(DEFAULTS.TRAIN_PROFILE_DIR))
-        # generate human-friendly text out of train profile filenames
-        profile_options = []
-        for filename in train_profile_filenames:
-            # one gpu type
-            split_filename = filename.split("_")
-            if len(split_filename) == 2:
-                gputype1 = split_filename[0]
-                num_gpu = split_filename[1].split(".")[0]
-                profile_options.append(f"Nvidia {gputype1} {num_gpu} ({filename})")
-            # two gpu types
-            elif len(split_filename) == 3:
-                gputype1 = split_filename[0]
-                gputype2 = split_filename[1]
-                num_gpu = split_filename[2].split(".")[0]
-                profile_options.append(
-                    f"Nvidia {gputype1}/{gputype2} {num_gpu} ({filename})"
-                )
-        click.echo(
-            "Please choose a train profile to use.\nTrain profiles assist with the complexity of configuring specific GPU hardware with the InstructLab Training library.\nYou can still take advantage of hardware acceleration for training even if your hardware is not listed."
-        )
-        click.echo("[0] No profile (CPU, Apple Metal, AMD ROCm)")
-        for i, value in enumerate(profile_options):
-            click.echo(f"[{i+1}] {value}")
-        train_profile_selection = click.prompt(
-            "Enter the number of your choice [hit enter for no profile]",
-            type=int,
-            default=0,
-        )
-        if 1 <= train_profile_selection <= len(profile_options):
-            click.echo(f"You selected: {profile_options[train_profile_selection - 1]}")
-            cfg.train = read_train_profile(
-                join(
-                    DEFAULTS.TRAIN_PROFILE_DIR,
-                    train_profile_filenames[train_profile_selection - 1],
-                )
+        (
+            chosen_profile,
+            chosen_vram,
+            detected_train_profile,
+            edited_cfg,
+        ) = hw_auto_detect()
+        yn = None
+        if chosen_vram is not None and chosen_profile is not None:
+            click.echo(
+                f"We chose {chosen_profile} as your designated training profile. This is for systems with {chosen_vram} GB of vRAM."
             )
-        elif train_profile_selection == 0:
+            if edited_cfg:
+                click.echo(
+                    "This profile is the best approximation for your system based off of the amount of vRAM. We modified it to match the number of GPUs you have."
+                )
+            yn = click.confirm("Is this profile correct?", default=True)
+        if yn is None or not yn:
+            train_profile_filenames = sorted(listdir(DEFAULTS.TRAIN_PROFILE_DIR))
+            # generate human-friendly text out of train profile filenames
+            profile_options = []
+            for filename in train_profile_filenames:
+                # one gpu type
+                split_filename = filename.split("_")
+                if len(split_filename) == 2:
+                    gputype1 = split_filename[0]
+                    num_gpu = split_filename[1].split(".")[0]
+                    profile_options.append(f"Nvidia {gputype1} {num_gpu} ({filename})")
+                # two gpu types
+                elif len(split_filename) == 3:
+                    gputype1 = split_filename[0]
+                    gputype2 = split_filename[1]
+                    num_gpu = split_filename[2].split(".")[0]
+                    profile_options.append(
+                        f"Nvidia {gputype1}/{gputype2} {num_gpu} ({filename})"
+                    )
             click.echo(
-                "No profile selected - any hardware acceleration for training must be configured manually."
+                "Please choose a train profile to use.\nTrain profiles assist with the complexity of configuring InstructLab training for specific GPU hardware.\nYou can still take advantage of hardware acceleration for training even if your hardware is not listed."
             )
-        else:
-            click.secho(
-                "Invalid selection. Please select a valid train profile option.",
-                fg="red",
+            click.echo("[0] No profile (CPU, Apple Metal, AMD ROCm)")
+            for i, value in enumerate(profile_options):
+                click.echo(f"[{i+1}] {value}")
+            train_profile_selection = click.prompt(
+                "Enter the number of your choice [hit enter for hardware defaults]",
+                type=int,
+                default=0,
             )
-            raise click.exceptions.Exit(1)
+            if 1 <= train_profile_selection <= len(profile_options):
+                click.echo(
+                    f"You selected: {profile_options[train_profile_selection - 1]}"
+                )
+                cfg.train = read_train_profile(
+                    join(
+                        DEFAULTS.TRAIN_PROFILE_DIR,
+                        train_profile_filenames[train_profile_selection - 1],
+                    )
+                )
+            elif train_profile_selection == 0:
+                click.echo(
+                    "No profile selected - any hardware acceleration for training must be configured manually."
+                )
+            else:
+                click.secho(
+                    "Invalid selection. Please select a valid train profile option.",
+                    fg="red",
+                )
+                raise click.exceptions.Exit(1)
+        else:
+            cfg.train = detected_train_profile
 
     # we should not override all paths with the serve model if special ENV vars exist
     if param_source != click.core.ParameterSource.ENVIRONMENT:
@@ -183,6 +208,114 @@ def init(
     )
 
 
+def hw_auto_detect() -> tuple[str | None, int | None, _train, bool]:
+    # Third Party
+    import torch
+
+    gpus = 0
+    total_vram = 0
+    train = _train()
+    chosen_vram = None
+    chosen_profile_name = None
+    gpu_name = ""
+    edited_cfg = False
+    # try nvidia
+    if torch.cuda.is_available():
+        click.echo("Detecting Hardware...")
+        gpus = torch.cuda.device_count()
+        for i in range(gpus):
+            properties = torch.cuda.get_device_properties(i)
+            gpu_name = properties.name
+            total_vram += properties.total_memory  # memory in B
+    vram = int(floor(convert_bytes_to_proper_mag(total_vram)[0]))
+    if vram == 0:
+        return None, None, train, False
+    matched, chosen_vram, chosen_profile_name, train = lookup_card(
+        gpu_name=gpu_name, gpu_count=gpus, vram=vram
+    )
+    # if we used the vRAM to determine, we need to tell the user that
+    if not matched:
+        # edit nproc per node in the case that we didnt match an exact profile, so we use all GPUs on our system
+        train.nproc_per_node = gpus
+        edited_cfg = True
+    return (chosen_profile_name, chosen_vram, train, edited_cfg)
+
+
+# lookup_card checks if the user's GPU exists in our supported profile map. If not, we choose one based off of their vRAM
+def lookup_card(
+    gpu_name, gpu_count, vram
+) -> tuple[bool, int | None, str | None, _train]:
+    profiles = PROFILE_MAPPINGS
+    card_entry = profiles.get(gpu_name)
+    chosen_profile_name = None
+    if card_entry is not None:
+        chosen_profile_name = f"Nvidia {gpu_count}x {gpu_name}"
+        # it is not guaranteed we match, even if we find an entry
+        # what if the vram isn't found or what if the gpu count doesn't match
+        matched, train = match_profile_based_on_gpu_count(
+            card_entry=card_entry, gpu_count=gpu_count, vram=vram
+        )
+        if matched:
+            return True, vram, chosen_profile_name, train
+    vram, chosen_profile_name, train = match_profile_based_on_vram(vram=vram)
+    # return False with the results of the vRAM matching to indicate we didn't find an exact match for this GPU configuration in our supported profiles
+    return False, vram, chosen_profile_name, train
+
+
+# match_profile_based_on_vram chooses a training profile based off the amount of vram in each profile
+# this profile will be modified to match the number of GPUs on the user's system
+def match_profile_based_on_vram(vram) -> tuple[int | None, str | None, _train]:
+    chosen_profile_name = None
+    chosen_vram = None
+    train = _train()
+    for group_gpu_name, list_of_count_vram_and_config in PROFILE_MAPPINGS.items():
+        for gpu_count_configs in list_of_count_vram_and_config:
+            if (
+                not isinstance(gpu_count_configs["gpu_count"], int)
+                or not isinstance(gpu_count_configs["vram_and_config"], dict)
+                or not isinstance(gpu_count_configs["vram_and_config"]["vram"], int)
+                or not isinstance(
+                    gpu_count_configs["vram_and_config"]["config"], _train
+                )
+            ):
+                click.secho("Unable to retrieve train profiles", fg="red")
+                raise click.exceptions.Exit(1)
+            if (
+                chosen_vram is None
+                or chosen_vram > gpu_count_configs["vram_and_config"]["vram"]
+            ) and vram < gpu_count_configs["vram_and_config"]["vram"]:
+                # set our return values
+                gpus = gpu_count_configs["gpu_count"]
+                chosen_profile_name = f"Nvidia {gpus}x {group_gpu_name}"
+                chosen_vram = gpu_count_configs["vram_and_config"]["vram"]
+                train = gpu_count_configs["vram_and_config"]["config"]
+    return chosen_vram, chosen_profile_name, train
+
+
+# match_profile_based_on_gpu_count looks for a training profile which directly matches the gpu name and count
+# this training profile will be used as is
+def match_profile_based_on_gpu_count(
+    card_entry, gpu_count, vram
+) -> tuple[bool, _train]:
+    train = _train()
+    matched = False
+    for gpu_count_configs in card_entry:
+        if not isinstance(gpu_count_configs["gpu_count"], int) or not isinstance(
+            gpu_count_configs["vram_and_config"], dict
+        ):
+            click.secho("Unable to retrieve train profiles", fg="red")
+            raise click.exceptions.Exit(1)
+        # if GPU name and number matches, this is a direct match
+        if (
+            gpu_count == gpu_count_configs["gpu_count"]
+            and vram == gpu_count_configs["vram_and_config"]["vram"]
+        ):
+            train = gpu_count_configs["vram_and_config"]["config"]
+            matched = True
+            break
+    return matched, train
+
+
 def check_if_configs_exist(fresh_install) -> bool:
     if exists(DEFAULTS.CONFIG_FILE):
         overwrite = click.confirm(
diff --git a/src/instructlab/configuration.py b/src/instructlab/configuration.py
index 75521d50ee..9e98b200a2 100644
--- a/src/instructlab/configuration.py
+++ b/src/instructlab/configuration.py
@@ -503,6 +503,303 @@ class Config(BaseModel):
     )
 
 
+EIGHT_GPU_TRAIN_AH = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=8,
+    effective_batch_size=128,
+    lora_quantize_dtype=None,
+    lora_rank=0,
+    max_batch_len=60000,
+    max_seq_len=4096,
+    save_samples=1000,
+    deepspeed_cpu_offload_optimizer=False,
+    is_padding_free=True,
+    num_epochs=8,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+FOUR_GPU_TRAIN_AH = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=4,
+    effective_batch_size=128,
+    lora_quantize_dtype=None,
+    lora_rank=0,
+    max_batch_len=54000,
+    max_seq_len=4096,
+    save_samples=1000,
+    deepspeed_cpu_offload_optimizer=False,
+    is_padding_free=True,
+    num_epochs=8,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+TWO_GPU_TRAIN_AH = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+        "deepspeed_cpu_offload_optimizer_ratio": 1,
+        "deepspeed_cpu_offload_optimizer_pin_memory": True,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=2,
+    effective_batch_size=128,
+    lora_quantize_dtype=None,
+    lora_rank=0,
+    max_batch_len=60000,
+    max_seq_len=4096,
+    save_samples=1000,
+    deepspeed_cpu_offload_optimizer=True,
+    is_padding_free=True,
+    num_epochs=8,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+EIGHT_L_FORTY_GPU = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=8,
+    effective_batch_size=128,
+    lora_quantize_dtype=None,
+    lora_rank=0,
+    max_batch_len=30000,
+    max_seq_len=4096,
+    save_samples=1000,
+    deepspeed_cpu_offload_optimizer=False,
+    is_padding_free=True,
+    num_epochs=8,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+FOUR_L_FORTY_GPU = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+        "deepspeed_cpu_offload_optimizer_ratio": 1,
+        "deepspeed_cpu_offload_optimizer_pin_memory": True,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=4,
+    effective_batch_size=128,
+    lora_quantize_dtype=None,
+    lora_rank=0,
+    max_batch_len=30000,
+    max_seq_len=4096,
+    save_samples=1000,
+    num_epochs=8,
+    deepspeed_cpu_offload_optimizer=True,
+    is_padding_free=True,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+EIGHT_L_FOUR_GPU = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+        "deepspeed_cpu_offload_optimizer_ratio": 1,
+        "deepspeed_cpu_offload_optimizer_pin_memory": True,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=8,
+    effective_batch_size=128,
+    lora_quantize_dtype=None,
+    lora_rank=0,
+    max_batch_len=30000,
+    max_seq_len=4096,
+    save_samples=1000,
+    num_epochs=8,
+    deepspeed_cpu_offload_optimizer=True,
+    is_padding_free=True,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+SINGLE_SERVER_GPU_TRAIN = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+        "deepspeed_cpu_offload_optimizer_ratio": 1,
+        "deepspeed_cpu_offload_optimizer_pin_memory": True,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=1,
+    effective_batch_size=96,
+    lora_quantize_dtype="nf4",
+    lora_rank=2,
+    max_batch_len=60000,
+    max_seq_len=4096,
+    save_samples=1000,
+    deepspeed_cpu_offload_optimizer=True,
+    is_padding_free=True,
+    num_epochs=8,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+SINGLE_CONSUMER_GPU_TRAIN = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+        "deepspeed_cpu_offload_optimizer_ratio": 1,
+        "deepspeed_cpu_offload_optimizer_pin_memory": True,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=1,
+    effective_batch_size=96,
+    lora_quantize_dtype="nf4",
+    lora_rank=2,
+    max_batch_len=30000,
+    max_seq_len=4096,
+    save_samples=1000,
+    deepspeed_cpu_offload_optimizer=True,
+    is_padding_free=True,
+    num_epochs=5,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+
+MULTI_CONSUMER_GPU_TRAIN = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+        "deepspeed_cpu_offload_optimizer_ratio": 1,
+        "deepspeed_cpu_offload_optimizer_pin_memory": True,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=2,
+    effective_batch_size=96,
+    lora_quantize_dtype="nf4",
+    lora_rank=2,
+    max_batch_len=30000,
+    max_seq_len=4096,
+    save_samples=1000,
+    deepspeed_cpu_offload_optimizer=True,
+    is_padding_free=True,
+    num_epochs=5,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+
+SINGLE_L4 = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+        "deepspeed_cpu_offload_optimizer_ratio": 1,
+        "deepspeed_cpu_offload_optimizer_pin_memory": True,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=1,
+    effective_batch_size=96,
+    lora_quantize_dtype="nf4",
+    lora_rank=2,
+    max_batch_len=30000,
+    max_seq_len=4096,
+    save_samples=1000,
+    deepspeed_cpu_offload_optimizer=True,
+    is_padding_free=True,
+    num_epochs=5,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+SINGLE_L40 = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+        "deepspeed_cpu_offload_optimizer_ratio": 1,
+        "deepspeed_cpu_offload_optimizer_pin_memory": True,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=1,
+    effective_batch_size=96,
+    lora_quantize_dtype="nf4",
+    lora_rank=2,
+    max_batch_len=30000,
+    max_seq_len=4096,
+    save_samples=1000,
+    deepspeed_cpu_offload_optimizer=True,
+    is_padding_free=True,
+    num_epochs=5,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+SINGLE_A100_H100 = _train(
+    additional_args={
+        "warmup_steps": 25,
+        "learning_rate": 2e-5,
+        "lora_dropout": 0.1,
+        "lora_alpha": 32,
+        "deepspeed_cpu_offload_optimizer_ratio": 1,
+        "deepspeed_cpu_offload_optimizer_pin_memory": True,
+    },
+    ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
+    data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
+    data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
+    nproc_per_node=1,
+    effective_batch_size=128,
+    lora_quantize_dtype=None,
+    lora_rank=0,
+    max_batch_len=60000,
+    max_seq_len=4096,
+    save_samples=1000,
+    deepspeed_cpu_offload_optimizer=True,
+    is_padding_free=True,
+    num_epochs=8,
+    model_path=os.path.join(DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"),
+)
+
+
 def get_default_config() -> Config:
     """Generates default configuration for CLI"""
     return Config()
@@ -814,163 +1111,6 @@ def recreate_train_profiles(overwrite: bool = False) -> bool:
                     loaded = yaml.load(d)
                     yaml.dump(loaded, outfile)
     else:
-        # else render the defaults
-        EIGHT_GPU_TRAIN_AH = _train(
-            additional_args={
-                "warmup_steps": 25,
-                "learning_rate": 2e-5,
-                "lora_dropout": 0.1,
-                "lora_alpha": 32,
-            },
-            ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
-            data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
-            data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
-            nproc_per_node=8,
-            effective_batch_size=128,
-            lora_quantize_dtype=None,
-            lora_rank=0,
-            max_batch_len=60000,
-            max_seq_len=4096,
-            save_samples=1000,
-            deepspeed_cpu_offload_optimizer=False,
-            is_padding_free=True,
-            num_epochs=8,
-            model_path=os.path.join(
-                DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"
-            ),
-        )
-
-        FOUR_GPU_TRAIN_AH = _train(
-            additional_args={
-                "warmup_steps": 25,
-                "learning_rate": 2e-5,
-                "lora_dropout": 0.1,
-                "lora_alpha": 32,
-            },
-            ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
-            data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
-            data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
-            nproc_per_node=4,
-            effective_batch_size=128,
-            lora_quantize_dtype=None,
-            lora_rank=0,
-            max_batch_len=54000,
-            max_seq_len=4096,
-            save_samples=1000,
-            deepspeed_cpu_offload_optimizer=False,
-            is_padding_free=True,
-            num_epochs=8,
-            model_path=os.path.join(
-                DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"
-            ),
-        )
-
-        TWO_GPU_TRAIN_AH = _train(
-            additional_args={
-                "warmup_steps": 25,
-                "learning_rate": 2e-5,
-                "lora_dropout": 0.1,
-                "lora_alpha": 32,
-                "deepspeed_cpu_offload_optimizer_ratio": 1,
-                "deepspeed_cpu_offload_optimizer_pin_memory": True,
-            },
-            ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
-            data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
-            data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
-            nproc_per_node=2,
-            effective_batch_size=128,
-            lora_quantize_dtype=None,
-            lora_rank=0,
-            max_batch_len=60000,
-            max_seq_len=4096,
-            save_samples=1000,
-            deepspeed_cpu_offload_optimizer=True,
-            is_padding_free=True,
-            num_epochs=8,
-            model_path=os.path.join(
-                DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"
-            ),
-        )
-
-        EIGHT_L_FORTY_GPU = _train(
-            additional_args={
-                "warmup_steps": 25,
-                "learning_rate": 2e-5,
-                "lora_dropout": 0.1,
-                "lora_alpha": 32,
-            },
-            ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
-            data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
-            data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
-            nproc_per_node=8,
-            effective_batch_size=128,
-            lora_quantize_dtype=None,
-            lora_rank=0,
-            max_batch_len=30000,
-            max_seq_len=4096,
-            save_samples=1000,
-            deepspeed_cpu_offload_optimizer=False,
-            is_padding_free=True,
-            num_epochs=8,
-            model_path=os.path.join(
-                DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"
-            ),
-        )
-
-        FOUR_L_FORTY_GPU = _train(
-            additional_args={
-                "warmup_steps": 25,
-                "learning_rate": 2e-5,
-                "lora_dropout": 0.1,
-                "lora_alpha": 32,
-                "deepspeed_cpu_offload_optimizer_ratio": 1,
-                "deepspeed_cpu_offload_optimizer_pin_memory": True,
-            },
-            ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
-            data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
-            data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
-            nproc_per_node=4,
-            effective_batch_size=128,
-            lora_quantize_dtype=None,
-            lora_rank=0,
-            max_batch_len=30000,
-            max_seq_len=4096,
-            save_samples=1000,
-            num_epochs=8,
-            deepspeed_cpu_offload_optimizer=True,
-            is_padding_free=True,
-            model_path=os.path.join(
-                DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"
-            ),
-        )
-
-        EIGHT_L_FOUR_GPU = _train(
-            additional_args={
-                "warmup_steps": 25,
-                "learning_rate": 2e-5,
-                "lora_dropout": 0.1,
-                "lora_alpha": 32,
-                "deepspeed_cpu_offload_optimizer_ratio": 1,
-                "deepspeed_cpu_offload_optimizer_pin_memory": True,
-            },
-            ckpt_output_dir=os.path.join(DEFAULTS._data_dir, "checkpoints"),
-            data_output_dir=os.path.join(DEFAULTS._data_dir, "internal"),
-            data_path=os.path.join(DEFAULTS._data_dir, "datasets"),
-            nproc_per_node=8,
-            effective_batch_size=128,
-            lora_quantize_dtype=None,
-            lora_rank=0,
-            max_batch_len=30000,
-            max_seq_len=4096,
-            save_samples=1000,
-            num_epochs=8,
-            deepspeed_cpu_offload_optimizer=True,
-            is_padding_free=True,
-            model_path=os.path.join(
-                DEFAULTS._cache_home, "models/instructlab/granite-7b-lab"
-            ),
-        )
-
         to_write = {
             DEFAULTS.TRAIN_A100_H100_X4_PROFILE: FOUR_GPU_TRAIN_AH,
             DEFAULTS.TRAIN_A100_H100_X8_PROFILE: EIGHT_GPU_TRAIN_AH,
@@ -1155,3 +1295,54 @@ def storage_dirs_exist() -> bool:
         DEFAULTS.PHASED_DIR,
     ]
     return all(os.path.exists(dirpath) for dirpath in dirs_to_check)
+
+
+# {"GPU Name": ({ "gpu_count": NUM_GPUS, "vram_and_config": {"vram": TOTAL_VRAM, "config": TRAIN_CONFIG}})...}
+PROFILE_MAPPINGS = {
+    "L40s": (
+        {"gpu_count": 1, "vram_and_config": {"vram": 80, "config": SINGLE_L40}},
+        {
+            "gpu_count": 4,
+            "vram_and_config": {"vram": 192, "config": FOUR_L_FORTY_GPU},
+        },
+        {
+            "gpu_count": 8,
+            "vram_and_config": {"vram": 384, "config": EIGHT_L_FORTY_GPU},
+        },
+    ),
+    "L4": (
+        {"gpu_count": 1, "vram_and_config": {"vram": 24, "config": SINGLE_L4}},
+        {
+            "gpu_count": 8,
+            "vram_and_config": {"vram": 192, "config": EIGHT_L_FOUR_GPU},
+        },
+    ),
+    "A100": (
+        {
+            "gpu_count": 2,
+            "vram_and_config": {"vram": 160, "config": TWO_GPU_TRAIN_AH},
+        },
+        {
+            "gpu_count": 4,
+            "vram_and_config": {"vram": 320, "config": FOUR_GPU_TRAIN_AH},
+        },
+        {
+            "gpu_count": 8,
+            "vram_and_config": {"vram": 640, "config": EIGHT_GPU_TRAIN_AH},
+        },
+    ),
+    "H100": (
+        {
+            "gpu_count": 2,
+            "vram_and_config": {"vram": 160, "config": TWO_GPU_TRAIN_AH},
+        },
+        {
+            "gpu_count": 4,
+            "vram_and_config": {"vram": 320, "config": FOUR_GPU_TRAIN_AH},
+        },
+        {
+            "gpu_count": 8,
+            "vram_and_config": {"vram": 640, "config": EIGHT_GPU_TRAIN_AH},
+        },
+    ),
+}


Test patch:
diff --git a/tests/test_lab_init.py b/tests/test_lab_init.py
index 02f76b0de9..024c4c2744 100644
--- a/tests/test_lab_init.py
+++ b/tests/test_lab_init.py
@@ -15,6 +15,20 @@
 
 
 class TestLabInit:
+    @patch(
+        "instructlab.config.init.convert_bytes_to_proper_mag", return_value=(89.0, "GB")
+    )
+    def test_ilab_config_init_auto_detection(
+        self, convert_bytes_to_proper_mag_mock, cli_runner: CliRunner
+    ):
+        result = cli_runner.invoke(lab.ilab, ["config", "init", "--interactive"])
+        assert result.exit_code == 0, result.stdout
+        convert_bytes_to_proper_mag_mock.assert_called_once()
+        assert (
+            "We chose Nvidia 2x A100 as your designated training profile. This is for systems with 160 GB of vRAM"
+            in result.stdout
+        )
+
     # When using `from X import Y` you need to understand that Y becomes part
     # of your module, so you should use `my_module.Y`` to patch.
     # When using `import X`, you should use `X.Y` to patch.