Repository: instructlab/instructlab

Issue:
In 'lab chat', tell the user if you are using a default model, please validate --model or -m before running
After running lab init, lab check, lab generate, lab train --device cuda (on Linux), I end up with a new .gguf file:

```bash
$ ls -l models/
total 14145392
-rw-rw-r-- 1 dperique dperique 14484862880 Mar 25 16:43 ggml-model-f16.gguf
lrwxrwxrwx 1 dperique dperique         148 Mar 15 20:56 merlinite-7b-Q4_K_M.gguf -> ../../../../../../../../.cache/huggingface/hub/models--ibm--merlinite-7b-GGUF/blobs/94f3a16321c9604ca22e970f3b89931ae5b4bbfd4c5d996e2bb606c506590666
```

This is the merged model (i.e., original model plus adapter).  If I try to chat with it via `lab chat -m ggml-model-f16.gguf`, it appears to load:

```bash
$ lab chat -m ggml-model-f16.gguf 
╭──────────────────────────────────────────────────────────── system ────────────────────────────────────────────────────────────╮
│ Welcome to Chat CLI w/ GGML-MODEL-F16.GGUF (type /h for help)                                                                  │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
>>>                                                                                                                   [S][default]
```

But that is NOT the correct model -- even though it confidently implies it in the "Welcome ..." message.

You can observe this more clearly using a non existent file:

```bash
$ lab chat -m something_non_existent.lala
╭──────────────────────────────────────────────────────────── system ────────────────────────────────────────────────────────────╮
│ Welcome to Chat CLI w/ SOMETHING_NON_EXISTENT.LALA (type /h for help)                                                          │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
>>>            
```

Clearly, we're defaulting to something (probably in the `config.yaml`) but the user should be informed explicitly -- silence is not acceptable.

With silence, the user may try to test the model manually only to find it does not appear to be fine-tuned.  This in fact was my case as I spent a lot of time trying to understand why my resultant gguf file didn't seem to incorporate my dataset.

Using the correct file loads the expected model:

```bash
$ lab chat -m ./models/ggml-model-f16.gguf
╭──────────────────────────────────────────────────────────── system ────────────────────────────────────────────────────────────╮
│ Welcome to Chat CLI w/ ./MODELS/GGML-MODEL-F16.GGUF (type /h for help)                                                         │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
>>> 
```

And my tests show the model was fine-tuned as expected.

We should validate the existence of the value of the `--model` or `-m` parameter to avoid the user being confused about what model is being used and/or invalidating long hours of testing.




Discussion:
I'll add a commit in https://github.com/instruct-lab/cli/pull/758 to fix this, it's the same code path triggering https://github.com/instruct-lab/cli/issues/683 as well. Thanks!
Digging further the. `-m` argument is just the name of the model, not the path. It's just used to display the name during the during chat session.

Perhaps a maintainer could chime in and give more context. However, I understand that from a user experience this might be confusing.
yes, it's confusing because most of the endpoint would actually check model name from the client side (i.e. `ilab chat`) so this would be avoided (or invalidated).
the complication is that 
- `ilab chat` now starts `ilab serve` automatically without having a way to specific the model to load
- `ilab serve` doesn't check (or have) a model name to validate
@cdoern could you take a closer look at this one? I remember you were telling me how you were helping someone out with `ilab chat` indepth today.
@cdoern suggesting for 5/30, when user specifies --model-path, we funnel that into the next lab chat unless they specify --model, and we wire --model up to ab chat to do something meaningful. when they do --model-path in lab serve, changes it in the config. so config.serv.modelpath is used for source of truth for chatting. Something in there needs to be changed for 5.2. 

user experience - if you serve model A, then do chat... it doesn't necessarily use model A (it uses the model configured as default in the config.) 

Default should probably be that chat always uses the model being served... or the model most recently served... 



Patch:
diff --git a/src/instructlab/chat/chat.py b/src/instructlab/chat/chat.py
index 2e0659e48a..f0da3b329e 100644
--- a/src/instructlab/chat/chat.py
+++ b/src/instructlab/chat/chat.py
@@ -22,7 +22,7 @@
 import openai
 
 # Local
-from ..config import DEFAULT_CONNECTION_TIMEOUT
+from ..config import DEFAULT_CONNECTION_TIMEOUT, DEFAULT_MODEL_OLD
 from ..utils import get_sysprompt
 
 HELP_MD = """
@@ -482,6 +482,19 @@ def chat_cli(
         timeout=DEFAULT_CONNECTION_TIMEOUT,
         http_client=httpx.Client(cert=cert, verify=verify),
     )
+    # ensure the model specified exists on the server. with backends like vllm, this is crucial.
+    model_list = client.models.list().data
+    model_ids = []
+    for m in model_list:
+        model_ids.append(m.id)
+    if not any(model == m for m in model_ids):
+        if model == DEFAULT_MODEL_OLD:
+            logger.info(
+                f"Model {model} is not a full path. Try running ilab init or edit your config to have the full model path for serving, chatting, and generation."
+            )
+        raise ChatException(
+            f"Model {model} is not served by the server. These are the served models: {model_ids}"
+        )
 
     # Load context/session
     loaded = {}
diff --git a/src/instructlab/config.py b/src/instructlab/config.py
index a57ffdfd09..cb629b0fd9 100644
--- a/src/instructlab/config.py
+++ b/src/instructlab/config.py
@@ -19,8 +19,10 @@
 
 DEFAULT_API_KEY = "no_api_key"
 DEFAULT_CONFIG = "config.yaml"
-DEFAULT_MODEL = "merlinite-7b-lab-Q4_K_M"
-DEFAULT_MODEL_PATH = f"models/{DEFAULT_MODEL}.gguf"
+# TODO: Consolidate --model and --model-path into one --model-path flag since we always need a path now
+DEFAULT_MODEL_OLD = "merlinite-7b-lab-Q4_K_M"
+DEFAULT_MODEL = "models/merlinite-7b-lab-Q4_K_M.gguf"
+DEFAULT_MODEL_PATH = "models/merlinite-7b-lab-Q4_K_M.gguf"
 DEFAULT_TAXONOMY_REPO = "https://github.com/instructlab/taxonomy.git"
 DEFAULT_TAXONOMY_PATH = "taxonomy"
 DEFAULT_TAXONOMY_BASE = "origin/main"
diff --git a/src/instructlab/generator/utils.py b/src/instructlab/generator/utils.py
index 0de079f3c6..2a11d075a3 100644
--- a/src/instructlab/generator/utils.py
+++ b/src/instructlab/generator/utils.py
@@ -16,7 +16,7 @@
 import httpx
 
 # Local
-from ..config import DEFAULT_API_KEY
+from ..config import DEFAULT_API_KEY, DEFAULT_MODEL_OLD
 from ..utils import get_sysprompt
 
 StrOrOpenAIObject = Union[str, object]
@@ -133,6 +133,21 @@ def openai_completion(
             http_client=httpx.Client(cert=cert, verify=verify),
         )
 
+        # ensure the model specified exists on the server. with backends like vllm, this is crucial.
+        model_list = client.models.list().data
+        model_ids = []
+        for model in model_list:
+            model_ids.append(model.id)
+        if not any(model_name == m for m in model_ids):
+            if model_name == DEFAULT_MODEL_OLD:
+                logging.info(
+                    "Model %s is not a full path. Try running ilab init or edit your config to have the full model path for serving, chatting, and generation.",
+                    model_name,
+                )
+            raise GenerateException(
+                f"Model {model_name} is not served by the server. These are the served models {model_ids}"
+            )
+
         messages = [
             {"role": "system", "content": get_sysprompt()},
             {"role": "user", "content": prompt_batch[batch_id]},
diff --git a/src/instructlab/lab.py b/src/instructlab/lab.py
index 0697ab58f1..30dee88061 100644
--- a/src/instructlab/lab.py
+++ b/src/instructlab/lab.py
@@ -4,7 +4,7 @@
 
 # Standard
 from glob import glob
-from os.path import basename, dirname, exists, splitext
+from os.path import basename, dirname, exists
 import json
 import logging
 import multiprocessing
@@ -215,9 +215,8 @@ def init(
         )
     click.echo(f"Generating `{config.DEFAULT_CONFIG}` in the current directory...")
     cfg = config.get_default_config()
-    model = splitext(basename(model_path))[0]
-    cfg.chat.model = model
-    cfg.generate.model = model
+    cfg.chat.model = model_path
+    cfg.generate.model = model_path
     cfg.serve.model_path = model_path
     cfg.generate.taxonomy_path = taxonomy_path
     cfg.generate.taxonomy_base = taxonomy_base
@@ -308,6 +307,8 @@ def diff(ctx, taxonomy_path, taxonomy_base, yaml_rules, quiet):
 @click.option(
     "--model-path",
     type=click.Path(),
+    default=config.DEFAULT_MODEL_PATH,
+    show_default=True,
     help="Path to the model used during generation.",
 )
 @click.option(
@@ -586,6 +587,8 @@ def generate(
 @click.option(
     "-m",
     "--model",
+    default=config.DEFAULT_MODEL,
+    show_default=True,
     help="Model name to print in chat process",
 )
 @click.option(


Test patch:
diff --git a/scripts/functional-tests.sh b/scripts/functional-tests.sh
index 6e7e50c12f..ac4c3136f2 100755
--- a/scripts/functional-tests.sh
+++ b/scripts/functional-tests.sh
@@ -344,28 +344,12 @@ test_model_print(){
         }
     '
 
-    # validate that we print the model from the CLI
+    # validate that we fail on invalid model
     expect -c '
         set timeout 30
         spawn ilab chat -m bar
         expect {
-            -re "Welcome to InstructLab Chat w/ \\\u001b\\\[1mBAR\\\u001b\\\[0m" { exit 0 }
-            eof { catch wait result; exit [lindex $result 3] }
-            timeout { exit 1 }
-        }
-    '
-
-    # validate that we print the model from the config
-    expect -c '
-        exec sed -i.bak "s/merlinite-7b-lab-Q4_K_M/baz/g" config.yaml
-        spawn ilab chat
-        expect {
-            -re "Welcome to InstructLab Chat w/ \\\u001b\\\[1mBAZ\\\u001b\\\[0m" {
-                exec sed -i.bak "s/baz/merlinite-7b-lab-Q4_K_M/g" config.yaml
-                exit 0
-            }
-            eof { catch wait result; exit [lindex $result 3] }
-            timeout { exit 1 }
+           "Executing chat failed with: Model bar is not served by the server. These are the served models: ['models/foo.gguf']" { exit 0 }
         }
     '
 
diff --git a/tests/test_config.py b/tests/test_config.py
index b0cacd1b8c..053b19e27f 100644
--- a/tests/test_config.py
+++ b/tests/test_config.py
@@ -17,7 +17,7 @@ def _assert_defaults(self, cfg):
         assert cfg.general.log_level == "INFO"
 
         assert cfg.chat is not None
-        assert cfg.chat.model == "merlinite-7b-lab-Q4_K_M"
+        assert cfg.chat.model == "models/merlinite-7b-lab-Q4_K_M.gguf"
         assert not cfg.chat.vi_mode
         assert cfg.chat.visible_overflow
         assert cfg.chat.context == "default"
@@ -26,7 +26,7 @@ def _assert_defaults(self, cfg):
         assert not cfg.chat.greedy_mode
 
         assert cfg.generate is not None
-        assert cfg.generate.model == "merlinite-7b-lab-Q4_K_M"
+        assert cfg.generate.model == "models/merlinite-7b-lab-Q4_K_M.gguf"
         assert cfg.generate.taxonomy_path == "taxonomy"
         assert cfg.generate.taxonomy_base == "origin/main"
         assert cfg.generate.num_cpus == 10
@@ -52,9 +52,9 @@ def test_minimal_config(self):
         with open(config_path, "w", encoding="utf-8") as config_file:
             config_file.write(
                 """chat:
-  model: merlinite-7b-lab-Q4_K_M
+  model: models/merlinite-7b-lab-Q4_K_M.gguf
 generate:
-  model: merlinite-7b-lab-Q4_K_M
+  model: models/merlinite-7b-lab-Q4_K_M.gguf
   taxonomy_base: origin/main
   taxonomy_path: taxonomy
 serve:
@@ -75,12 +75,12 @@ def test_full_config(self):
   context: default
   greedy_mode: false
   logs_dir: data/chatlogs
-  model: merlinite-7b-lab-Q4_K_M
+  model: models/merlinite-7b-lab-Q4_K_M.gguf
   session: null
   vi_mode: false
   visible_overflow: true
 generate:
-  model: merlinite-7b-lab-Q4_K_M
+  model: models/merlinite-7b-lab-Q4_K_M.gguf
   num_cpus: 10
   num_instructions: 100
   output_dir: generated
@@ -106,9 +106,9 @@ def test_config_unexpected_fields(self):
         with open(config_path, "w", encoding="utf-8") as config_file:
             config_file.write(
                 """chat:
-  model: merlinite-7b-lab-Q4_K_M
+  model: models/merlinite-7b-lab-Q4_K_M.gguf
 generate:
-  model: merlinite-7b-lab-Q4_K_M
+  model: models/merlinite-7b-lab-Q4_K_M.gguf
   taxonomy_base: origin/main
   taxonomy_path: taxonomy
 serve:
@@ -145,7 +145,7 @@ def test_config_missing_required_fields(self):
                 """general:
   log_level: INFO
 generate:
-  model: merlinite-7b-lab-Q4_K_M
+  model: models/merlinite-7b-lab-Q4_K_M.gguf
 """
             )
         with pytest.raises(