Repository: vllm-project/vllm

Issue:
v0.2.0 KeyError: 'base_lrs' 
Hi, 
when switching from 0.1.7. to 0.2.0 I encountered this error seen below. When switching back to 0.1.7 it works again. 
I run a Llama 13B model inside a Docker container. 
Any idea how to fix this error? 
```
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/project/vllm_app/run.py", line 33, in <module>
    vllm_single()
  File "/project/vllm_app/run.py", line 14, in vllm_single
    single_test.run_vllm_single()
  File "/project/vllm_app/vllm_samples/single_test.py", line 12, in run_vllm_single
    llm = LLM(model="data/models/llama/")
  File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/llm.py", line 89, in __init__
    self.llm_engine = LLMEngine.from_engine_args(engine_args)
  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 229, in from_engine_args
    engine = cls(*engine_configs,
  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 108, in __init__
    self._init_workers(distributed_init_method)
  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 140, in _init_workers
    self._run_workers(
  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 692, in _run_workers
    output = executor(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker.py", line 68, in init_model
    self.model = get_model(self.model_config)
  File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader.py", line 101, in get_model
    model.load_weights(model_config.model, model_config.download_dir,
  File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/llama.py", line 399, in load_weights
    param = state_dict[name]
KeyError: 'base_lrs'
```
Thank you all and have a nice day!


Discussion:
Seeing the same when trying to run samantha-mistral-instruct-7b.
I think this is because the model directory includes `scheduler.pt` or any `*.pt` file not for model weights. For now, please remove (or rename) the `*.pt` files from the model directory.

Resolving patch:
diff --git a/vllm/model_executor/weight_utils.py b/vllm/model_executor/weight_utils.py
--- a/vllm/model_executor/weight_utils.py
+++ b/vllm/model_executor/weight_utils.py
@@ -144,8 +144,18 @@ def prepare_hf_model_weights(
     for pattern in allow_patterns:
         hf_weights_files += glob.glob(os.path.join(hf_folder, pattern))
     if not use_safetensors:
+        # Exclude files that are not needed for inference.
+        # https://github.com/huggingface/transformers/blob/v4.34.0/src/transformers/trainer.py#L227-L233
+        blacklist = [
+            "training_args.bin",
+            "optimizer.bin",
+            "optimizer.pt",
+            "scheduler.pt",
+            "scaler.pt",
+        ]
         hf_weights_files = [
-            x for x in hf_weights_files if not x.endswith("training_args.bin")
+            f for f in hf_weights_files
+            if not any(f.endswith(x) for x in blacklist)
         ]
 
     if len(hf_weights_files) == 0 and use_safetensors and fall_back_to_pt:


Test patch: