KeyError: 'base_model.model.model.layers.0.mlp.down_proj.lora_A.weight'
Hey I did finetuning and I tried to download my model but I keep getting this mistake :
Can you help me please ? :)
INFO 04-23 22:15:15 weight_utils.py:177] Using model weights format ['*.safetensors']
Process SpawnProcess-3:
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.10/dist-packages/uvicorn/_subprocess.py", line 78, in subprocess_started
target(sockets=sockets)
File "/usr/local/lib/python3.10/dist-packages/uvicorn/server.py", line 65, in run
return asyncio.run(self.serve(sockets=sockets))
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.10/dist-packages/uvicorn/server.py", line 69, in serve
await self._serve(sockets)
File "/usr/local/lib/python3.10/dist-packages/uvicorn/server.py", line 76, in _serve
config.load()
File "/usr/local/lib/python3.10/dist-packages/uvicorn/config.py", line 433, in load
self.loaded_app = import_from_string(self.app)
File "/usr/local/lib/python3.10/dist-packages/uvicorn/importer.py", line 19, in import_from_string
module = importlib.import_module(module_str)
File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/root/AI_labo/jobmaker/api.py", line 6, in
llm = LLM(model="DevSelego/Mistral7b_summarizer_v10")
File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/llm.py", line 112, in init
self.llm_engine = LLMEngine.from_engine_args(
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 196, in from_engine_args
engine = cls(
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 110, in init
self.model_executor = executor_class(model_config, cache_config,
File "/usr/local/lib/python3.10/dist-packages/vllm/executor/gpu_executor.py", line 37, in init
self._init_worker()
File "/usr/local/lib/python3.10/dist-packages/vllm/executor/gpu_executor.py", line 66, in _init_worker
self.driver_worker.load_model()
File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker.py", line 107, in load_model
self.model_runner.load_model()
File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 95, in load_model
self.model = get_model(
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader.py", line 101, in get_model
model.load_weights(model_config.model, model_config.download_dir,
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/llama.py", line 401, in load_weights
param = params_dict[name]
KeyError: 'base_model.model.model.layers.0.mlp.down_proj.lora_A.weight'