Hugging Face's logo

BUAADreamer
/
Yi-VL-6B-hf

Visual Question Answering
Transformers
Safetensors
Chinese
English
llava
image-text-to-text
llama-factory
yi-vl
Inference Endpoints
Model card Files Community
5

ERROR | stderr | [rank0]: assert param.size() == loaded_weight.size()

#5
by visualjoyce - opened
Discussion
visualjoyce

Any hints on the error, I am using llamafactory from git repo and vllm=0.5.3post1

INFO 08-04 19:43:59 model_runner.py:680] Starting to load model /data/pretrained_models/Yi-VL-6B-hf...
INFO 08-04 19:43:59 selector.py:151] Cannot use FlashAttention-2 backend for Volta and Turing GPUs.
INFO 08-04 19:43:59 selector.py:54] Using XFormers backend.
2024-08-04 19:43:59 | ERROR | stderr | Loading safetensors checkpoint shards:   0% Completed | 0/5 [00:00<?, ?it/s]
2024-08-04 19:43:59 | ERROR | stderr |
2024-08-04 19:44:00 | ERROR | stderr | Loading safetensors checkpoint shards:  20% Completed | 1/5 [00:00<00:03,  1.22it/s]
2024-08-04 19:44:00 | ERROR | stderr |
2024-08-04 19:44:02 | ERROR | stderr | Loading safetensors checkpoint shards:  40% Completed | 2/5 [00:02<00:03,  1.08s/it]
2024-08-04 19:44:02 | ERROR | stderr |
2024-08-04 19:44:02 | ERROR | stderr | [rank0]: Traceback (most recent call last):
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "<frozen runpy>", line 198, in _run_module_as_main
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "<frozen runpy>", line 88, in _run_code
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/git/XinHaiLLM/backend/src/xinhai/workers/mllm.py", line 490, in <module>
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     worker = MLLMWorker()
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:              ^^^^^^^^^^^^
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/git/XinHaiLLM/backend/src/xinhai/workers/mllm.py", line 89, in __init__
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     self.engine: "BaseEngine" = VllmEngine(model_args, data_args, finetuning_args, generating_args)
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/git/XinHaiLLM/related_repos/LLaMA-Factory/src/llamafactory/chat/vllm_engine.py", line 102, in __init__
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     self.model = AsyncLLMEngine.from_engine_args(AsyncEngineArgs(**engine_args))
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/engine/async_llm_engine.py", line 466, in from_engine_args
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     engine = cls(
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:              ^^^^
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/engine/async_llm_engine.py", line 380, in __init__
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     self.engine = self._init_engine(*args, **kwargs)
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/engine/async_llm_engine.py", line 547, in _init_engine
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     return engine_class(*args, **kwargs)
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 251, in __init__
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     self.model_executor = executor_class(
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:                           ^^^^^^^^^^^^^^^
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 47, in __init__
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     self._init_executor()
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/executor/gpu_executor.py", line 36, in _init_executor
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     self.driver_worker.load_model()
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/worker/worker.py", line 139, in load_model
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     self.model_runner.load_model()
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/worker/model_runner.py", line 682, in load_model
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     self.model = get_model(model_config=self.model_config,
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/model_executor/model_loader/__init__.py", line 21, in get_model
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     return loader.load_model(model_config=model_config,
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/model_executor/model_loader/loader.py", line 283, in load_model
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     model.load_weights(
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/model_executor/models/llava.py", line 349, in load_weights
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     weight_loader(param, loaded_weight)
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:   File "/home/vimos/anaconda3/envs/xinhai/lib/python3.11/site-packages/vllm/model_executor/model_loader/weight_utils.py", line 468, in default_weight_loader
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:     assert param.size() == loaded_weight.size()
2024-08-04 19:44:02 | ERROR | stderr | [rank0]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-04 19:44:02 | ERROR | stderr | [rank0]: AssertionError
2024-08-04 19:44:03 | ERROR | stderr | Loading safetensors checkpoint shards:  40% Completed | 2/5 [00:03<00:04,  1.51s/it]
2024-08-04 19:44:03 | ERROR | stderr |
2024-08-04 19:44:03 | ERROR | stderr |

Sign up or log in to comment