lmsys

/

vicuna-13b-delta-v0

like 454

I have the same issue:

Loading base model
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████| 3/3 [00:04<00:00, 1.65s/it]
Loading delta
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████| 3/3 [00:14<00:00, 4.68s/it]
Applying delta
Applying delta: 0%| | 0/403 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/opt/homebrew/Cellar/python@3.10/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/homebrew/Cellar/python@3.10/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/Users/hardo/git/FastChat/fastchat/model/apply_delta.py", line 49, in
apply_delta(args.base_model_path, args.target_model_path, args.delta_path)
File "/Users/hardo/git/FastChat/fastchat/model/apply_delta.py", line 34, in apply_delta
param.data += delta.state_dict()[name]
RuntimeError: The size of tensor a (32001) must match the size of tensor b (32000) at non-singleton dimension 0

I have the same problem, and I don't know how to solve it. someone help me?

It seems you were using a newer version of fschat with these old weights.
Please checkout the version compatibility here https://github.com/lm-sys/FastChat/blob/main/docs/weights_version.md
We suggest you use the newer v1.1 weights.

I also get the similar problem when i deploy minigpt4 : RuntimeError: The size of tensor a (32001) must match the size of tensor b (32003) at non-singleton dimension 0
vicuna-13b-delta-v0
llama-13b-hf
fschat version is 0.1.10

after updating fastchat, and using v1.1, it worked.

Thanks!