internlm

/

internlm-chat-20b-4bit

like 24

When ran with lmdeploy, while converting the model i ran into this error:

layers.59.attention_norm.weight torch.Size([5120])
layers.59.ffn_norm.weight torch.Size([5120])
tok_embeddings.weight torch.Size([103168, 5120])
norm.weight torch.Size([5120])
output.weight torch.Size([103168, 5120])
Traceback (most recent call last):
File "/home/user/miniconda3/envs/internlm2/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/user/miniconda3/envs/internlm2/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/user/miniconda3/envs/internlm2/lib/python3.10/site-packages/lmdeploy/serve/turbomind/deploy.py", line 1028, in
fire.Fire(main)
File "/home/user/miniconda3/envs/internlm2/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/home/user/miniconda3/envs/internlm2/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/home/user/miniconda3/envs/internlm2/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/home/user/miniconda3/envs/internlm2/lib/python3.10/site-packages/lmdeploy/serve/turbomind/deploy.py", line 992, in main
res = deploy_hf(model_name, model_path, tokenizer_path,
File "/home/user/miniconda3/envs/internlm2/lib/python3.10/site-packages/lmdeploy/serve/turbomind/deploy.py", line 541, in deploy_hf
return export(model_name,
File "/home/user/miniconda3/envs/internlm2/lib/python3.10/site-packages/lmdeploy/serve/turbomind/deploy.py", line 202, in export
assert inter_size > 0
AssertionError