baichuan-inc

/

Baichuan2-13B-Chat

like 407

When loading model, it reports below Error:

File "/Baichuan2/test_model.py", line 5, in
model = AutoModelForCausalLM.from_pretrained("baichuan-inc/Baichuan2-13B-Chat-4bits", device_map="auto", torch_dtype=torch.bfloat16, trust_remote_code=True)
File "/opt/conda/envs/baichuan2/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 558, in from_pretrained
return model_class.from_pretrained(
File "/root/.cache/huggingface/modules/transformers_modules/Baichuan2-13B-Chat-4bits/modeling_baichuan.py", line 664, in from_pretrained
dispatch_model(model, device_map=device_map)
File "/opt/conda/envs/baichuan2/lib/python3.10/site-packages/accelerate/big_modeling.py", line 343, in dispatch_model
check_device_map(model, device_map)
File "/opt/conda/envs/baichuan2/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 1129, in check_device_map
all_model_tensors = [name for name, _ in model.state_dict().items()]
File "/opt/conda/envs/baichuan2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1897, in state_dict
module.state_dict(destination=destination, prefix=prefix + name + '.', keep_vars=keep_vars)
File "/opt/conda/envs/baichuan2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1897, in state_dict
module.state_dict(destination=destination, prefix=prefix + name + '.', keep_vars=keep_vars)
File "/opt/conda/envs/baichuan2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1897, in state_dict
module.state_dict(destination=destination, prefix=prefix + name + '.', keep_vars=keep_vars)
[Previous line repeated 2 more times]
File "/opt/conda/envs/baichuan2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1894, in state_dict
self._save_to_state_dict(destination, prefix, keep_vars)
File "/opt/conda/envs/baichuan2/lib/python3.10/site-packages/bitsandbytes/nn/modules.py", line 237, in _save_to_state_dict
for k, v in self.weight.quant_state.as_dict(packed=True).items():
AttributeError: 'list' object has no attribute 'as_dict'