Qwen

/

Qwen2-VL-7B-Instruct-AWQ

like 43

Follow

Qwen 7.99k

This recent config.json change has broken the model for me: https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct-AWQ/commit/1dca9854ff028edf7df85575c4e86546f7faa9df

  File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3984, in from_pretrained
    ) = cls._load_pretrained_model(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4472, in _load_pretrained_model
    new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 973, in _load_state_dict_into_meta_model
    set_module_tensor_to_device(model, param_name, param_device, **set_module_kwargs)
  File "/usr/local/lib/python3.11/site-packages/accelerate/utils/modeling.py", line 373, in set_module_tensor_to_device
    raise ValueError(
ValueError: Trying to set a tensor of shape torch.Size([3584]) in "bias" (which has shape torch.Size([8192])), this looks incorrect.

reverting the change (revision 9d72ae62396aaa1817b006e07ddbbd121024f50d) fixes the issue, this is with git transformers.