Bug in tranformers/modeling_utils.py
Author: nightandweather
Pull request: no

### System Info

I use torch 1.13.1 / and following bitsandbytes cmake install
```python
------------------------------------------------------------------------------------

Training...
  0%|                                                                                                                                                             | 0/3000 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/home/kanghoun/1.Language_Model/amc_emr_oncology_LLM/llama_test.py", line 394, in <module>
    train(model, tokenizer, dataset, file_path)
  File "/home/kanghoun/1.Language_Model/amc_emr_oncology_LLM/llama_test.py", line 352, in train
    train_result = trainer.train()
  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 1780, in train
    return inner_training_loop(
  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2134, in _inner_training_loop
    self.current_flos += float(self.floating_point_ops(inputs))
  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 3813, in floating_point_ops
    return self.model.floating_point_ops(inputs)
  File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py", line 1141, in floating_point_ops
    return 6 * self.estimate_tokens(input_dict) * self.num_parameters(exclude_embeddings=exclude_embeddings)
  File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py", line 1089, in num_parameters
    param.numel() * 2 * self.hf_quantizer.quantization_config.bnb_4bit_quant_storage.itemsize
AttributeError: 'torch.dtype' object has no attribute 'itemsize'
----------------------------------------------------------------------------------------

I changed self.hf_quantizer.quantization_config.bnb_4bit_quant_storage.itemsize 
-> torch_dtype_itemsize(self.hf_quantizer.quantization_config.bnb_4bit_quant_storage)

`import torch

def torch_dtype_itemsize(dtype):
    """
    Get the element size of the given torch.dtype object.

    Parameters:
        dtype (torch.dtype): The torch data type.

    Returns:
        int: The element size of the given data type.
    """
    # Dictionary mapping torch dtype to their respective sizes
    dtype_sizes = {
        torch.uint8: 1,
        torch.int8: 1,
        torch.int16: 2,
        torch.int32: 4,
        torch.int64: 8,
        torch.float16: 2,
        torch.float32: 4,
        torch.float64: 8,
    }

    # Retrieve the size from the dictionary, or return None if not found
    return dtype_sizes.get(dtype, None)
```
# Example usage
```python
tensor = torch.tensor([1, 2, 3], dtype=torch.uint8)
itemsize = torch_dtype_itemsize(tensor.dtype)
print("itemsize:", itemsize)
```
### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. I use torch 1.13.1 / and following bitsandbytes cmake install

2. in torch 1.13.1, there is no torch.dtype.itemsize

3. but tranformers/modeling_utils.py https://github.com/huggingface/transformers/blob/c5f0288bc7d76f65996586f79f69fba8867a0e67/src/transformers/modeling_utils.py#L1089
use itemsize()

so I make new function to work in



### Expected behavior

---

Comments:
EDIT AFTER INITIAL POST: Problem seems to be related to Python 3.11. Managed to run with Python 3.10 and  torch-2.2.1-cp310-cp310-manylinux1_x86_64.whl.metadata

I have encountered the same issue. 

My params are:

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_compute_dtype=compute_dtype,
    bnb_4bit_use_double_quant=True,
)

Error:

File ~/miniconda3/envs/311_gpu/lib/python3.11/site-packages/transformers/modeling_utils.py:1089, in ModuleUtilsMixin.num_parameters(self, only_trainable, exclude_embeddings)
   1084 if param.requires_grad or not only_trainable:
   1085     # For 4bit models, we need to multiply the number of parameters by 2 as half of the parameters are
   1086     # used for the 4bit quantization (uint8 tensors are stored)
   1087     if is_loaded_in_4bit and isinstance(param, bnb.nn.Params4bit):
   1088         total_numel.append(
-> 1089             param.numel() * 2 * self.hf_quantizer.quantization_config.bnb_4bit_quant_storage.itemsize
   1090         )
   1091     else:
   1092         total_numel.append(param.numel())

AttributeError: 'torch.dtype' object has no attribute 'itemsize'

THANK YOU!! 

cc @SunMarc @younesbelkada as it looks like it's quantization code which is triggering this, and should be compatible with multiple versions of python and torch


Indeed ! https://github.com/huggingface/transformers/pull/30162 should fix it - we've seen a simiar issue on PEFT recently https://github.com/huggingface/peft/pull/1635