Loading quantization model video memory occupancy problem
Author: Doraemon20190612
Pull request: no

### System Info

The original model is loaded as an 8bit model and saved. When the saved quantization model is loaded again, the video memory occupancy is the same as that of the original model.


- `transformers` version: 4.29.0.dev0
- Platform: Windows-10-10.0.19045-SP0
- Python version: 3.9.12
- Huggingface_hub version: 0.13.3
- Safetensors version: not installed
- PyTorch version (GPU?): 1.12.0+cu113 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@sgugger @younesbelkada @Arthur

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction
code:
```
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('D:/DL/pretrain_model/bloom-560m')
model_q = AutoModelForCausalLM.from_pretrained('D:/DL/pretrain_model/bloom-560m', load_in_8bit=True, device_map='auto')
print(model.get_memory_footprint())
print(model_q.get_memory_footprint())

model_q.save_pretrained('D:/DL/model_result/bloom-560m-8bit')
model_q_again = AutoModelForCausalLM.from_pretrained('D:/DL/model_result/bloom-560m-8bit')
print(model_q_again.get_memory_footprint())
```


output:
```
C:\ProgramData\Anaconda3\envs\torch\lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Overriding torch_dtype=None with `torch_dtype=torch.float16` due to requirements of `bitsandbytes` to enable model loading in mixed int8. Either pass torch_dtype=torch.float16 or don't pass this argument at all to remove this warning.
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
CUDA SETUP: CUDA runtime path found: C:\ProgramData\Anaconda3\envs\torch\bin\cudart64_110.dll
CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 116
CUDA SETUP: Loading binary C:\ProgramData\Anaconda3\envs\torch\lib\site-packages\bitsandbytes\libbitsandbytes_cuda116.dll...
C:\ProgramData\Anaconda3\envs\torch\lib\site-packages\bitsandbytes\cuda_setup\main.py:141: UserWarning: C:\ProgramData\Anaconda3\envs\torch did not contain cudart64_110.dll as expected! Searching further paths...
  warn(msg)
C:\ProgramData\Anaconda3\envs\torch\lib\site-packages\bitsandbytes\cuda_setup\main.py:141: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('C:/ProgramData/Anaconda3/envs/torch/Library/mingw-w64/bin'), WindowsPath('C:/Program Files/MongoDB/Server/6.0/bin'), WindowsPath('C:/ProgramData/Anaconda3/envs/torch/Library/usr/bin')}
  warn(msg)
2236858368
816439296
Detected the presence of a `quantization_config` attribute in the model's configuration but you don't have the correct `bitsandbytes` version to support int8 serialization. Please install the latest version of `bitsandbytes` with  `pip install --upgrade bitsandbytes`.
2236858368
```

### Expected behavior

I hope that when I load the quantized saved model again, the video memory usage can correspond to the actual size of the quantized model

Comments:
Hi @Doraemon20190612 
Thanks for your interest in using this feature!
As stated on the warning:
```bash
Detected the presence of a `quantization_config` attribute in the model's configuration but you don't have the correct `bitsandbytes` version to support int8 serialization. Please install the latest version of `bitsandbytes` with  `pip install --upgrade bitsandbytes`.
```
Therefore you need to upgrade `bitsandbytes` as stated on the warning. Can you try:
```bash
pip install --upgrade bitsandbytes
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.