PretrainedModel.from_pretrained does not work with load_in_8bit=True, llm_int8_enable_fp32_cpu_offload=True and device_map='auto'
Author: sgsdxzy
Pull request: no

### System Info

- `transformers` version: 4.27.0.dev0
- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.10.8
- Huggingface_hub version: 0.12.1
- PyTorch version (GPU?): 1.13.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help?

@Nas

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
quantization_config = BitsAndBytesConfig(load_in_8bit=True, llm_int8_enable_fp32_cpu_offload=True)
AutoModelForCausalLM.from_pretrained(path, device_map='auto', quantization_config=quantization_config)
```

If the model does not fit into VRAM, it reports:
```
Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit
the quantized model. If you have set a value for `max_memory` you should increase that. To have
an idea of the modules that are set on the CPU or RAM you can print model.hf_device_map.
```

auto-created device map example:
```
{'model.decoder.embed_tokens': 0, 'model.decoder.layers.0': 0, 'model.decoder.layers.1': 0, 'model.decoder.layers.2': 0, 'model.decoder.layers.3': 0, 'model.decoder.layers.4': 0, 'model.decoder.layers.5': 0, 'model.decoder.layers.6': 0, 'model.decoder.layers.7': 0, 'model.decoder.layers.8': 0, 'model.decoder.layers.9': 0, 'model.decoder.layers.10': 0, 'model.decoder.layers.11': 0, 'model.decoder.layers.12': 0, 'model.decoder.layers.13': 0, 'model.decoder.layers.14': 0, 'model.decoder.layers.15': 0, 'model.decoder.layers.16': 0, 'model.decoder.layers.17': 0, 'model.decoder.layers.18': 0, 'model.decoder.layers.19': 0, 'model.decoder.layers.20': 0, 'model.decoder.layers.21': 0, 'model.decoder.layers.22': 0, 'model.decoder.layers.23': 0, 'model.decoder.layers.24': 0, 'model.decoder.layers.25': 'cpu', 'model.decoder.layers.26': 'cpu', 'model.decoder.layers.27': 'cpu', 'model.decoder.layers.28': 'cpu', 'model.decoder.layers.29': 'cpu', 'model.decoder.layers.30': 'cpu', 'model.decoder.layers.31': 'cpu', 'model.decoder.layers.32': 'cpu', 'model.decoder.layers.33': 'cpu', 'model.decoder.layers.34': 'cpu', 'model.decoder.layers.35': 'cpu', 'model.decoder.layers.36': 'cpu', 'model.decoder.layers.37': 'cpu', 'model.decoder.layers.38': 'cpu', 'model.decoder.layers.39': 'cpu', 'model.decoder.norm': 'cpu', 'lm_head': 'cpu'}
```

### Expected behavior

It should auto create device_map, quantize what's in VRAM to int8, and keep what on cpu/RAM as float32.

In fact if the `device_map` is passed manually it runs correctly. The problem is that `PretrainedModel.from_pretrained` expand `device='auto'` to acutal mapping after populating `modules_to_not_convert`, so modules automatically offloaded to RAM are missing from the list. If I edit modeling_utils.py and expand `device='auto'` before `replace_8bit_linear` it works correctly.

Comments:
All the weights offloaded to the CPU won't be in int8 though, so the model is not loaded in 8 bits as requested. This is why we throw an error and choose not to support this use case (cc @younesbelkada ).

Yes, as explained by @sgugger we don't support `device_map=auto` + `llm_int8_enable_fp32_cpu_offload` you need to pass a custom device map as explained in https://huggingface.co/docs/transformers/main/en/main_classes/quantization#offload-between-cpu-and-gpu 
The main motivation behind that is that we want to avoid unexpected behavior for users that are new to this feature and prefer to support this only for advanced use cases where users know exactly what they are doing. 
I agree though the warning message is slightly misleading and we can phrase it differently 

@sgsdxzy 
As #22026 has been merged it closed the issue, feel free to re-open the issue if you think that there is something that needs to be fixed
Thanks!