Repository: huggingface/accelerate

Issue:
Using `balanced_low_0` on single GPU crashes
### System Info

```Shell
- `Accelerate` version: 0.20.3
- Platform: Linux-6.3.7-arch1-1-x86_64-with-glibc2.37
- Python version: 3.10.8
- Numpy version: 1.23.5
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- PyTorch XPU available: False
- System RAM: 15.57 GB
- GPU type: NVIDIA GeForce GTX 970
- `Accelerate` default config:
	Not found
```


### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] One of the scripts in the examples/ folder of Accelerate or an officially supported `no_trainer` script in the `examples` folder of the `transformers` repo (such as `run_no_trainer_glue.py`)
- [X] My own task or dataset (give details below)

### Reproduction

Single GPU machine

```python
from transformers import AutoModel
model = AutoModel.from_pretrained("gpt2", device_map="balanced_low_mem_0")
```

Will crash:

```
│                                                                                                  │
│ /home/nicolas/.pyenv/versions/3.10.8/lib/python3.10/site-packages/accelerate/utils/modeling.py:5 │
│ 88 in get_balanced_memory                                                                        │
│                                                                                                  │
│    585 │   │   num_devices = len([d for d in max_memory if torch.device(d).type == "xpu" and ma  │
│    586 │                                                                                         │
│    587 │   module_sizes = compute_module_sizes(model, dtype=dtype, special_dtypes=special_dtype  │
│ ❱  588 │   per_gpu = module_sizes[""] // (num_devices - 1 if low_zero else num_devices)          │
│    589 │                                                                                         │
│    590 │   # We can't just set the memory to model_size // num_devices as it will end being too  │
│    591 │   # slightly less layers and some layers will end up offload at the end. So this funct  │
```

### Expected behavior

Few different options:

- Either using `low_mem_0` is not wanted for single GPU setup, then just show a proper error beforehand
- Or properly load the model sparing space on GPU0 (as much as possible).

The actual use case comes from quantization purposes : 
We need to load the model efficiently (ideally not at all).
Then start to load the head of the model to run it, then layer per layer run things in order to create the quantization.

https://github.com/huggingface/text-generation-inference/pull/438/files#diff-0cf72039dd2a1e24a0663ccb387ccc49ad7a510ca4ed0ed354695c5dc6693294R659-R667

`low_mem_0` seems like the simplest option. I also considered
```
model = AutoModel.from_pretrained("gpt2", device_map="auto", max_memory={0: "0GiB"})
```
which crashes with the same issue.


Bonus points if there's an easy way to recover layers that have actually been offloaded to disk (currently `layer[i].to(dev)`) will crash because they are on device `meta`, so we need to recover where the weights are.


Discussion:
`balanced_low_0` is not meant to be used on one GPU only, but we can easily fix the bug by doing nothing.

Resolving patch:
diff --git a/src/accelerate/utils/modeling.py b/src/accelerate/utils/modeling.py
--- a/src/accelerate/utils/modeling.py
+++ b/src/accelerate/utils/modeling.py
@@ -591,6 +591,10 @@ def get_balanced_memory(
             ]
         )
 
+    if num_devices == 1:
+        # We cannot do low_zero on just one GPU
+        low_zero = False
+
     module_sizes = compute_module_sizes(model, dtype=dtype, special_dtypes=special_dtypes)
     per_gpu = module_sizes[""] // (num_devices - 1 if low_zero else num_devices)
 


Test patch: