Repository: huggingface/accelerate

Issue:
_prepare_deepspeed(): TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'
### System Info

```Shell
- `Accelerate` version: 0.18.0.dev0
- Platform: Linux-6.2.7-zen1-1-zen-x86_64-with-glibc2.37
- Python version: 3.10.10
- Numpy version: 1.23.5
- PyTorch version (GPU?): 2.1.0a0+gitbdaf402 (True)
- `Accelerate` default config:
        - compute_environment: LOCAL_MACHINE
        - distributed_type: DEEPSPEED
        - mixed_precision: fp16
        - use_cpu: False
        - num_processes: 1
        - machine_rank: 0
        - num_machines: 1
        - rdzv_backend: static
        - same_network: True
        - main_training_function: main
        - deepspeed_config: {'gradient_accumulation_steps': 6, 'gradient_clipping': 1.0, 'offload_optimizer_device': 'none', 'offload_param_device': 'none', 'zero3_init_flag': False, 'zero_stage': 2}
        - fsdp_config: {}
        - megatron_lm_config: {}
        - downcast_bf16: no
        - tpu_use_cluster: False
        - tpu_use_sudo: False
        - tpu_env: []
        - dynamo_config: {'dynamo_backend': 'NVFUSER'}
```


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] One of the scripts in the examples/ folder of Accelerate or an officially supported `no_trainer` script in the `examples` folder of the `transformers` repo (such as `run_no_trainer_glue.py`)
- [X] My own task or dataset (give details below)

### Reproduction

# steps

1. have deepspeed enabled. i don't think the specific deepspeed config will be especially relevant here.

2. use `accelerate.prepare(*args)` where a `DataLoader` is present in `args`. the `DataLoader` must be the only `obj` in `args` that has the attribute `batch_size`. the value of `batch_size` in the `DataLoader` must be `None`.

### a wild `None` appeared

a value of `None` for `batch_size` appears to happen when `batch_sampler` is set for initializing the `DataLoader`. i believe this is intended behaviour.

in the relevant torch code from `DataLoader`'s `__init__` found in `torch/utils/data/dataloader.py`, we see:
```python
if batch_sampler is not None:
    # auto_collation with custom batch_sampler
    if batch_size != 1 or shuffle or sampler is not None or drop_last:
        raise ValueError('batch_sampler option is mutually exclusive '
                         'with batch_size, shuffle, sampler, and '
                         'drop_last')
    batch_size = None
    drop_last = False
```
then below:
```python
self.batch_size = batch_size
self.drop_last = drop_last
self.sampler = sampler
self.batch_sampler = batch_sampler
self.generator = generator
```

so when the `DataLoader` is passed to `accelerate.prepare()`, it has a `batch_size` attribute, but it's set to `None`.

### a speedy deceleration

now, in `_prepare_deepspeed()` found in `accelerate.py`, we have:
```python
is_dataloader_present = any(isinstance(obj, torch.utils.data.DataLoader) for obj in args)
if deepspeed_plugin.deepspeed_config["train_micro_batch_size_per_gpu"] == "auto" or is_dataloader_present:
    result = [
        self._prepare_one(obj, first_pass=True) if isinstance(obj, torch.utils.data.DataLoader) else obj
        for obj in args
    ]

    batch_sizes = [obj.batch_size for obj in args if hasattr(obj, "batch_size")]
    if self.split_batches:
        batch_sizes = [batch_size // self.num_processes for batch_size in batch_sizes]
    if len(batch_sizes) == 0:
        raise ValueError(
            "When using DeepSpeed `accelerate.prepare()` requires you to pass at least one of training or evaluation dataloaders "
            "or alternatively set an integer value in `train_micro_batch_size_per_gpu` in the deepspeed config file"
            "or assign integer value to `AcceleratorState().deepspeed_plugin.deepspeed_config['train_micro_batch_size_per_gpu']`."
        )

batch_size_per_device = min(batch_sizes) if deepspeed_plugin.is_train_batch_min else max(batch_sizes)
```

since we have a `DataLoader`, we enter the if statement. when `batch_sizes` is set using that list comprehension that uses `hasattr()`, with our setup, it should get set to `[None]`.

interestingly, python let's you use `min()` and `max()` on a list with `None` iff `None` is the only element in that list. in that case, it returns `None`. otherwise, you get a similar error to this one with `>`.

**consequently, `batch_size_per_device` gets set to `None`.**

so when we reach
```python
config_kwargs = {
    "train_micro_batch_size_per_gpu": batch_size_per_device,
    "train_batch_size": batch_size_per_device
    * deepspeed_plugin.deepspeed_config["gradient_accumulation_steps"]
    * self.num_processes,
    "gradient_clipping": 1.0,
    "zero_optimization.stage3_gather_16bit_weights_on_model_save": False,
}
```
we end up multiplying with a `Nonetype` when setting `"train_batch_size"`, leading to the error in the title.

### Expected behavior

```Shell
`batch_size_per_device` in `_prepare_deepspeed` gets set properly based on the min or max of `batch_sizes`.
```



Discussion:
cc @pacman100 

Resolving patch:
diff --git a/src/accelerate/accelerator.py b/src/accelerate/accelerator.py
--- a/src/accelerate/accelerator.py
+++ b/src/accelerate/accelerator.py
@@ -1279,6 +1279,13 @@ def _prepare_deepspeed(self, *args):
             batch_sizes = [obj.batch_size for obj in args if hasattr(obj, "batch_size")]
             if self.split_batches:
                 batch_sizes = [batch_size // self.num_processes for batch_size in batch_sizes]
+
+            if any(bs is None for bs in batch_sizes):
+                raise ValueError(
+                    "At least one of the dataloaders passed to `accelerate.prepare()` has `None` as batch size."
+                    "Please set an integer value in `train_micro_batch_size_per_gpu` in the deepspeed config file"
+                    "or assign integer value to `AcceleratorState().deepspeed_plugin.deepspeed_config['train_micro_batch_size_per_gpu']`."
+                )
             if len(batch_sizes) == 0:
                 raise ValueError(
                     "When using DeepSpeed `accelerate.prepare()` requires you to pass at least one of training or evaluation dataloaders "


Test patch: