Training does not stop as expected with finite iterable dataset
Author: qgallouedec
Pull request: no

### System Info

- `transformers` version: 4.32.1
- Platform: macOS-13.5-arm64-arm-64bit
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.23.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): 0.6.0 (cpu)
- Jax version: 0.3.25
- JaxLib version: 0.3.25
- Using GPU in script?: MPS device
- Using distributed or parallel set-up in script?: no

### Who can help?

@muellerzr and @pacman100

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

According to the documentation, training with a finite iterable dataset may stop before reaching the defined max_steps if all data is exhausted (see https://huggingface.co/docs/transformers/main_classes/trainer#transformers.TrainingArguments.max_steps). However, in practice, I observed that training does not stop as expected under these conditions.

```python
import torch
from datasets import Dataset
from torch import nn
from transformers import Trainer, TrainingArguments


class MyModule(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(2, 2)

    def forward(self, a, return_loss=True):
        output = self.linear(a)
        return {"loss": output.sum().abs()}


data = torch.tensor([[i, i] for i in range(10)], dtype=torch.float32)  # [[0., 0.], [1., 1.], [2., 2.], ...]
dataset = Dataset.from_dict({"a": data}).to_iterable_dataset()

args = TrainingArguments(output_dir=".", per_device_train_batch_size=1, max_steps=20)
trainer = Trainer(model=MyModule(), args=args, train_dataset=dataset)
trainer.train()
```

### Expected behavior

Based on the documentation, training should stop before `max_steps` if all data in a finite iterable dataset is used (10 steps in the example).

**Actual Behavior**

Learning continues until `max_steps` is reached, looping through the iterable dataset.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

This issue still needs to be addressed 

@qgallouedec I'm not sure said dataset is considered finite? Just iterable. So its expected, but not the expectation you have (where it's finite). Take this minimal example:

```python
import torch
from datasets import Dataset
from torch.utils.data import DataLoader


data = torch.tensor([[i, i] for i in range(10)], dtype=torch.float32)
dataset = Dataset.from_dict({"a": data}).to_iterable_dataset()

dl = DataLoader(dataset, batch_size=2)

for _ in range(2):
    for batch in dl:
        print(batch)
```

You'll find we iterate through the dataloader twice here, and as such the dataset twice. This would be a question for the Datasets library, not transformers.  

> @qgallouedec I'm not sure said dataset is considered finite? Just iterable. 

What is a finite iterable dataset then?

> You'll find we iterate through the dataloader twice here, and as such the dataset twice. 

That's not precisely true: the iterable is recreated when you do `for batch in dl`. So technically, we create two iterators, and we iterate only once through each of them.

> This would be a question for the Datasets library, not transformers.

I would disagree on that. The question is about the role of `max_steps` as a trainer argument. According to the doc:

> `max_steps` (`int`, _optional_, defaults to -1) — If set to a positive number, the total number of training steps to perform. Overrides num_train_epochs. In case of using a finite iterable dataset the training may stop before reaching the set number of steps when all data is exhausted

I may be wrong, but `Dataset.from_dict({"a": torch.tensor([[i, i] for i in range(10)])}).to_iterable_dataset()` seems to be an finite iterable dataset (since calling `next(it)` enough time raises a `StopIteration`)

There are two possibilities:
- Either the `max_step` documentation is unclear, at least to me.
- Or it's a bug, because the training doesn't stop when expected (in the example, after 10 iterations, instead of 20).

The docs are incorrect then, as the behavior is to re-iterate until max steps. This is true as well after talking to more core maintainers. Would you like to open a PR? (How long it's been like this, I can't likely say. But my guess is quite a while, at least 3-6 months)