Using IterableDataset, Trainer never calls `set_epoch` to increment and resets the epochs to 0 at the beginning of each epoch.  
Author: ssharpe42
Pull request: no

### System Info

I am using a Huggingface implementation of IterableDataset with the `set_epoch` method with the standard Trainer class. However, during training the `_epoch` attribute of the dataset is never changed.(https://github.com/huggingface/datasets/blob/0cc77d7f45c73698c31eab4f8cfff901044d0020/src/datasets/iterable_dataset.py#L1829)

In the Trainer docs, it says for an IterableDataset to "have a `set_epoch()` method that internally sets the seed of the RNGs used". Im not sure how to use this if Trainer doesn't internally call this at every epoch. 

Additionally, the IterableDatasetShard implementation in `accelerate` (https://github.com/huggingface/accelerate/blob/48d96319e0033fb8c8979072d97edf3995639029/src/accelerate/data_loader.py#L220) is different from `transformers` (https://github.com/huggingface/transformers/blob/bffac926ca6bc6c965a92bfbfd00c567a2c0fb90/src/transformers/trainer_pt_utils.py#L731) in the fact that it doesn't have the seed setting at the beginning of the iterator call. 

```
        if (
            not hasattr(self.dataset, "set_epoch")
            and hasattr(self.dataset, "generator")
            and isinstance(self.dataset.generator, torch.Generator)
        ):
            self.dataset.generator.manual_seed(self.seed + self.epoch)
```


`set_epoch` used to be called in the Trainer [here](https://github.com/huggingface/transformers/blob/75b13f82e91d03bed88bf6cf0e2efb85346fb311/src/transformers/trainer.py#L1354C20-L1354C20)  until this recent change by @muellerzr  in June https://github.com/huggingface/transformers/commit/ebd94b0f6f215f6bc0f70e61eba075eb9196f9ef

This official example assumes it is taken care of in the Trainer https://github.com/huggingface/transformers/blob/v4.33.3/examples/research_projects/robust-speech-event/run_speech_recognition_ctc_streaming.py

@pacman100

### Who can help?

@pacman100 @muellerzr

### Information

- [x] The official example scripts
- [X] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```
import torch
from torch.utils.data import IterableDataset
from transformers import (
    AutoModelForMaskedLM,
    AutoTokenizer,
    DataCollatorForLanguageModeling,
    Trainer,
    TrainingArguments,
)


data = [
    {
        "input_ids": torch.tensor([101, 2040, 2001, 1999, 14936, 102]),
        "token_type_ids": torch.tensor([0, 0, 0, 0, 0, 0]),
        "attention_mask": torch.tensor([1, 1, 1, 1, 1, 1]),
    }
]

class ExampleDataset(IterableDataset):
    def __init__(self, data):
        super().__init__()
        self.data = data * 20
        self.epoch = 0

    def set_epoch(self, epoch):
        self.epoch = epoch

    def __iter__(self):
        print("\nThis is the epoch: ", self.epoch, "\n")

        for x in self.data:
            yield x

    def __len__(self):
        return len(self.data)

tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
model = AutoModelForMaskedLM.from_pretrained("bert-base-cased")
train_args = TrainingArguments(
    output_dir="output",
    num_train_epochs=3,
    per_device_train_batch_size=5,
    # gradient_accumulation_steps=1,
    # dataloader_num_workers=3,
)
dc = DataCollatorForLanguageModeling(tokenizer=tokenizer)

trainer = Trainer(
    train_dataset=ExampleDataset(data),
    model=model,
    args=train_args,
    data_collator=dc,
)
trainer.train()
```

Output: 

```
This is the epoch:  0 

You're using a BertTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
/opt/conda/envs/fmmodel/lib/python3.8/site-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
  warnings.warn('Was asked to gather along dimension 0, but all '
 33%|██████████████████████████████████████████████                                                                                            | 7/21 [00:02<00:02,  4.75it/s]
This is the epoch:  0 

 62%|████████████████████████████████████████████████████████████████████████████████████▊                                                    | 13/21 [00:02<00:00,  9.72it/s]
This is the epoch:  0 


```

### Expected behavior

Based on the docstring in trainer describing the IterableDataset behavior I would expect "This is the epoch:  0" to be incremented to "This is the epoch:  1" and "This is the epoch:  2"

Comments:
Hi, thanks for your patience. https://github.com/huggingface/accelerate/pull/2057 will be aiming to fix this