IterableDatasetShard' object has no attribute '_epoch'
Author: johnchienbronci
Pull request: no

transformers>=4.35.2
finetune wav2vec2 ctc using multiple gpu by streaming mode have error.

This error occurs on multiple gpus. ( No errors will occur on a single GPU or transformers version 4.30.2)

code:
```
    class ShuffleCallback(TrainerCallback):
        def on_epoch_begin(self, args, state, control, train_dataloader, **kwargs):
            if isinstance(train_dataloader.dataset, IterableDatasetShard):
                pass  # set_epoch() is handled by the Trainer
            elif isinstance(train_dataloader.dataset, IterableDataset):
                train_dataloader.dataset.set_epoch(train_dataloader.dataset._epoch + 1)
```

error message:
```
Traceback (most recent call last):
  File "/workspace/wav2vec2/speech_recognition/run_speech_recognition_ctc_streaming.py", line 980, in <module>
    main()
  File "/workspace/wav2vec2/speech_recognition/run_speech_recognition_ctc_streaming.py", line 929, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/transformers/trainer.py", line 1537, in train
    return inner_training_loop(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/transformers/trainer.py", line 1807, in _inner_training_loop
    self.control = self.callback_handler.on_epoch_begin(args, self.state, self.control)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/transformers/trainer_callback.py", line 377, in on_epoch_begin
    return self.call_event("on_epoch_begin", args, state, control)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/transformers/trainer_callback.py", line 414, in call_event
    result = getattr(callback, event)(
  File "/workspace/wav2vec2/speech_recognition/run_speech_recognition_ctc_streaming.py", line 894, in on_epoch_begin
    train_dataloader.dataset.set_epoch(train_dataloader.dataset._epoch + 1)
AttributeError: 'IterableDatasetShard' object has no attribute '_epoch'. Did you mean: 'epoch'?
```


Comments:
Hi @johnchienbronci, thanks for raising this issue! 

The examples in the research projects, e.g. run_speech_recognition_ctc_streaming.py, aren't actively maintained. Have you observed this error anywhere else when using the library? 

I only found the problem with this script, because `ShuffleCallback` only appears in run_speech_recognition_ctc_streaming.py

@johnchienbronci OK. The `ShuffleCallback` is just a class in this script. To get the research example to work, you can modify the code to use `epoch` instead of `_epoch` in the call, which should resolve this.

@amyeroberts thanks, my current solution is check condition of multiple or one gpu
I found train_dataloader.dataset object is not same:
1 gpu:is `IterableDataset` (datasets.iterable_dataset.IterableDataset )
multiple gpu: `IterableDatasetShard` (accelerate.data_loader.IterableDatasetShard)

```
class ShuffleCallback(TrainerCallback):
    def on_epoch_begin(self, args, state, control, train_dataloader, **kwargs):
        if isinstance(train_dataloader.dataset, IterableDatasetShard):
            pass  # set_epoch() is handled by the Trainer
        elif isinstance(train_dataloader.dataset, IterableDataset):
            # train_dataloader.dataset.set_epoch(train_dataloader.dataset._epoch + 1)
            if int(os.environ["WORLD_SIZE"]) == 1: 
                train_dataloader.dataset.set_epoch(train_dataloader.dataset._epoch + 1)
            else:
                train_dataloader.dataset.set_epoch(train_dataloader.dataset.epoch + 1)
```

@johnchienbronci Thanks for sharing! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.