object of type 'IterableDataset' has no len() 
Author: johnchienbronci
Pull request: no

**run script**: 
    run_speech_recognition_ctc_streaming.py (Multi GPU CTC with Dataset Streaming)
```
for split, dataset in raw_datasets.items():
    vectorized_datasets[split] = (
        dataset.map(prepare_dataset)
        .remove_columns(raw_column_names[split] + ["target_text"])
        .with_format("torch")
    )
    if split == "train":
        vectorized_datasets[split] = vectorized_datasets[split].shuffle(
            buffer_size=data_args.shuffle_buffer_size,
            seed=training_args.seed,
        )
....
trainer = Trainer(
    model=model,
    data_collator=data_collator,
    args=training_args,
    compute_metrics=compute_metrics,
    train_dataset=vectorized_datasets["train"] if training_args.do_train else None,
    eval_dataset=vectorized_datasets["eval"] if training_args.do_eval else None,
    tokenizer=processor,
    callbacks=[ShuffleCallback()],
)
train_result = trainer.train(resume_from_checkpoint=checkpoint)
```
**Error**: 
```
Traceback (most recent call last):
  File "/usr/local/bin/wav2vec2/run_speech_recognition_ctc_streaming.py", line 702, in <module>
    main()
  File "/usr/local/bin/wav2vec2/run_speech_recognition_ctc_streaming.py", line 656, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/usr/local/lib/python3.10/dist-packages/transformers-4.30.0.dev0-py3.10.egg/transformers/trainer.py", line 1664, in train
    return inner_training_loop(
  File "/usr/local/lib/python3.10/dist-packages/transformers-4.30.0.dev0-py3.10.egg/transformers/trainer.py", line 1909, in _inner_training_loop
    for step, inputs in enumerate(epoch_iterator):
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py", line 633, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py", line 677, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/fetch.py", line 32, in fetch
    data.append(next(self.dataset_iter))
  File "/usr/local/lib/python3.10/dist-packages/datasets/iterable_dataset.py", line 981, in __iter__
    for key, example in ex_iterable:
  File "/usr/local/lib/python3.10/dist-packages/datasets/iterable_dataset.py", line 647, in __iter__
    for x in self.ex_iterable:
  File "/usr/local/lib/python3.10/dist-packages/datasets/iterable_dataset.py", line 512, in __iter__
    if self.remove_columns:
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataset.py", line 276, in __len__
    total += len(d)  # type: ignore[arg-type]
TypeError: object of type 'IterableDataset' has no len()
```
I'm try fix  but error change to  "_IterDataPipeSerializationWrapper' object has no attribute 'set_epoch"
```
    trainer = Trainer(
        model=model,
        data_collator=data_collator,
        args=training_args,
        compute_metrics=compute_metrics,
        train_dataset=IterableWrapper(vectorized_datasets["train"]) if training_args.do_train else None,
        eval_dataset=IterableWrapper(vectorized_datasets["eval"]) if training_args.do_eval else None,
        tokenizer=processor,
        callbacks=[ShuffleCallback()],
    )
```
Any thoughts on this?

Comments:
cc @sanchit-gandhi 

Hey @johnchienbronci - could you verify that all the columns `raw_column_names[split] + ["target_text"]` are in `raw_datasets[split]` for each split prior to calling the `.map` method? Failing that, could you paste the results of:
```
transformers-cli env
```

And also provide a reproducible code-snippet so that I can run the script locally on my end and reproduce the error result? Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.