Title: Dataset wikipedia and Bookcorpusopen cannot be fetched from dataloader.
Author: hobbitlzy (NONE)
State: closed
Created: 2021-10-11T11:54:59Z
Labels: bug

## Describe the bug
I have used the previous version of `transformers` and `datasets`. The dataset `wikipedia` can be successfully used. Recently, I upgrade them to the newest version and find it raises errors. I also tried other datasets. The `wikitext` works and the `bookcorpusopen` raises the same errors as `wikipedia`.

## Steps to reproduce the bug
Run the `run_mlm_no_trainer.py` and the given script on this [link](https://github.com/huggingface/transformers/tree/master/examples/pytorch/language-modeling). Change the dataset from wikitext to wikipedia or bookcorpusopen. BTW, the library transformers is of version 4.11.3.

## Expected results
The data batchs are fetched from the data loader and train.

## Actual results
The first time to fetch data batch occurs error.
`Traceback (most recent call last):
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 705, in convert_to_tensors
    tensor = as_tensor(value)
ValueError: too many dimensions 'str'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/original_run_mlm_no_trainer.py", line 528, in <module>
    main()
  File "src/original_run_mlm_no_trainer.py", line 488, in main
    for step, batch in enumerate(train_dataloader):
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/accelerate/data_loader.py", line 303, in __iter__
    for batch in super().__iter__():
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
    data = self._next_data()
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 557, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/data/data_collator.py", line 41, in __call__
    return self.torch_call(features)
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/data/data_collator.py", line 671, in torch_call
    batch = self.tokenizer.pad(examples, return_tensors="pt", pad_to_multiple_of=self.pad_to_multiple_of)
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2774, in pad
    return BatchEncoding(batch_outputs, tensor_type=return_tensors)
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 210, in __init__
    self.convert_to_tensors(tensor_type=tensor_type, prepend_batch_axis=prepend_batch_axis)
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 722, in convert_to_tensors
    "Unable to create tensor, you should probably activate truncation and/or padding "
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.
`


## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Linux-5.8.0-59-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.7.6
- PyArrow version: 5.0.0

Comments:
Hi ! I think this issue is more related to the `transformers` project. Could you open an issue on https://github.com/huggingface/transformers ?

Anyway I think the issue could be that both wikipedia and bookcorpusopen have an additional "title" column, contrary to wikitext which only has a "text" column. After calling `load_dataset`, can you try doing `dataset = dataset.remove_columns("title")` ?

Removing the "title" column works! Thanks for your advice.

Maybe I should still create an issue to `transformers' to mark this solution?