Can't download `trivia_qa/unfiltered`
Opened by VictorSanh (closed)

## Describe the bug

For some reason, I can't download `trivia_qa/unfilted`. A file seems to be missing... I am able to see it fine though the viewer tough...

## Steps to reproduce the bug
```python
>>> from datasets import load_dataset
>>> load_dataset("trivia_qa", "unfiltered")
Downloading and preparing dataset trivia_qa/unfiltered (download: 3.07 GiB, generated: 27.23 GiB, post-processed: Unknown size, total: 30.30 GiB) to /gpfsscratch/rech/six/commun/datasets/trivia_qa/unfiltered/1.1.0/9977a5d6f72acfd92f587de052403e8138b43bb0d1ce595016c3baf7e14deba6...
Traceback (most recent call last):
  File "/gpfswork/rech/six/commun/modules/datasets_modules/datasets/trivia_qa/9977a5d6f72acfd92f587de052403e8138b43bb0d1ce595016c3baf7e14deba6/trivia_qa.py", line 251, in _add_context
    with open(os.path.join(file_dir, fname), encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/gpfsscratch/rech/six/commun/datasets/downloads/extracted/9fcb7eddc6afd46fd074af3c5128931dfe4b548f933c925a23847faf4c1995ad/evidence/wikipedia/Peanuts.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/load.py", line 852, in load_dataset
    use_auth_token=use_auth_token,
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 616, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 693, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 1107, in _prepare_split
    disable=bool(logging.get_verbosity() == logging.NOTSET),
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/tqdm/std.py", line 1133, in __iter__
    for obj in iterable:
  File "/gpfswork/rech/six/commun/modules/datasets_modules/datasets/trivia_qa/9977a5d6f72acfd92f587de052403e8138b43bb0d1ce595016c3baf7e14deba6/trivia_qa.py", line 303, in _generate_examples
    example = parse_example(article)
  File "/gpfswork/rech/six/commun/modules/datasets_modules/datasets/trivia_qa/9977a5d6f72acfd92f587de052403e8138b43bb0d1ce595016c3baf7e14deba6/trivia_qa.py", line 274, in parse_example
    _add_context(article.get("EntityPages", []), "WikiContext", wiki_dir),
  File "/gpfswork/rech/six/commun/modules/datasets_modules/datasets/trivia_qa/9977a5d6f72acfd92f587de052403e8138b43bb0d1ce595016c3baf7e14deba6/trivia_qa.py", line 253, in _add_context
    except (IOError, datasets.Value("errors").NotFoundError):
  File "<string>", line 5, in __init__
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/features.py", line 265, in __post_init__
    self.pa_type = string_to_arrow(self.dtype)
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/features.py", line 134, in string_to_arrow
    f"Neither {datasets_dtype} nor {datasets_dtype + '_'} seems to be a pyarrow data type. "
ValueError: Neither errors nor errors_ seems to be a pyarrow data type. Please make sure to use a correct data type, see: https://arrow.apache.org/docs/python/api/datatypes.html#factory-functions
```

## Expected results
I am able to load another subset (`rc`), but unable to load.
I am not sure why the try/except doesn't catch it...
https://github.com/huggingface/datasets/blob/9675a5a1e7b99a86f9c250f6ea5fa5d1e6d5cc7d/datasets/trivia_qa/trivia_qa.py#L253

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform: Linux-4.18.0-147.51.2.el8_1.x86_64-x86_64-with-redhat-8.1-Ootpa
- Python version: 3.7.10
- PyArrow version: 3.0.0


Comments:
wooo that was fast! thank you @lhoestq !
it is able to process now, though it's ignoring all files and ending up with 0 examples now haha :/

For subset "unfiltered":
```python
>>> load_dataset("trivia_qa", "unfiltered")
Downloading and preparing dataset trivia_qa/unfiltered (download: 3.07 GiB, generated: 27.23 GiB, post-processed: Unknown size, total: 30.30 GiB) to /gpfsscratch/rech/six/commun/datasets/trivia_qa/unfiltered/1.1.0/910043a609bb2bdf62b4874f68e0c24fb648cf81e40a358f4bd54c919d72c9ab...
100%|███████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 1354.53it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 40.60it/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/load.py", line 1198, in load_dataset
    use_auth_token=use_auth_token,
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 647, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 748, in _download_and_prepare
    verify_splits(self.info.splits, split_dict)
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/utils/info_utils.py", line 74, in verify_splits
    raise NonMatchingSplitsSizesError(str(bad_splits))
datasets.utils.info_utils.NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='test', num_bytes=2906575347, num_examples=10832, dataset_name='trivia_qa'), 'recorded': SplitInfo(name='test', num_bytes=0, num_examples=0, dataset_name='trivia_qa')}, {'expected': SplitInfo(name='validation', num_bytes=3038966234, num_examples=11313, dataset_name='trivia_qa'), 'recorded': SplitInfo(name='validation', num_bytes=0, num_examples=0, dataset_name='trivia_qa')}]
```
For subset "rc":
```python
>>> load_dataset("trivia_qa", "rc")
Downloading and preparing dataset trivia_qa/rc (download: 2.48 GiB, generated: 14.92 GiB, post-processed: Unknown size, total: 17.40 GiB) to /gpfsscratch/rech/six/commun/datasets/trivia_qa/rc/1.1.0/910043a609bb2bdf62b4874f68e0c24fb648cf81e40a358f4bd54c919d72c9ab...
100%|███████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 3806.08it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 51.57it/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/load.py", line 1198, in load_dataset
    use_auth_token=use_auth_token,
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 647, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 748, in _download_and_prepare
    verify_splits(self.info.splits, split_dict)
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/utils/info_utils.py", line 74, in verify_splits
    raise NonMatchingSplitsSizesError(str(bad_splits))
datasets.utils.info_utils.NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='test', num_bytes=1577814583, num_examples=17210, dataset_name='trivia_qa'), 'recorded': SplitInfo(name='test', num_bytes=0, num_examples=0, dataset_name='trivia_qa')}, {'expected': SplitInfo(name='train', num_bytes=12750976012, num_examples=138384, dataset_name='trivia_qa'), 'recorded': SplitInfo(name='train', num_bytes=0, num_examples=0, dataset_name='trivia_qa')}, {'expected': SplitInfo(name='validation', num_bytes=1688535379, num_examples=18669, dataset_name='trivia_qa'), 'recorded': SplitInfo(name='validation', num_bytes=0, num_examples=0, dataset_name='trivia_qa')}]
```

Could you look into that when you get a chance?
I wonder if it's not something they changed on the file to download? i couldn't find any information

@VictorSanh have you tried passing `download_mode="force_redownload"`?
```python
ds = load_dataset("trivia_qa", "unfiltered", download_mode="force_redownload")
```

I aggressively rmed caches, especially rming the `datasets/downloads/extracted/c3d265fa20d99a147a76e4f5e...` solved the issue.
thank you both!