[Dataset] RACE dataset Checksums error
Author: abarbosa94
State: closed
Labels: name: dataset bug

Hi there, I just would like to use this awesome lib to perform a dataset fine-tuning on RACE dataset. I have performed the following steps:

```
dataset = nlp.load_dataset("race")
len(dataset["train"]), len(dataset["validation"])
```

But then I got the following error:

```
---------------------------------------------------------------------------
NonMatchingChecksumError                  Traceback (most recent call last)
<ipython-input-15-8bf7603ce0ed> in <module>
----> 1 dataset = nlp.load_dataset("race")
      2 len(dataset["train"]), len(dataset["validation"])

~/miniconda3/envs/masters/lib/python3.8/site-packages/nlp/load.py in load_dataset(path, name, version, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, **config_kwargs)
    546 
    547     # Download and prepare data
--> 548     builder_instance.download_and_prepare(
    549         download_config=download_config, download_mode=download_mode, ignore_verifications=ignore_verifications,
    550     )

~/miniconda3/envs/masters/lib/python3.8/site-packages/nlp/builder.py in download_and_prepare(self, download_config, download_mode, ignore_verifications, try_from_hf_gcs, dl_manager, **download_and_prepare_kwargs)
    460                         logger.info("Dataset not on Hf google storage. Downloading and preparing it from source")
    461                 if not downloaded_from_gcs:
--> 462                     self._download_and_prepare(
    463                         dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
    464                     )

~/miniconda3/envs/masters/lib/python3.8/site-packages/nlp/builder.py in _download_and_prepare(self, dl_manager, verify_infos, **prepare_split_kwargs)
    519         # Checksums verification
    520         if verify_infos:
--> 521             verify_checksums(
    522                 self.info.download_checksums, dl_manager.get_recorded_sizes_checksums(), "dataset source files"
    523             )

~/miniconda3/envs/masters/lib/python3.8/site-packages/nlp/utils/info_utils.py in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     36     if len(bad_urls) > 0:
     37         error_msg = "Checksums didn't match" + for_verification_name + ":\n"
---> 38         raise NonMatchingChecksumError(error_msg + str(bad_urls))
     39     logger.info("All the checksums matched successfully" + for_verification_name)
     40 

NonMatchingChecksumError: Checksums didn't match for dataset source files:
['http://www.cs.cmu.edu/~glai1/data/race/RACE.tar.gz']
```

Comments: `NonMatchingChecksumError` means that the checksum of the downloaded file is not the expected one.
Either the file you downloaded was corrupted along the way, or the host updated the file.
Could you try to clear your cache and run `load_dataset` again ? If the error is still there, it means that there was an update in the data, and we may have to update the expected checksum value.

I just cleared the cache an run it again. The error persists ):

```
 nlp (master) $ rm -rf  /Users/abarbosa/.cache/huggingface/
 nlp (master) $ python
Python 3.8.5 (default, Aug  5 2020, 03:39:04)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import nlp
>>> dataset = nlp.load_dataset("race")
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.39k/4.39k [00:00<00:00, 661kB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.81k/1.81k [00:00<00:00, 644kB/s]
Using custom data configuration default
Downloading and preparing dataset race/default (download: 84.52 MiB, generated: 132.61 MiB, post-processed: Unknown size, total: 217.13 MiB) to /Users/abarbosa/.cache/huggingface/datasets/race/default/0.1.0/5461327f1a83549ca0d845a3159c806d2baf4f8d0d8f7d657157ce7cdf3899c2...
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25.4M/25.4M [01:03<00:00, 401kB/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/abarbosa/Documents/nlp/src/nlp/load.py", line 550, in load_dataset
    builder_instance.download_and_prepare(
  File "/Users/abarbosa/Documents/nlp/src/nlp/builder.py", line 471, in download_and_prepare
    self._download_and_prepare(
  File "/Users/abarbosa/Documents/nlp/src/nlp/builder.py", line 530, in _download_and_prepare
    verify_checksums(
  File "/Users/abarbosa/Documents/nlp/src/nlp/utils/info_utils.py", line 38, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
nlp.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['http://www.cs.cmu.edu/~glai1/data/race/RACE.tar.gz']
>>>
```

Dealing with the same issue please update the checksum on nlp library end. The data seems to have changed on their end.

We have a discussion on this datasets here: https://github.com/huggingface/nlp/pull/540

Feel free to participate if you have some opinion on the scope of data which should be included in this dataset.

At least for me, the file that was downloaded from CMU isn't the complete dataset, but a small subset of it (~25MB vs ~85MB). I've previously downloaded the dataset directly, so for my personal needs I could just swap out the corrupted file with the correct one. Perhaps you could host it like you do for the Wikipedia and BookCorpus datasets.



> At least for me, the file that was downloaded from CMU isn't the complete dataset, but a small subset of it (~25MB vs ~85MB). I've previously downloaded the dataset directly, so for my personal needs I could just swap out the corrupted file with the correct one. Perhaps you could host it like you do for the Wikipedia and BookCorpus datasets.

Could you upload this please?

> > At least for me, the file that was downloaded from CMU isn't the complete dataset, but a small subset of it (~25MB vs ~85MB). I've previously downloaded the dataset directly, so for my personal needs I could just swap out the corrupted file with the correct one. Perhaps you could host it like you do for the Wikipedia and BookCorpus datasets.
> 
> Could you upload this please?

Not sure if I can upload it according to their license ("You agree not to reproduce, duplicate, copy, sell, trade, resell or exploit for any commercial purpose, any portion of the contexts and any portion of derived data.").

I managed to fix it in #540 :)

Closing since @540 is merged

Thanks again @abarbosa94