Checksum mismatch for the reddit_tifu dataset
Opened by anna-kay (closed)

## Describe the bug
A checksum occurs when downloading the reddit_tifu data (both long & short).

## Steps to reproduce the bug
reddit_tifu_dataset = load_dataset('reddit_tifu', 'long')

## Expected results
The expected result is for the dataset to be downloaded and cached locally.

## Actual results
  File "/.../lib/python3.9/site-packages/datasets/utils/info_utils.py", line 40, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/uc?export=download&id=1ffWfITKFMJeqjT8loC8aiCLRNJpc_XnF']

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.3
- Platform: Linux-5.13.0-30-generic-x86_64-with-glibc2.31
- Python version: 3.9.7
- PyArrow version: 7.0.0


Comments:
Thanks for reporting, @anna-kay. We are fixing it.

@albertvillanova Thank you for the fast response! However I am still getting the same error:

Downloading: 2.23kB [00:00, ?B/s]
Traceback (most recent call last):
  File "C:\Users\Anna\PycharmProjects\summarization\main.py", line 17, in <module>
    dataset = load_dataset('reddit_tifu', 'long')
  File "C:\Users\Anna\Desktop\summarization\summarization_env\lib\site-packages\datasets\load.py", line 1702, in load_dataset
    builder_instance.download_and_prepare(
  File "C:\Users\Anna\Desktop\summarization\summarization_env\lib\site-packages\datasets\builder.py", line 594, in download_and_prepare
    self._download_and_prepare(
  File "C:\Users\Anna\Desktop\summarization\summarization_env\lib\site-packages\datasets\builder.py", line 665, in _download_and_prepare
    verify_checksums(
  File "C:\Users\Anna\Desktop\summarization\summarization_env\lib\site-packages\datasets\utils\info_utils.py", line 40, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/uc?export=download&id=1ffWfITKFMJeqjT8loC8aiCLRNJpc_XnF']

I have cleaned the cache/huggingface/datasets & cache/huggingface/modules files and also tried on another machine with a fresh installation of trasnformers & datasets. 
The reddit_tifu.py that gets downloaded still has the previous url on line 51, _URL = "https://drive.google.com/uc?export=download&id=1ffWfITKFMJeqjT8loC8aiCLRNJpc_XnF" 

Hi @anna-kay, I'm sorry I didn't clearly explain the details to you:
- the error has been fixed in our `master` branch on GitHub: https://github.com/huggingface/datasets/commit/8ae21bf6a77175dc803ce2f1b93d18b8fbf45586
- the fix will not be accessible to users in PyPI until our next release of the `datasets` library
  - our latest release (version 1.18.3) was made 23 days ago: https://github.com/huggingface/datasets/releases/tag/1.18.3
- in the meantime, you can get the fix if you install datasets from our GitHub `master` branch:
  ```
  pip install git+https://github.com/huggingface/datasets#egg=datasets
  ```

@albertvillanova  Ok great, makes sence. Thank you very much for the explanation!