Repository: EleutherAI/lm-evaluation-harness
Issue: Checksum failure on downloading pubmed_qa
Root cause: External Dependency Breakage

Related to #249 

Running `python ./tasks/eval_harness/download.py` in the eval_harness branch of Megatron-DeepSpeed creates the following error with the pubmed_qa dataset:

```
Downloading and preparing dataset pubmed_qa/pqa_labeled (download: 656.02 MiB, generated: 1.99 MiB, post-processed: Unknown size, total: 658.01 MiB) to /home/shogg/.cache/huggingface/datasets/pubmed_qa/pqa_labeled/1.0.0/2e65addecca4197502cd10ab8ef1919a47c28672f62d7abac7cc9afdcf24fb2d...
Downloading: 2.58MB [00:00, 17.5MB/s]                                                                                                                                                     | 0/3 [00:00<?, ?it/s]
Downloading: 3.29kB [00:00, 300kB/s]██████████████████████████▋                                                                                                                   | 1/3 [00:00<00:01,  1.10it/s]
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 533M/533M [01:36<00:00, 5.52MB/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [01:41<00:00, 33.87s/it]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 2208.69it/s]
Traceback (most recent call last):
  File "./tasks/eval_harness/download.py", line 20, in <module>
    main()
  File "./tasks/eval_harness/download.py", line 17, in main
    tasks.get_task_dict(task_list)
  File "/home/shogg/.pyenv/versions/Megatron-DeepSpeed/lib/python3.8/site-packages/lm_eval-0.1.0-py3.8.egg/lm_eval/tasks/__init__.py", line 320, in get_task_dict
  File "/home/shogg/.pyenv/versions/Megatron-DeepSpeed/lib/python3.8/site-packages/lm_eval-0.1.0-py3.8.egg/lm_eval/tasks/__init__.py", line 321, in <dictcomp>
  File "/home/shogg/.pyenv/versions/Megatron-DeepSpeed/lib/python3.8/site-packages/lm_eval-0.1.0-py3.8.egg/lm_eval/tasks/common.py", line 11, in __init__
  File "/home/shogg/.pyenv/versions/Megatron-DeepSpeed/lib/python3.8/site-packages/lm_eval-0.1.0-py3.8.egg/lm_eval/base.py", line 350, in __init__
  File "/home/shogg/.pyenv/versions/Megatron-DeepSpeed/lib/python3.8/site-packages/lm_eval-0.1.0-py3.8.egg/lm_eval/tasks/common.py", line 14, in download
  File "/home/shogg/.pyenv/versions/Megatron-DeepSpeed/lib/python3.8/site-packages/datasets/load.py", line 1632, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/shogg/.pyenv/versions/Megatron-DeepSpeed/lib/python3.8/site-packages/datasets/builder.py", line 607, in download_and_prepare
    self._download_and_prepare(
  File "/home/shogg/.pyenv/versions/Megatron-DeepSpeed/lib/python3.8/site-packages/datasets/builder.py", line 679, in _download_and_prepare
    verify_checksums(
  File "/home/shogg/.pyenv/versions/Megatron-DeepSpeed/lib/python3.8/site-packages/datasets/utils/info_utils.py", line 40, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/uc?export=download&id=1RsGLINVce-0GsDkCLDuLZmoLuzfmoCuQ']
```

This seems to be a HuggingFace `datasets` checksum error. Related issue: https://github.com/huggingface/datasets/issues/3820

Thanks for raising this issue. Fixed with update to `datasets==2.0.0`.