load_dataset("squad") doesn't work in 2.7.1 and 2.10.1
Opened by yaseen157 (closed)

### Describe the bug

There is an issue that seems to be unique to the "squad" dataset, in which it cannot be loaded using standard methods. This issue is most quickly reproduced from the command line, using the HF examples to verify a dataset is loaded properly.

This is not a problem with "squad_v2" dataset for example.

### Steps to reproduce the bug

cmd line
> $ python -c "from datasets import load_dataset; print(load_dataset('squad', split='train')[0])"

OR

Python IDE
> from datasets import load_dataset
> load_dataset("squad")

### Expected behavior

I expected to either see the output described here from running the very same command in command line ([https://huggingface.co/docs/datasets/installation]), or any output that does not raise Python's TypeError.

There is some funky behaviour in the dataset builder portion of the codebase that means it is trying to import the squad dataset with an incorrect path, or the squad dataset couldn't be downloaded. I'm not really sure what the problem is beyond that. Messing around with caching I did manage to get it to load the dataset once, and then couldn't repeat this.

### Environment info

datasets=2.7.1 **or** 2.10.1, python=3.10.8, Linux 3.10.0-1160.36.2.el7.x86_64 **or** Windows 10-64


Comments:
Thanks for reporting, @yaseen157.

Could you please give the complete error stack trace?

I am not able to reproduce your issue: the dataset loads perfectly on my local machine and on a Colab notebook: https://colab.research.google.com/drive/1Fbdoa1JdNz8DOdX6gmIsOK1nCT8Abj4O?usp=sharing
```python
In [1]: from datasets import load_dataset

In [2]: ds = load_dataset("squad")
Downloading builder script: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5.27k/5.27k [00:00<00:00, 3.22MB/s]
Downloading metadata: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.36k/2.36k [00:00<00:00, 1.60MB/s]
Downloading readme: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7.67k/7.67k [00:00<00:00, 4.58MB/s]
Downloading and preparing dataset squad/plain_text to ...t/.cache/huggingface/datasets/squad/plain_text/1.0.0/d6ec3ceb99ca480ce37cdd35555d6cb2511d223b9150cce08a837ef62ffea453...
Downloading data: 30.3MB [00:00, 91.8MB/s]                                                                                                                                           | 0/2 [00:00<?, ?it/s]
Downloading data: 4.85MB [00:00, 75.3MB/s]                                                                                                                                                                 
Downloading data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  2.31it/s]
Extracting data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 2157.01it/s]
Dataset squad downloaded and prepared to .../.cache/huggingface/datasets/squad/plain_text/1.0.0/d6ec3ceb99ca480ce37cdd35555d6cb2511d223b9150cce08a837ef62ffea453. Subsequent calls will reuse this data.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 463.95it/s]

In [3]: ds
Out[3]: 
DatasetDict({
    train: Dataset({
        features: ['id', 'title', 'context', 'question', 'answers'],
        num_rows: 87599
    })
    validation: Dataset({
        features: ['id', 'title', 'context', 'question', 'answers'],
        num_rows: 10570
    })
})
```

I am at a complete loss for what's happening here. A quick summary, I have 3 machines to try this with:
1) My windows 10 laptop
2) Linux machine1, super computer login node
3) Linux machine2, super computer compute node

Let's define the following as a test script for the machines:

```
import traceback
import datasets
print(f"{datasets.__version__=}")
try:
    ds = datasets.load_dataset("squad")
except:
    traceback.print_exc()
else:
    print("Success!")
```

The Windows laptop enters some sort of traceback recursion loop:

> datasets.__version__='2.7.1'
> Downloading and preparing dataset squad/plain_text to C:/Users/yr3g17/.cache/huggingface/datasets/squad/plain_text/1.0.0/d6ec3ceb99ca480ce37cdd35555d6cb2511d223b9150cce08a837ef62ffea453...
> Downloading data files: 100%|██████████| 2/2 [00:00<?, ?it/s]
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
>   File "C:\Users\yr3g17\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main
>     exitcode = _main(fd, parent_sentinel)
>   File "C:\Users\yr3g17\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 125, in _main
>     prepare(preparation_data)
>   File "C:\Users\yr3g17\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 236, in prepare
>     _fixup_main_from_path(data['init_main_from_path'])
>   File "C:\Users\yr3g17\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
>     main_content = runpy.run_path(main_path,
>   File "C:\Users\yr3g17\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 267, in run_path
>     code, fname = _get_code_from_file(run_name, path_name)
>   File "C:\Users\yr3g17\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 237, in _get_code_from_file
>     with io.open_code(decoded_path) as f:
> OSError: [Errno 22] Invalid argument: 'C:\\Users\\yr3g17\\OneDrive - University of Southampton\\Documents\\PhD-repository\\<input>'
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
>   File "C:\Users\yr3g17\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main
>     exitcode = _main(fd, parent_sentinel)
>   File "C:\Users\yr3g17\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 125, in _main
>     prepare(preparation_data)
**this error traceback is endlessly recursive**

This is a brand new issue that started today and I didn't even realise was a thing, as I had been using my windows machine to follow tracebacks for the other machines...

I suspect this issue had something to do with my filepath naming, but I couldn't confirm this when I spent time trying to debug this myself weeks ago, something to do with files being locked and never released. I'm not too concerned about my laptop not working here because I've had so many issues with Microsoft OneDrive and my filesystem.

Linux machines 1 and 2 were working fine for months, but have all of a sudden stopped working. Trying to run linux machine 1 (login node), I get:

> datasets.__version__='2.10.1'
> Downloading and preparing dataset json/squad to /home/yr3g17/.cache/hugg
ingface/datasets/json/squad-d733af945be1d2c2/0.0.0/0f7e3662623656454fcd2
b650f34e886a7db4b9104504885bd462096cc7a9f51...
> Downloading data files: 100%|███████████████████████████████████████████
█████████████████████████████████████████████| 2/2 [00:00<00:00, 4042.70
it/s]
>Extracting data files: 100%|███████████████████████████████████████
███████████████████████████████████████████████████| 2/2 [00:00<00:00, 1
11.15it/s]
> Generating train split: 0 examples [00:00, ? examples/s]

 and hangs here. This has not happened to me before on the Linux machine. If I forcefully keyboard interrupt, I get:
 
> Traceback (most recent call last):
>   File "<stdin>", line 2, in <module>
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/d
> atasets/load.py", line 1782, in load_dataset
>     builder_instance.download_and_prepare(
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/d
> atasets/builder.py", line 793, in download_and_prepare
>     with FileLock(lock_path) if is_local else contextlib.nullcontext():
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/d
> atasets/utils/filelock.py", line 320, in __enter__
>     self.acquire()
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/d
> atasets/utils/filelock.py", line 282, in acquire
>     time.sleep(poll_intervall)

Which also appears to be file lock related! I resolved this by navigating to my ~/.cache/huggingface/datasets directory and wiping out anything to do with the squad dataset in *.lock files. Now I get:

```
from datasets import load_dataset
dataset_load("squad")

```
> Downloading and preparing dataset squad/plain_text to /home/yr3g17/.cache/huggingface/datasets/squad/plain_text/1.0.0/d6ec3ceb99ca480ce37cdd35555d6cb
> 2511d223b9150cce08a837ef62ffea453...
> Downloading data files: 100%|██████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 44.75it/s]
> Extracting data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  8.54it/s]
> Dataset squad downloaded and prepared to /home/yr3g17/.cache/huggingface/datasets/squad/plain_text/1.0.0/d6ec3ceb99ca480ce37cdd35555d6cb2511d223b9150
> cce08a837ef62ffea453. Subsequent calls will reuse this data.
> 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 19.77it/s]
> DatasetDict({
>     train: Dataset({
>         features: ['id', 'title', 'context', 'question', 'answers'],
>         num_rows: 87599
>     })
>     validation: Dataset({
>         features: ['id', 'title', 'context', 'question', 'answers'],
>         num_rows: 10570
>     })
> })
> 

Which all seems fine right, it's doing what it should be. But now, without ever leaving the IDE, I "make a subsequent call" to reuse the data by repeating the command. I encounter the following traceback

`load_dataset("squad")`

> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/datasets/load.py", line 1759, in load_dataset
>     builder_instance = load_dataset_builder(
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/datasets/load.py", line 1496, in load_dataset_builder
>     dataset_module = dataset_module_factory(
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/datasets/load.py", line 1151, in dataset_module_factory
>     ).get_module()
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/datasets/load.py", line 631, in get_module
>     data_files = DataFilesDict.from_local_or_remote(
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/datasets/data_files.py", line 796, in from_local_or_remote
>     DataFilesList.from_local_or_remote(
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/datasets/data_files.py", line 764, in from_local_or_remote
>     data_files = resolve_patterns_locally_or_by_urls(base_path, patterns, allowed_extensions)
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/datasets/data_files.py", line 369, in resolve_patterns_locally_or_by_urls
>     raise FileNotFoundError(error_msg)
> FileNotFoundError: Unable to resolve any data file that matches '['train[-._ 0-9/]**', '**[-._ 0-9/]train[-._ 0-9/]**', 'training[-._ 0-9/]**', '**[-
> ._ 0-9/]training[-._ 0-9/]**']' at /mainfs/home/yr3g17/.cache/huggingface/datasets/squad with any supported extension ['csv', 'tsv', 'json', 'jsonl',
>  'parquet', 'txt', 'blp', 'bmp', 'dib', 'bufr', 'cur', 'pcx', 'dcx', 'dds', 'ps', 'eps', 'fit', 'fits', 'fli', 'flc', 'ftc', 'ftu', 'gbr', 'gif', 'gr
> ib', 'h5', 'hdf', 'png', 'apng', 'jp2', 'j2k', 'jpc', 'jpf', 'jpx', 'j2c', 'icns', 'ico', 'im', 'iim', 'tif', 'tiff', 'jfif', 'jpe', 'jpg', 'jpeg', '
> mpg', 'mpeg', 'msp', 'pcd', 'pxr', 'pbm', 'pgm', 'ppm', 'pnm', 'psd', 'bw', 'rgb', 'rgba', 'sgi', 'ras', 'tga', 'icb', 'vda', 'vst', 'webp', 'wmf', '
> emf', 'xbm', 'xpm', 'BLP', 'BMP', 'DIB', 'BUFR', 'CUR', 'PCX', 'DCX', 'DDS', 'PS', 'EPS', 'FIT', 'FITS', 'FLI', 'FLC', 'FTC', 'FTU', 'GBR', 'GIF', 'G
> RIB', 'H5', 'HDF', 'PNG', 'APNG', 'JP2', 'J2K', 'JPC', 'JPF', 'JPX', 'J2C', 'ICNS', 'ICO', 'IM', 'IIM', 'TIF', 'TIFF', 'JFIF', 'JPE', 'JPG', 'JPEG',
> 'MPG', 'MPEG', 'MSP', 'PCD', 'PXR', 'PBM', 'PGM', 'PPM', 'PNM', 'PSD', 'BW', 'RGB', 'RGBA', 'SGI', 'RAS', 'TGA', 'ICB', 'VDA', 'VST', 'WEBP', 'WMF',
> 'EMF', 'XBM', 'XPM', 'aiff', 'au', 'avr', 'caf', 'flac', 'htk', 'svx', 'mat4', 'mat5', 'mpc2k', 'ogg', 'paf', 'pvf', 'raw', 'rf64', 'sd2', 'sds', 'ir
> cam', 'voc', 'w64', 'wav', 'nist', 'wavex', 'wve', 'xi', 'mp3', 'opus', 'AIFF', 'AU', 'AVR', 'CAF', 'FLAC', 'HTK', 'SVX', 'MAT4', 'MAT5', 'MPC2K', 'O
> GG', 'PAF', 'PVF', 'RAW', 'RF64', 'SD2', 'SDS', 'IRCAM', 'VOC', 'W64', 'WAV', 'NIST', 'WAVEX', 'WVE', 'XI', 'MP3', 'OPUS', 'zip']

It doesn't even appear like I can reliably repeat this process. I'll nuke squad files in my dataset cache and run the Python code again (which downloads a new copy of the dataset to cache). It will either fail (as it just did in the quote above), or it will successfully recall the dataset.

I repeated this nuking process a few times until calling load_dataset was reliably giving me the correct result (no filelocking issues or tracebacks). I then sent the test script as a job to the supercomputer compute nodes (which do not have internet access and therefore depend on cached data from Linux machine 1 login nodes)

> Using the latest cached version of the module from /home/yr3g17/.cache/huggingface/modules/datasets_modules/datasets/squad/8730650fed465361f38ac4d810
> ccdd16e8fc87b56498e52fb7e2cadaefc1f177 (last modified on Tue Feb 14 10:12:56 2023) since it couldn't be found locally at squad., or remotely on the Hugging Face Hub.
> Traceback (most recent call last):
>   File "/mainfs/scratch/yr3g17/squad_qanswering/3054408/0/../../main.py", line 5, in <module>
>     dataset = load_dataset("squad")
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/datasets/load.py", line 1759, in load_dataset
>     builder_instance = load_dataset_builder(
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/datasets/load.py", line 1522, in load_dataset_builder
>     builder_instance: DatasetBuilder = builder_cls(
> TypeError: 'NoneType' object is not callable

and I have absolutely no idea why the second and third machines are producing different tracebacks. I have previously run these exact scripts successfully on the login and compute nodes of the supercomputer, this issue I'm raising has appeared fairly recently for me. This, is where I encounter the TypeError that I opened this issue with, which I was able to traceback (using my laptop before it too started not working) to whatever was dynamically importing "builder_cls". That bit of code wasn't doing importing builder_cls correctly and would effectively make the assignment "builder_cls=None" resulting in the TypeError.  Does any of this help?

I'm back on linux machine 1 (login node) now. After submitting that as a job to machine 2 and it failing with TypeError, linux machine 1 now produces identical traceback to machine 2:

> (arkroyal) [yr3g17@cyan52 squad_qanswering]$ python
> Python 3.10.8 (main, Nov 24 2022, 14:13:03) [GCC 11.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>
> from datasets import load_dataset
> load_dataset("squad")
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/datasets/load.py", line 1759, in load_dataset
>     builder_instance = load_dataset_builder(
>   File "/home/yr3g17/.conda/envs/arkroyal/lib/python3.10/site-packages/datasets/load.py", line 1522, in load_dataset_builder
>     builder_instance: DatasetBuilder = builder_cls(
> TypeError: 'NoneType' object is not callable

I thought it might be useful to provide you with my cache file structure:

>_home_yr3g17_.cache_huggingface_datasets_casino_default_1.1.0_302c3b1ac78c48091deabe83a11f4003c7b472a4e11a8eb92799653785bd5da1.lock
>_home_yr3g17_.cache_huggingface_datasets_imdb_plain_text_1.0.0_2fdd8b9bcadd6e7055e742a706876ba43f19faee861df134affd7a3f60fc38a1.lock
>_home_yr3g17_.cache_huggingface_datasets_squad_plain_text_1.0.0_d6ec3ceb99ca480ce37cdd35555d6cb2511d223b9150cce08a837ef62ffea453.lock
>_home_yr3g17_.cache_huggingface_datasets_yelp_review_full_yelp_review_full_1.0.0_e8e18e19d7be9e75642fc66b198abadb116f73599ec89a69ba5dd8d1e57ba0bf.lock
> casino
> downloads
> imdb
> json
> squad
> squad_v2
> yelp_review_full

The inside of squad/plain_text/1.0.0/ looks like

> d6ec3ceb99ca480ce37cdd35555d6cb2511d223b9150cce08a837ef62ffea453
> d6ec3ceb99ca480ce37cdd35555d6cb2511d223b9150cce08a837ef62ffea453.incomplete_info.lock
> d6ec3ceb99ca480ce37cdd35555d6cb2511d223b9150cce08a837ef62ffea453_builder.lock


I see this is quite a complex use case...

Let's try multiple things:
- First, update `datasets` and make sure you use the same version in all machines, so that we can easily compare different behaviors.
  ```
  pip install -U datasets
  ```
- Second, wherever you run the `load_dataset("squad")` command, make sure there is not a local directory named "squad". The datasets library gives priority to any local file/directory over the datasets on the Hugging Face Hub
  - I tell you this, because in one of your trace backs, it seems it refers to a local directory:
    ```
    Downloading and preparing dataset json/squad to /home/yr3g17/.cache/huggingface/datasets/json/squad-d733af945be1d2c2/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51...
    ```
- Third, to use the "squad" dataset from the Hub, you need to have internet connection, so that you can download the "squad" Python loading script from the Hub. Do all your machines have internet connection?
  - I ask this because of this error message:
    ```
    Using the latest cached version of the module from /home/yr3g17/.cache/huggingface/modules/datasets_modules/datasets/squad/8730650fed465361f38ac4d810ccdd16e8fc87b56498e52fb7e2cadaefc1f177 (last modified on Tue Feb 14 10:12:56 2023) since it couldn't be found locally at squad., or remotely on the Hugging Face Hub.
    ```
- Fourth, to be sure that we avoid any issues with the cache, it is a good idea to remove it and regenerate it. Remove `.cache/huggingface/datasets` and also `.cache/huggingface/modules`
- Fifth, as an additional debugging tool, let's be sure we use the latest "squad" Python loading script by passing the revision parameter:
  ```
  ds = load_dataset("squad", revision="5fe18c4c680f9922d794e3f4dd673a751c74ee37")
  ```

Additionally, we just had an infrastructure issue on the Hugging Face Hub at around 11:30 today. That might have contributed to the connectivity issue... It is fixed now.

https://status.huggingface.co/

Hi again, thanks for your help and insight Albert Villanova.

It's all working now, so thank you for that. For the benefit of anyone else who ends up in this thread, I solved the problem by addressing Albert's advice:

(1) Both Windows and Linux machine 1 (have internet access) and can now access the SQuAD dataset. The supercomputer login node can only access version 2.7.1, but my Windows laptop is running on datasets 2.11.0 just fine. I suspect it was just a perfect storm alongside the aforementioned "infrastructure issue".

(2) I did have a local directory called squad, because I was using a local copy of evaluate's "SQuAD" metric. The supercomputer compute nodes do not have internet access and treat `metric = evaluate.load('<x>')` as a way of loading a metric at the local path `./<x>/<x>.py`, which could've been a related issue as I was storing the metric under `squad/squad.py`. Don't be lazy like me and store the evaluation code under a path with a name that can be misinterpreted.

(3) I can't give internet access to the supercomputer compute nodes, so local files do just fine here.

(4) The windows and Linux machine 1 can both access the internet and were getting fresh copies of the dataset from the huggingface hub. Linux machine 2 was working after I cleared the contents of ~/.cache/huggingface/....

I feel silly now, knowing it was all so simple! Sorry about that Albert, and thanks again for the help. I've not raised a Github issue like this before, so I'm not sure if I should be close my own issues or if this is something you guys do?

Thanks for your detailed feedback which for sure will be useful to other community members.