Title: load_dataset hang on file_lock
Author: tomacai (NONE)
State: closed
Created: 2021-01-01T10:25:07Z

I am trying to load the squad dataset. Fails on Windows 10 but succeeds in Colab.
Transformers:  3.3.1
Datasets:  1.0.2
Windows 10 (also tested in WSL)

```
datasets.logging.set_verbosity_debug()
datasets.
train_dataset = load_dataset('squad', split='train')
valid_dataset = load_dataset('squad', split='validation')

train_dataset.features
```

```
https://raw.githubusercontent.com/huggingface/datasets/1.0.2/datasets/squad/squad.py not found in cache or force_download set to True, downloading to C:\Users\simpl\.cache\huggingface\datasets\tmpzj_o_6u7
Downloading:
5.24k/? [00:00<00:00, 134kB/s]
storing https://raw.githubusercontent.com/huggingface/datasets/1.0.2/datasets/squad/squad.py in cache at C:\Users\simpl\.cache\huggingface\datasets\f6877c8d2e01e8fcb60dc101be28b54a7522feac756deb9ac5c39c6d8ebef1ce.85f43de978b9b25921cb78d7a2f2b350c04acdbaedb9ecb5f7101cd7c0950e68.py
creating metadata file for C:\Users\simpl\.cache\huggingface\datasets\f6877c8d2e01e8fcb60dc101be28b54a7522feac756deb9ac5c39c6d8ebef1ce.85f43de978b9b25921cb78d7a2f2b350c04acdbaedb9ecb5f7101cd7c0950e68.py

Checking C:\Users\simpl\.cache\huggingface\datasets\f6877c8d2e01e8fcb60dc101be28b54a7522feac756deb9ac5c39c6d8ebef1ce.85f43de978b9b25921cb78d7a2f2b350c04acdbaedb9ecb5f7101cd7c0950e68.py for additional imports.
Found main folder for dataset https://raw.githubusercontent.com/huggingface/datasets/1.0.2/datasets/squad/squad.py at C:\Users\simpl\.cache\huggingface\modules\datasets_modules\datasets\squad
Found specific version folder for dataset https://raw.githubusercontent.com/huggingface/datasets/1.0.2/datasets/squad/squad.py at C:\Users\simpl\.cache\huggingface\modules\datasets_modules\datasets\squad\1244d044b266a5e4dbd4174d23cb995eead372fbca31a03edc3f8a132787af41
Found script file from https://raw.githubusercontent.com/huggingface/datasets/1.0.2/datasets/squad/squad.py to C:\Users\simpl\.cache\huggingface\modules\datasets_modules\datasets\squad\1244d044b266a5e4dbd4174d23cb995eead372fbca31a03edc3f8a132787af41\squad.py
Couldn't find dataset infos file at https://raw.githubusercontent.com/huggingface/datasets/1.0.2/datasets/squad\dataset_infos.json
Found metadata file for dataset https://raw.githubusercontent.com/huggingface/datasets/1.0.2/datasets/squad/squad.py at C:\Users\simpl\.cache\huggingface\modules\datasets_modules\datasets\squad\1244d044b266a5e4dbd4174d23cb995eead372fbca31a03edc3f8a132787af41\squad.json
No config specified, defaulting to first: squad/plain_text
```

Interrupting the jupyter kernel we are in a file lock.

In Google Colab the download is ok. In contrast to a local run in colab dataset_infos.json is downloaded
```
https://raw.githubusercontent.com/huggingface/datasets/1.0.2/datasets/squad/dataset_infos.json not found in cache or force_download set to True, downloading to /root/.cache/huggingface/datasets/tmptl9ha_ad

Downloading:
2.19k/? [00:00<00:00, 26.2kB/s]
```

Comments:
Can you try to upgrade to a more recent version of datasets?

Thank, upgrading to 1.1.3 resolved the issue.

Having the same issue with `datasets 1.1.3` of `1.5.0` (both tracebacks look the same) and `kilt_wikipedia`, Ubuntu 20.04

```py
In [1]: from datasets import load_dataset                                                                                                                                                                          

In [2]: wikipedia = load_dataset('kilt_wikipedia')['full']                                                                                                                   
Downloading: 7.37kB [00:00, 2.74MB/s]                                                                                                                                                                              
Downloading: 3.33kB [00:00, 1.44MB/s]                                                                                                                                                                              
^C---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~/anaconda3/envs/transformers2/lib/python3.7/site-packages/datasets/utils/filelock.py in _acquire(self)
    380         try:
--> 381             fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
    382         except (IOError, OSError):

OSError: [Errno 37] No locks available

During handling of the above exception, another exception occurred:

KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-2-f412d3d46ec9> in <module>
----> 1 wikipedia = load_dataset('kilt_wikipedia')['full']

~/anaconda3/envs/transformers2/lib/python3.7/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, sav
e_infos, script_version, **config_kwargs)
    601         hash=hash,
    602         features=features,
--> 603         **config_kwargs,
    604     )
    605 

~/anaconda3/envs/transformers2/lib/python3.7/site-packages/datasets/builder.py in __init__(self, *args, **kwargs)
    841     def __init__(self, *args, **kwargs):
    842         self._writer_batch_size = kwargs.pop("writer_batch_size", self._writer_batch_size)
--> 843         super(GeneratorBasedBuilder, self).__init__(*args, **kwargs)
    844 
    845     @abc.abstractmethod

~/anaconda3/envs/transformers2/lib/python3.7/site-packages/datasets/builder.py in __init__(self, cache_dir, name, hash, features, **config_kwargs)
    174             os.makedirs(self._cache_dir_root, exist_ok=True)
    175         lock_path = os.path.join(self._cache_dir_root, self._cache_dir.replace(os.sep, "_") + ".lock")
--> 176         with FileLock(lock_path):
    177             if os.path.exists(self._cache_dir):  # check if data exist
    178                 if len(os.listdir(self._cache_dir)) > 0:

~/anaconda3/envs/transformers2/lib/python3.7/site-packages/datasets/utils/filelock.py in __enter__(self)
    312 
    313     def __enter__(self):
--> 314         self.acquire()
    315         return self
    316 

~/anaconda3/envs/transformers2/lib/python3.7/site-packages/datasets/utils/filelock.py in acquire(self, timeout, poll_intervall)
    261                     if not self.is_locked:
    262                         logger().debug("Attempting to acquire lock %s on %s", lock_id, lock_filename)
--> 263                         self._acquire()
    264 
    265                 if self.is_locked:

~/anaconda3/envs/transformers2/lib/python3.7/site-packages/datasets/utils/filelock.py in _acquire(self)
    379 
    380         try:
--> 381             fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
    382         except (IOError, OSError):
    383             os.close(fd)

KeyboardInterrupt: 

```