Unable to dowload big_patent (FileNotFoundError)
State: closed | Author: slvcsl (NONE)
Labels: name: bug

name: duplicate

## Describe the bug
I am trying to download some splits of the big_patent dataset, using the following code:

`ds = load_dataset("big_patent", "g", split="validation", download_mode="force_redownload")
`
However, this leads to a FileNotFoundError.

FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-3-8d8a745706a9>](https://localhost:8080/#) in <module>()
      1 from datasets import load_dataset
----> 2 ds = load_dataset("big_patent", "g", split="validation", download_mode="force_redownload")

8 frames
[/usr/local/lib/python3.7/dist-packages/datasets/load.py](https://localhost:8080/#) in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, script_version, **config_kwargs)
   1705         ignore_verifications=ignore_verifications,
   1706         try_from_hf_gcs=try_from_hf_gcs,
-> 1707         use_auth_token=use_auth_token,
   1708     )
   1709 

[/usr/local/lib/python3.7/dist-packages/datasets/builder.py](https://localhost:8080/#) in download_and_prepare(self, download_config, download_mode, ignore_verifications, try_from_hf_gcs, dl_manager, base_path, use_auth_token, **download_and_prepare_kwargs)
    593                     if not downloaded_from_gcs:
    594                         self._download_and_prepare(
--> 595                             dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
    596                         )
    597                     # Sync info

[/usr/local/lib/python3.7/dist-packages/datasets/builder.py](https://localhost:8080/#) in _download_and_prepare(self, dl_manager, verify_infos, **prepare_split_kwargs)
    659         split_dict = SplitDict(dataset_name=self.name)
    660         split_generators_kwargs = self._make_split_generators_kwargs(prepare_split_kwargs)
--> 661         split_generators = self._split_generators(dl_manager, **split_generators_kwargs)
    662 
    663         # Checksums verification

[/root/.cache/huggingface/modules/datasets_modules/datasets/big_patent/bdefa7c0b39fba8bba1c6331b70b738e30d63c8ad4567f983ce315a5fef6131c/big_patent.py](https://localhost:8080/#) in _split_generators(self, dl_manager)
    123         split_types = ["train", "val", "test"]
    124         extract_paths = dl_manager.extract(
--> 125             {k: os.path.join(dl_path, "bigPatentData", k + ".tar.gz") for k in split_types}
    126         )
    127         extract_paths = {k: os.path.join(extract_paths[k], k) for k in split_types}

[/usr/local/lib/python3.7/dist-packages/datasets/utils/download_manager.py](https://localhost:8080/#) in extract(self, path_or_paths, num_proc)
    282         download_config.extract_compressed_file = True
    283         extracted_paths = map_nested(
--> 284             partial(cached_path, download_config=download_config), path_or_paths, num_proc=num_proc, disable_tqdm=False
    285         )
    286         path_or_paths = NestedDataStructure(path_or_paths)

[/usr/local/lib/python3.7/dist-packages/datasets/utils/py_utils.py](https://localhost:8080/#) in map_nested(function, data_struct, dict_only, map_list, map_tuple, map_numpy, num_proc, types, disable_tqdm)
    260         mapped = [
    261             _single_map_nested((function, obj, types, None, True))
--> 262             for obj in utils.tqdm(iterable, disable=disable_tqdm)
    263         ]
    264     else:

[/usr/local/lib/python3.7/dist-packages/datasets/utils/py_utils.py](https://localhost:8080/#) in <listcomp>(.0)
    260         mapped = [
    261             _single_map_nested((function, obj, types, None, True))
--> 262             for obj in utils.tqdm(iterable, disable=disable_tqdm)
    263         ]
    264     else:

[/usr/local/lib/python3.7/dist-packages/datasets/utils/py_utils.py](https://localhost:8080/#) in _single_map_nested(args)
    194     # Singleton first to spare some computation
    195     if not isinstance(data_struct, dict) and not isinstance(data_struct, types):
--> 196         return function(data_struct)
    197 
    198     # Reduce logging to keep things readable in multiprocessing with tqdm

[/usr/local/lib/python3.7/dist-packages/datasets/utils/file_utils.py](https://localhost:8080/#) in cached_path(url_or_filename, download_config, **download_kwargs)
    314     elif is_local_path(url_or_filename):
    315         # File, but it doesn't exist.
--> 316         raise FileNotFoundError(f"Local file {url_or_filename} doesn't exist")
    317     else:
    318         # Something unknown

FileNotFoundError: Local file /root/.cache/huggingface/datasets/downloads/extracted/ad068abb3e11f9f2f5440b62e37eb2b03ee515df9de1637c55cd1793b68668b2/bigPatentData/train.tar.gz doesn't exist

I have tried this in a number of machines, including on Colab, so I think this is not environment dependent.

How do I load the bigPatent dataset?

Comments: Hi @slvcsl, thanks for reporting.

Yesterday we just made a patch release of our `datasets` library that fixes this issue: version 1.18.4.
https://pypi.org/project/datasets/#history

Please, feel free to update `datasets` library to the latest version: 
```shell
pip install -U datasets
```
And then you should force redownload of the data file to update your local cache: 
```python
ds = load_dataset("big_patent", "g", split="validation", download_mode="force_redownload")
```
- Note that before the fix, you just downloaded and cached the Google Drive virus scan warning page, instead of the data file

This issue was already reported 
- #3784

and its root cause is a change in the Google Drive service. See:
- #3786 

We already fixed it. See:
- #3787