GitHub issue #6006: NotADirectoryError when loading gigawords
State: closed | Created: 2023-07-05T06:23:41Z

### Describe the bug

got `NotADirectoryError` whtn loading gigawords dataset

### Steps to reproduce the bug

When running
```
import datasets
datasets.load_dataset('gigaword')
```

Got the following exception:
```bash
Traceback (most recent call last):                                                                                                    [0/1862]
  File "/home/x/.conda/envs/dataproc/lib/python3.8/site-packages/datasets/builder.py", line 1629, in _prepare_split_single          
    for key, record in generator:                  
  File "/home/x/.cache/huggingface/modules/datasets_modules/datasets/gigaword/ea83a8b819190acac5f2dae011fad51dccf269a0604ec5dd24795b
64efb424b6/gigaword.py", line 115, in _generate_examples            
    with open(src_path, encoding="utf-8") as f_d, open(tgt_path, encoding="utf-8") as f_s:
  File "/home/x/.conda/envs/dataproc/lib/python3.8/site-packages/datasets/streaming.py", line 71, in wrapper
    return function(*args, use_auth_token=use_auth_token, **kwargs)
  File "/home/x/.conda/envs/dataproc/lib/python3.8/site-packages/datasets/download/streaming_download_manager.py", line 493, in xope
n                                     
    return open(main_hop, mode, *args, **kwargs)                                     
NotADirectoryError: [Errno 20] Not a directory: '/home/x/.cache/huggingface/datasets/downloads/6da52431bb5124d90cf51a0187d2dbee9046e
89780c4be7599794a4f559048ec/org_data/train.src.txt'
                                              
The above exception was the direct cause of the following exception:
                                                                                                                        
Traceback (most recent call last):        
  File "gigaword.py", line 38, in <module>                                                                                        
    main()                     
  File "gigaword.py", line 35, in main                                                                                              
    train, dev, test = dataset.generate_k_shot_data(k=32, seed=seed, path="../data/")
  File "/home/x/MICL/preprocess/fewshot_gym_dataset.py", line 199, in generate_k_shot_data                                
    dataset = self.load_dataset()                               
  File "gigaword.py", line 29, in load_dataset                                                                               
    return datasets.load_dataset('gigaword')                
  File "/home/x/.conda/envs/dataproc/lib/python3.8/site-packages/datasets/load.py", line 1809, in load_dataset            
    builder_instance.download_and_prepare(                                               
  File "/home/x/.conda/envs/dataproc/lib/python3.8/site-packages/datasets/builder.py", line 909, in download_and_prepare
    self._download_and_prepare(                                                                                                               
  File "/home/x/.conda/envs/dataproc/lib/python3.8/site-packages/datasets/builder.py", line 1670, in _download_and_prepare
    super()._download_and_prepare(                                                                                  
  File "/home/x/.conda/envs/dataproc/lib/python3.8/site-packages/datasets/builder.py", line 1004, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)                                                                              
  File "/home/x/.conda/envs/dataproc/lib/python3.8/site-packages/datasets/builder.py", line 1508, in _prepare_split
    for job_id, done, content in self._prepare_split_single(                                                
  File "/home/x/.conda/envs/dataproc/lib/python3.8/site-packages/datasets/builder.py", line 1665, in _prepare_split_single          
    raise DatasetGenerationError("An error occurred while generating the dataset") from e
datasets.builder.DatasetGenerationError: An error occurred while generating the dataset
```


### Expected behavior

Download and process the dataset successfully

### Environment info

- `datasets` version: 2.13.1
- Platform: Linux-5.0.0-1032-azure-x86_64-with-glibc2.10
- Python version: 3.8.0
- Huggingface_hub version: 0.15.1
- PyArrow version: 12.0.1
- Pandas version: 2.0.3