OSError with a custom dataset loading script
Opened by suzanab (closed)

## Describe the bug
I am getting an OS error when trying to load the newly uploaded dataset classla/janes_tag. What puzzles me is that I have already uploaded a very similar dataset - classla/reldi_hr - with no issues. The loading scripts for the two datasets are almost identical and they have the same directory structure, yet I am only getting an error with janes_tag.

## Steps to reproduce the bug
```python
dataset = datasets.load_dataset('classla/janes_tag', split='validation')
```

## Expected results
Dataset correctly loaded.

## Actual results

Traceback (most recent call last):
  File "C:/mypath/test.py", line 91, in <module>
    load_and_print('janes_tag')
  File "C:/mypath/test.py", line 32, in load_and_print
    dataset = datasets.load_dataset('classla/{}'.format(ds_name), split='validation')
  File "C:\mypath\venv\lib\site-packages\datasets\load.py", line 1632, in load_dataset
    use_auth_token=use_auth_token,
  File "C:\mypath\venv\lib\site-packages\datasets\builder.py", line 608, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "C:\mypath\venv\lib\site-packages\datasets\builder.py", line 704, in _download_and_prepare
    ) from None
OSError: Cannot find data file. 
Original error:
[Errno 2] No such file or directory: 'C:\\mypath\\.cache\\huggingface\\datasets\\downloads\\2c9996e44bdc5af9c89bffb9e6d7a3e42fdb2f56bacab45de13b20f3032ea7ca\\data\\train_all.conllup'

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.14.0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.7.5
- PyArrow version: 3.0.0


Comments:
Hi,

there is a difference in how the `data_dir` is zipped between the `classla/janes_tag` and the `classla/reldi_hr` dataset. After unzipping, for the former, the data files (`*.conllup`) are in the root directory (root -> data files), and for the latter, they are inside the `data` directory (root -> `data` -> data files).

This can be fixed by removing the `os.path.join` call in https://huggingface.co/datasets/classla/janes_tag/blob/main/janes_tag.py#L86

Let me know if this works for you.

Hi Mario,

I had already tried that before, but it didn't work. I have now recreated the `classla/janes_tag`  zip file so that it also contains the `data` directory, but I am still getting the same error.

Hi,

I just tried to download the `classla/janes_tag` dataset, and this time the zip file is extracted correctly. However, the script is now throwing the IndexError, probably due to a bug in the `_generate_examples`.

Let me know if you are still getting the same error.

I am still getting the same error.

Hi, 

could you try to download the dataset with a different `cache_dir` like so:
```python
import datasets
dataset = datasets.load_dataset('classla/janes_tag', split='validation', cache_dir="path/to/different/cache/dir")
```
If this works, then most likely the cached extracted data is causing issues. This data is stored at `~/.cache/huggingface/datasets/downloads/extracted` and needs to be deleted, and then it should work (you can easily locate the directory with the path given in the `OSError` message). Additionally, I'd suggest you to update `datasets` to the newest version with:
```
pip install -U datasets
```

Thank you, deleting the `~/.cache/huggingface/datasets/downloads/extracted` directory helped. However, I am still having problems.

There was indeed a bug in the script that was throwing an `IndexError`, which I have now corrected (added the condition to skip the lines starting with '# text') and it is working locally, but still throws an error when I try to load the dataset from HuggingFace. I literally copied and pasted the `_generate_examples` function and ran it on the `dev_all.conllup` file, which I even re-downloaded from the repository to be certain that the files are exactly the same. I also deleted everything again just in case, but it didn't help. The code works locally, but throws an `IndexError` when loading from `datasets.`

Hi,

Did some investigation.

To fix the dataset script on the Hub, append the following labels to the `names` list of the `upos_tags` field:
```'INTJ NOUN', 'AUX PRON', 'PART ADV', 'PRON ADP', 'INTJ INTJ', 'VERB NOUN', 'NOUN AUX'```.

This step is required to avoid an error due to missing labels in the following step which is:
```python
load_dataset("classla/janes_tag", split="validation", download_mode="force_redownload")
```
This will generate and cache the dataset, so specifying `download_mode` will not be required anymore unless you update the script/data on the Hub.

It works now, thank you!