SergeiGKS/huggingface-dataset-issues huggingface dataset:
Issue: Dataset has incorrect labels

Comment:
Hi @marshmellow77, thanks for reporting.

That issue is fixed since `datasets` version 1.9.0 (see 16bc665f2753677c765011ef79c84e55486d4347).

Please, update `datasets` with: `pip install -U datasets`
Issue: Dataset has incorrect labels

Comment:
Thanks. Please note that the dataset explorer (https://huggingface.co/datasets/viewer/?dataset=turkish_product_reviews) still shows the incorrect state. The sentiment for the first few customer reviews is actually negative and should be labelled with "0", see screenshot:

![Capture1](https://user-images.githubusercontent.com/63367770/135637150-93d9b09b-f1dd-4701-97a5-5cb2672ec0c7.PNG)
Issue: IndexError: Invalid key: 14 is out of bounds for size 0

Comment:
Hi ! Could you check the length of the `self.dataset` object (i.e. the Dataset object passed to the data loader) ? It looks like the dataset is empty.
Not sure why the SWA optimizer would cause this though.
Issue: IndexError: Invalid key: 14 is out of bounds for size 0

Comment:
Any updates on this? 
The same error occurred to me too when running `cardiffnlp/twitter-roberta-base-sentiment` on a custom dataset. This happened when I tried to do `model = torch.nn.DataParallel(model, device_ids=[0, 1, 2, 3])` without using sagemaker distribution. 
Python: 3.6.13
datasets: 1.6.2
Issue: IndexError: Invalid key: 14 is out of bounds for size 0

Comment:
Hi @ruisi-su, do you have this issue while using SWA as well, or just data parallel ?

If you have a code example to reproduce this issue it would also be helpful
Issue: IndexError: Invalid key: 14 is out of bounds for size 0

Comment:
@lhoestq I had this issue without SWA. I followed [this](https://github.com/huggingface/notebooks/blob/master/sagemaker/03_distributed_training_data_parallelism/sagemaker-notebook.ipynb) notebook to utilize multiple gpus on the [roberta-base](https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment) model. This tutorial could only work if I am on `ml.p3.16xlarge`, which I don't have access to. So I tried using just `model = torch.nn.DataParallel(model, device_ids=[0, 1, 2, 3]` before calling `trainer.fit()`. But maybe this is not the right way to do distributed training. I can provide a code example if that will be more helpful.
Issue: IndexError: Invalid key: 14 is out of bounds for size 0

Comment:
It might be an issue with old versions of `datasets`, can you try updating `datasets` ?
Issue: IndexError: Invalid key: 14 is out of bounds for size 0

Comment:
FYI I encountered the exact same error using the latest versions of `datasets`, `transformers` and `pyarrow`, without using any kind of SWA or dataparallel: 

```
# packages in environment at C:\Users\zhang\mambaforge:
#
# Name                    Version                   Build  Channel
cudatoolkit               11.0.3               h3f58a73_9    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
datasets                  1.17.0                   pypi_0    pypi
pyarrow                   6.0.1                    pypi_0    pypi
pytorch                   1.7.1           py3.9_cuda110_cudnn8_0    pytorch
tornado                   6.1              py39hb82d6ee_2    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
```

```
> python --version
> 3.9.7
```
Issue: IndexError: Invalid key: 14 is out of bounds for size 0

Comment:
Same error here! Datasets version `1.18.3` freshly updated.

`IndexError: Invalid key: 90 is out of bounds for size 0`

My task is finetuning the model for token classification.

**Solved**: I make a mistake while updating the dataset during the map, you should check that you return the correct values.
Issue: IndexError: Invalid key: 14 is out of bounds for size 0

Comment:
cc @sgugger This probably comes from the `Trainer` removing all the columns of a dataset, do you think we can improve the error message in this case ?
Issue: IndexError: Invalid key: 14 is out of bounds for size 0

Comment:
The `Trainer` clearly logs when it removes columns in the dataset. I'm not too sure of where the bug appears as I haven't seen a clear reproducer. Happy to display a more helpful error message, but I'd need a reproducer to see what the exact problem is to design the right test and warning :-)
Issue: IndexError: Invalid key: 14 is out of bounds for size 0

Comment:
Well, if I can try to suggest how to reproduce, please try by do not returning any updated content in the map function used to tokenize input (e.g., in TokenClassification). I can leave here my wrong version for reference:

```python
def preprocess_function(examples):

    text = examples["text"]
    
    inputs = tokenizer(
        text,
        max_length=512,
        truncation="only_second",
        return_offsets_mapping=True,
        padding="max_length",
    )

    offset_mapping = inputs.pop("offset_mapping")
    # ... processing code

    inputs["labels"] = label_ids
    #return inputs
    
train_ds = train_ds.map(preprocess_function, batched=False)
test_ds = test_ds.map(preprocess_function, batched=False)
eval_ds = eval_ds.map(preprocess_function, batched=False)
```

Of course, returning inputs solved the problem. As suggestion, a possible error message could display "IndexError: the `key` required by trainer are not found in the dataset" (just an hypothesis, I think there could be something better). 

Please tell me if you need more details to reproduce, glad to help!
Issue: IndexError: Invalid key: 14 is out of bounds for size 0

Comment:
That's the thing though. The `Trainer` has no idea which inputs are required or not since all models can have different kinds of inputs, and it can work for models outside of the Transformers library. I can add a clear error message if I get an empty batch, as this is easy to detect, but that's pretty much it.
Issue: ArrowInvalid: Can only convert 1-dimensional array values

Comment:
Hi @NielsRogge, thanks for reporting!

In `datasets`, we were handling N-dimensional arrays only when passed as an instance of `np.array`, not when passed as a list of `np.array`s.

I'm fixing it.
Issue: Exceeded maximum rows when reading large files

Comment:
Hi @zijwang, thanks for reporting this issue.

You did not mention which `datasets` version you are using, but looking at the code in the stack trace, it seems you are using an old version.

Could you please update `datasets` (`pip install -U datasets`) and check if the problem persists?
Issue: OpenSLR 25: ASR data for Amharic, Swahili and Wolof

Comment:
Whoever handles this just needs to: 

- [ ] fork the HuggingFace Datasets repo
- [ ] update the [existing dataset script](https://github.com/huggingface/datasets/blob/master/datasets/openslr/openslr.py) to add SLR25. Lots of copypasting from other sections of the script should make that easy. 
Amharic URL: https://www.openslr.org/resources/25/data_readspeech_am.tar.bz2. 
Swahili URL: https://www.openslr.org/resources/25/data_broadcastnews_sw.tar.bz2, 
Wolof URL: https://www.openslr.org/resources/25/data_readspeech_wo.tar.bz2
- [ ] update the [data card](https://github.com/huggingface/datasets/blob/master/datasets/openslr/README.md) to include information about SLR25. There's lots of other examples to draw from. 
- [ ] add the appropriate language tags to the data card as well. https://www.w3.org/International/questions/qa-choosing-language-tags, or just use `sw`, `am`, and `wo` for consistency. 
- [ ] make a pull request to merge your changes back into HuggingFace's repo
Issue: OpenSLR 25: ASR data for Amharic, Swahili and Wolof

Comment:
... also the example in "use in datasets library" should be updated. It currently says 
![image](https://user-images.githubusercontent.com/4109253/135115980-8583a44a-cae6-4121-b699-00667020849f.png)
But you actually have to specify a subset, e.g. 
```python
dataset = load_dataset("openslr", "SLR32")
```
Issue: Run CI tests against non-production server

Comment:
Hey @albertvillanova could you provide more context, including extracts from the discussion we had ?

Let's ping @Pierrci @julien-c and @n1t0 for their opinion about that
Issue: Run CI tests against non-production server

Comment:
@julien-c increased the huggingface.co production workers in order to see if it solve [the 502 you had this morning](https://app.circleci.com/pipelines/github/huggingface/datasets/7843/workflows/fc83fa32-18f5-4dc3-9e2f-ba277ae1af74)

For the decision process: be aware that moon-staging does not have persistent repos (they are deleted regularly). as a consequence, **if the moon-staging solution is validated**, you should consider a way to keep the repository that are loaded in tests. These are the ones I found : https://github.com/huggingface/datasets/blob/d488db2f64f312f88f72bbc57a09b7eddb329182/tests/test_load.py and https://github.com/huggingface/datasets/blob/40773111c3e7db8a992fa1c48af32d900a1018d6/tests/test_streaming_download_manager.
Issue: Impossible to load compressed csv

Comment:
Hi @Valahaar, thanks for reporting and for your investigation about the source cause.

You are right and that commit prevents `pandas` from inferring the compression. On the other hand, @lhoestq did that change to support loading that dataset in streaming mode. 

I'm fixing it.
Issue: Can't load dataset

Comment:
Hi @mskovalova, 

Some datasets have multiple configurations. Therefore, in order to load them, you have to specify both the *dataset name* and the *configuration name*.

In the error message you got, you have a usage example:
- To load the 'wikitext-103-raw-v1' configuration of the 'wikitext' dataset, you should use: 
  ```python
  load_dataset('wikitext', 'wikitext-103-raw-v1')
  ```

In your case, if you would like to load the 'wikitext-2-v1' configuration of the 'wikitext' dataset, please use:
```python
raw_datasets = load_dataset("wikitext", "wikitext-2-v1")
```
Issue: Can't load dataset

Comment:
Hi, if I want to load the dataset from local file, then how to specify the configuration name?
Issue: OSError: Not enough disk space.

Comment:
Maybe we can change the disk space calculating API from `shutil.disk_usage` to  `os.statvfs` in UNIX-like system, which can provide correct results.
```
statvfs = os.statvfs('path')
avail_space_bytes = statvfs.f_frsize * statvfs.f_bavail
```
Issue: OSError: Not enough disk space.

Comment:
`DownloadConfig` only sets the location to download the files. On the other hand, `cache_dir` sets the location for both downloading and caching the data. You can find more information here: https://huggingface.co/docs/datasets/loading_datasets.html#cache-directory
Issue: OSError: Not enough disk space.

Comment:
I had encountered the same error when running a command `ds = load_dataset('food101')` in a docker container. The error I got: `OSError: Not enough disk space. Needed: 9.43 GiB (download: 4.65 GiB, generated: 4.77 GiB, post-processed: Unknown size)`

In case anyone encountered the same issue, this was my fix:

```sh
# starting the container (mount project directory onto /app, so that the code and data in my project directory are available in the container)
docker run -it --rm -v $(pwd):/app my-demo:latest bash
```


```python
# other code ...
ds = load_dataset('food101', cache_dir="/app/data") # set cache_dir to the absolute path of a directory (e.g. /app/data) that's mounted from the host (MacOS in my case) into the docker container

# this assumes ./data directory exists in your project folder. If not, create it or point it to any other existing directory where you want to store the cache
```

Thanks @albertvillanova for posting the fix above :-)
Issue: medical-dialog error

Comment:
Hi @smeyerhot, thanks for reporting.

You are right: there is an issue with the dataset metadata. I'm fixing it.

In the meantime, you can circumvent the issue by passing `ignore_verifications=True`:
```python
raw_datasets = load_dataset("medical_dialog", "en", split="train", download_mode="force_redownload", data_dir="./Medical-Dialogue-Dataset-English", ignore_verifications=True)
```
Issue: `DatasetDict` cannot be exported to parquet if the splits have different features

Comment:
This is because you have to specify which split corresponds to what file:
```python
data_files = {"train": "train/split.parquet", "validation": "validation/split.parquet"}
brand_new_dataset_2 = load_dataset("ds", data_files=data_files)
```

Otherwise it tries to concatenate the two splits, and it fails because they don't have the same features.

It works with save_to_disk/load_from_disk because it also stores json files that contain the information about which files goes into which split
Issue: `DatasetDict` cannot be exported to parquet if the splits have different features

Comment:
I may be mistaken but I think the following doesn't work either:

```python
from datasets import load_dataset

ds = load_dataset("lhoestq/custom_squad")


def identical_answers(e):
    e['identical_answers'] = len(set(e['answers']['text'])) == 1
    return e


ds['validation'] = ds['validation'].map(identical_answers)
ds['train'].to_parquet("./ds/train/split.parquet")
ds['validation'].to_parquet("./ds/validation/split.parquet")

data_files = {"train": "train/split.parquet", "validation": "validation/split.parquet"}
brand_new_dataset_2 = load_dataset("ds", data_files=data_files)
```
Issue: `DatasetDict` cannot be exported to parquet if the splits have different features

Comment:
It works on my side as soon as the directories named `ds/train` and `ds/validation` exist (otherwise it returns a FileNotFoundError). What error are you getting ?
Issue: `DatasetDict` cannot be exported to parquet if the splits have different features

Comment:
Also we may introduce a default mapping for the data files:
```python
{
    "train": ["*train*"],
    "test": ["*test*"],
    "validation": ["*dev*", "valid"],
}
```
this way if you name your files according to the splits you won't have to specify the data_files parameter. What do you think ?

I moved this discussion to #3027
Issue: `DatasetDict` cannot be exported to parquet if the splits have different features

Comment:
I just tried again on colab by installing `datasets` from source with pyarrow 3.0.0 and didn't get any error.

You error seems to happen when doing
```python
ds = load_dataset("lhoestq/custom_squad")
```

More specifically it fails when trying to read the arrow file that just got generated. I haven't issues like this before. Can you make sure you have a recent version of `pyarrow` ? Maybe it was an old version that wrote the arrow file and some header was missing.
Issue: `DatasetDict` cannot be exported to parquet if the splits have different features

Comment:
Thank you for your pointer! This seems to have been linked to Python 3.9.7: it works flawlessly with Python 3.8.6. This can be closed, thanks a lot for your help.
Issue: Error when calculating Matthews Correlation Coefficient loaded with `load_metric`

Comment:
After some more tests I've realized that this "issue" is due to the `numpy.float64` to `float` conversion, but when defining a function named `compute_metrics` as it follows:

```python
def compute_metrics(eval_preds):
    metric = load_metric("matthews_correlation")
    logits, labels = eval_preds
    predictions = np.argmax(logits, axis=1)
    return metric.compute(predictions=predictions, references=labels)
```

It fails when the evaluation metrics are computed in the `Trainer` with the same error code `AttributeError: 'float' object has no attribute 'item'` as the output is not a `numpy.float64`... Maybe I'm doing something wrong, not sure!
Issue: Error when calculating Matthews Correlation Coefficient loaded with `load_metric`

Comment:
Ok after some more experiments I've realized that it's an issue from my side, at first I thought it was due to `fp16=True` in `TrainingArguments`, but in the end that may not be the issue, so I'll close this for now and check later, since the mistake is on my side :weary: Sorry for the inconvenience!
Issue: MultiWOZ Dataset NonMatchingChecksumError

Comment:
Hi Brady! I met the similar issue, it stuck in the downloading stage instead of download anything, maybe it is broken. After I change the downloading from URLs to one url of the [Multiwoz project](https://github.com/budzianowski/multiwoz/archive/44f0f8479f11721831c5591b839ad78827da197b.zip) and use dirs to get separate files, the problems gone.
Issue: Trying to get in touch regarding a security issue

Comment:
Hi @JamieSlome,

Thanks for reaching out. Yes, you are right: I'm opening a PR to add the `SECURITY.md` file and a contact method.

In the meantime, please feel free to report the security issue to: feedback@huggingface.co
Issue: Protect master branch

Comment:
@lhoestq now the 2 are implemented.

Please note that for the the second protection, finally I have chosen to protect the master branch only from **merge commits** (see update comment above), so no need to disable/re-enable the protection on each release (direct commits, different from merge commits, can be pushed to the remote master branch; and eventually reverted without messing up the repo history).
Issue: Add  `remove_columns` to `IterableDataset ` 

Comment:
Hi ! Good idea :)
If you are interested in contributing, feel free to give it a try and open a Pull Request. Also let me know if I can help you with this or if you have questions
Issue: Backwards compatibility broken for cached datasets that use `.filter()`

Comment:
Hi ! I guess the caching mechanism should have considered the new `filter` to be different from the old one, and don't use cached results from the old `filter`.
To avoid other users from having this issue we could make the caching differentiate the two, what do you think ?
Issue: Backwards compatibility broken for cached datasets that use `.filter()`

Comment:
If it's easy enough to implement, then yes please 😄  But this issue can be low-priority, since I've only encountered it in a couple of `transformers` CI tests.
Issue: Backwards compatibility broken for cached datasets that use `.filter()`

Comment:
Well it can cause issue with anyone that updates `datasets` and re-run some code that uses filter, so I'm creating a PR
Issue: Backwards compatibility broken for cached datasets that use `.filter()`

Comment:
I just merged a fix, let me know if you're still having this kind of issues :)

We'll do a release soon to make this fix available
Issue: load_dataset using default cache on Windows causes PermissionError: [WinError 5] Access is denied

Comment:
Hi @daqieq, thanks for reporting.

Unfortunately, I was not able to reproduce this bug:
```ipython
In [1]: from datasets import load_dataset
   ...: ds = load_dataset('wiki_bio')
Downloading: 7.58kB [00:00, 26.3kB/s]
Downloading: 2.71kB [00:00, ?B/s]
Using custom data configuration default
Downloading and preparing dataset wiki_bio/default (download: 318.53 MiB, generated: 736.94 MiB, post-processed: Unknown size, total: 1.03 GiB) to C:\Users\username\.cache\huggingface\datasets\wiki_bio\default\
1.1.0\5293ce565954ba965dada626f1e79684e98172d950371d266bf3caaf87e911c9...
Downloading: 334MB [01:17, 4.32MB/s]
Dataset wiki_bio downloaded and prepared to C:\Users\username\.cache\huggingface\datasets\wiki_bio\default\1.1.0\5293ce565954ba965dada626f1e79684e98172d950371d266bf3caaf87e911c9. Subsequent calls will reuse thi
s data.
```

This kind of error messages usually happen because:
- Your running Python script hasn't write access to that directory
- You have another program (the File Explorer?) already browsing inside that directory
Issue: load_dataset using default cache on Windows causes PermissionError: [WinError 5] Access is denied

Comment:
Thanks @albertvillanova for looking at it! I tried on my personal Windows machine and it downloaded just fine.

Running on my work machine and on a colleague's machine it is consistently hitting this error. It's not a write access issue because the `.incomplete` directory is written just fine. It just won't rename and then it deletes the directory in the `finally` step. Also the zip file is written and extracted fine in the downloads directory.

That leaves another program that might be interfering, and there are plenty of those in my work machine ... (full antivirus, data loss prevention, etc.). So the question remains, why not extend the `try` block to allow catching the error and circle back to the rename after the unknown program is finished doing its 'stuff'. This is the approach that I read about in the linked repo (see my comments above).

If it's not high priority, that's fine. However, if someone were to write an PR that solved this issue in our environment in an `except` clause, would it be reviewed for inclusion in a future release? Just wondering whether I should spend any more time on this issue.
Issue: load_dataset using default cache on Windows causes PermissionError: [WinError 5] Access is denied

Comment:
Hi @albertvillanova, even I am facing the same issue on my work machine:

`Downloading and preparing dataset json/c4-en-html-with-metadata to C:\Users\......\.cache\huggingface\datasets\json\c4-en-html-with-metadata-4635c2fd9249f62d\0.0.0\c90812beea906fcffe0d5e3bb9eba909a80a998b5f88e9f8acbd320aa91acfde...
100%|███████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 983.42it/s]
100%|███████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 209.01it/s]
Traceback (most recent call last):
  File "bsmetadata/preprocessing_utils.py", line 710, in <module>
    ds = load_dataset(
  File "C:\Users\.......\AppData\Roaming\Python\Python38\site-packages\datasets\load.py", line 1694, in load_dataset
    builder_instance.download_and_prepare(
  File "C:\Users\........\AppData\Roaming\Python\Python38\site-packages\datasets\builder.py", line 603, in download_and_prepare
    self._save_info()
  File "C:\Users\..........\AppData\Local\Programs\Python\Python38\lib\contextlib.py", line 120, in __exit__
    next(self.gen)
  File "C:\Users\.....\AppData\Roaming\Python\Python38\site-packages\datasets\builder.py", line 557, in incomplete_dir
    os.rename(tmp_dir, dirname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\.........\\.cache\\huggingface\\datasets\\json\\c4-en-html-with-metadata-4635c2fd9249f62d\\0.0.0\\c90812beea906fcffe0d5e3bb9eba909a80a998b5f88e9f8acbd320aa91acfde.incomplete' -> 'C:\\Users\\I355109\\.cache\\huggingface\\datasets\\json\\c4-en-html-with-metadata-4635c2fd9249f62d\\0.0.0\\c90812beea906fcffe0d5e3bb9eba909a80a998b5f88e9f8acbd320aa91acfde'`
Issue: load_dataset using default cache on Windows causes PermissionError: [WinError 5] Access is denied

Comment:
I'm facing the same issue.

## System Information

- OS Edition: Windows 10 21H1
- OS build: 19043.1826
- Python version: 3.10.6 (installed using `choco install python`)
- datasets: 2.4.0
- PyArrow: 6.0.1

## Troubleshooting steps

- Restart the computer, unfortunately doesn't work! 🌚
- Checked the permissions of `~./cache/...`, looks fine.
- Tested with a simple file operation using the `open()` function and writing a hello_world.txt, it works fine.
- Tested with a different `cache_dir` value on the `load_dataset()`, e.g. "./data"
- Tested different datasets: `conll2003`, `squad_v2`, and `wiki_bio`.
- Downgraded datasets from `2.4.0` to `2.1.0`, issue persists.
- Tested it on WSL (Ubuntu 20.04), and it works! 
- Python reinstallation, in the first time downloading `conll2003` works fine, but `squad` or `squad_v2` raises Access Denied.
  - After the system or VSCode restart, the issue comes back.

## Resolution

I fixed it by changing the following command:

https://github.com/huggingface/datasets/blob/68cffe30917a9abed68d28caf54b40c10f977602/src/datasets/builder.py#L666

for

```python
shutil.move(tmp_dir, dirname)
```