helmo/github-issues huggingface dataset:
Eventual Invalid Token Error at setup of private datasets
State: closed | Author: albertvillanova (MEMBER)
Labels: name: bug

## Describe the bug
From time to time, there appear Invalid Token errors with private datasets:

- https://app.circleci.com/pipelines/github/huggingface/datasets/8520/workflows/d44629f2-4749-40f8-a657-50931d0b3434/jobs/52534
  ```
  ____________ ERROR at setup of test_load_streaming_private_dataset _____________
  ValueError: Invalid token passed!

  ____ ERROR at setup of test_load_streaming_private_dataset_with_zipped_data ____
  ValueError: Invalid token passed!
  
  =========================== short test summary info ============================
  ERROR tests/test_load.py::test_load_streaming_private_dataset - ValueError: I...
  ERROR tests/test_load.py::test_load_streaming_private_dataset_with_zipped_data
  ```

- https://app.circleci.com/pipelines/github/huggingface/datasets/8557/workflows/a8383181-ba6d-4487-9d0a-f750b6dcb936/jobs/52763
  ```
  ____ ERROR at setup of test_load_streaming_private_dataset_with_zipped_data ____
  [gw1] linux -- Python 3.6.15 /home/circleci/.pyenv/versions/3.6.15/bin/python3.6

  hf_api = <huggingface_hub.hf_api.HfApi object at 0x7f4899bab908>
  hf_token = 'vgNbyuaLNEBuGbgCEtSBCOcPjZnngJufHkTaZvHwkXKGkHpjBPwmLQuJVXRxBuaRzNlGjlMpYRPbthfHPFWXaaEDTLiqTTecYENxukRYVAAdpeApIUPxcgsowadkTkPj'
  zip_csv_path = PosixPath('/tmp/pytest-of-circleci/pytest-0/popen-gw1/data16/dataset.csv.zip')

      @pytest.fixture(scope="session")
      def hf_private_dataset_repo_zipped_txt_data_(hf_api: HfApi, hf_token, zip_csv_path):
          repo_name = "repo_zipped_txt_data-{}".format(int(time.time() * 10e3))
          hf_api.create_repo(token=hf_token, name=repo_name, repo_type="dataset", private=True)
          repo_id = f"{USER}/{repo_name}"
          hf_api.upload_file(
              token=hf_token,
              path_or_fileobj=str(zip_csv_path),
              path_in_repo="data.zip",
              repo_id=repo_id,
  >           repo_type="dataset",
          )

  tests/hub_fixtures.py:68:

  ...

  ValueError: Invalid token passed!
  =========================== short test summary info ============================
  ERROR tests/test_load.py::test_load_streaming_private_dataset_with_zipped_data
  ```
Fix code quality bug in riddle_sense dataset
State: closed | Author: albertvillanova (MEMBER)
Labels: name: bug

## Describe the bug
```
datasets/riddle_sense/riddle_sense.py:36:21: W291 trailing whitespace
```

Comments: To give more context: https://github.com/psf/black/issues/318. `black` doesn't treat this as a bug, but `flake8` does.
Pin version exclusion for tensorflow incompatible with keras
State: closed | Author: albertvillanova (MEMBER)

Once `tensorflow` version 2.6.2 is released:
- https://github.com/tensorflow/tensorflow/commit/c1867f3bfdd1042f694df7a9870be51ba80543cb
- https://pypi.org/project/tensorflow/2.6.2/

with the patch:
- tensorflow/tensorflow#52927

we can remove the temporary fix we introduced in:
- #3208

Fix #3209.
Add ACAV100M Dataset
State: open | Author: nateraw (CONTRIBUTOR)
Labels: name: dataset request

name: vision

## Adding a Dataset
- **Name:** *ACAV100M*
- **Description:** *contains 100 million videos with high audio-visual correspondence, ideal for self-supervised video representation learning.*
- **Paper:** *https://arxiv.org/abs/2101.10803*
- **Data:** *https://github.com/sangho-vision/acav100m*
- **Motivation:** *The largest dataset (to date) for audio-visual learning.*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).
Sort files before loading
State: closed | Author: lvwerra (MEMBER)
Labels: name: enhancement

When loading a dataset that consists of several files (e.g. `my_data/data_001.json`, `my_data/data_002.json` etc.) they are not loaded in order when using `load_dataset("my_data")`.

This could lead to counter-intuitive results if, for example, the data files are sorted by date or similar since they would appear in different order in the `Dataset`.

The straightforward solution is to sort the list of files alphabetically before loading them.

cc @lhoestq 


Comments: This will be fixed by https://github.com/huggingface/datasets/pull/3221
Fix disable_nullable default value to False
State: closed | Author: lhoestq (MEMBER)

Currently the `disable_nullable` parameter is not consistent across all dataset transforms. For example it is `False` in `map` but `True` in `flatten_indices`.

This creates unexpected behaviors like this
```python
from datasets import Dataset, concatenate_datasets

d1 = Dataset.from_dict({"a": [0, 1, 2, 3]})
d2 = d1.filter(lambda x: x["a"] < 2).flatten_indices()
d1.data.schema == d2.data.schema  # False
```
This can cause issues when concatenating datasets for example.

For consistency I set `disable_nullable` to `False` in `flatten_indices` and I fixed some docstrings

cc @SBrandeis
ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.15.1/datasets/wmt16/wmt16.py
State: closed | Author: xiuzhilu (NONE)
Labels: name: dataset bug

when I use python examples/pytorch/translation/run_translation.py     --model_name_or_path examples/pytorch/translation/opus-mt-en-ro     --do_train     --do_eval     --source_lang en     --target_lang ro     --dataset_name wmt16     --dataset_config_name ro-en     --output_dir /tmp/tst-translation     --per_device_train_batch_size=4     --per_device_eval_batch_size=4     --overwrite_output_dir     --predict_with_generate to finetune translation model on huggingface, I get the issue"ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.15.1/datasets/wmt16/wmt16.py".But I can open the https://raw.githubusercontent.com/huggingface/datasets/1.15.1/datasets/wmt16/wmt16.py by using website. What should I do to solve the issue?

Comments: Hi ! Do you have some kind of proxy in your browser that gives you access to internet ?

Maybe you're having this error because you don't have access to this URL from python ?

Hi，do you fixed this error?
I still have this issue when use "use_auth_token=True"

You don't need authentication to access those github hosted files
Please check that you can access this URL from your browser and also from your terminal
CI error: Another metric with the same name already exists in Keras 2.7.0
State: closed | Author: albertvillanova (MEMBER)
Labels: name: bug

## Describe the bug
Release of TensorFlow 2.7.0 contains an incompatibility with Keras. See:
- keras-team/keras#15579

This breaks our CI test suite: https://app.circleci.com/pipelines/github/huggingface/datasets/8493/workflows/055c7ae2-43bc-49b4-9f11-8fc71f35a25c/jobs/52363
Add Multidoc2dial Dataset
State: closed | Author: sivasankalpp (CONTRIBUTOR)

This PR adds the MultiDoc2Dial dataset introduced in this [paper](https://arxiv.org/pdf/2109.12595v1.pdf )

Comments: @songfeng cc

Hi @sivasankalpp, thanks for your PR.

There was a bug in TensorFlow/Keras. We have made a temporary fix in our master branch. Please, merge master into your PR branch, so that the CI tests pass.

```
git checkout multidoc2dial
git fetch upstream master
git merge upstream/master
```

Hi @albertvillanova, I have merged master into my PR branch. All tests are passing. 
Please take a look when you get a chance, thanks! 


Thanks for your feedback @lhoestq. We addressed your comments in the latest commit. Let us know if everything looks okay :)
FileNotFoundError for TupleIE dataste
State: closed | Author: arda-vianai (NONE)
Labels: name: bug

Hi,
`dataset = datasets.load_dataset('tuple_ie', 'all')`

returns a FileNotFound error. Is the data not available? 

Many thanks.


Comments: @mariosasko @lhoestq Could you give me an update on how to load the dataset after the fix?
Thanks.

Hi @arda-vianai,

first, you can try:
```python
import datasets
dataset = datasets.load_dataset('tuple_ie', 'all', revision="master")
```
If this doesn't work, your version of `datasets` is missing some features that are required to run the dataset script, so install the master version with the following command:
```
pip install git+https://github.com/huggingface/datasets.git
```
and then:
```python
import datasets
dataset = datasets.load_dataset('tuple_ie', 'all')
```
should work (even without `revision`).

@mariosasko 
Thanks, it is working now. I actually did that before but I didn't restart the kernel. I restarted it and it works now. My bad!!!
Many thanks and great job!
-arda
Updated: DaNE - updated URL for download
State: closed | Author: MalteHB (CONTRIBUTOR)

It seems that DaNLP has updated their download URLs and it therefore also needs to be updated in here...

Comments: Actually it looks like the old URL is still working, and it's also the one that is mentioned in https://github.com/alexandrainst/danlp/blob/master/docs/docs/datasets.md

What makes you think we should use the new URL ?

@lhoestq Sorry! I might have jumped to conclusions a bit too fast here... 

I was working in Google Colab and got an error that it was unable to use the URL. I then forked the project, updated the URL, ran it locally and it worked. I therefore assumed that my URL update fixed the issue, however, I see now that it might rather be a Google Colab issue... 

Still - this seems to be the official URL for downloading the dataset, and I think that it will be most beneficial to use. :-) 

It looks like they're using these new urls for their new datasets. Maybe let's change to the new URL in case the old one stops working at one point. Thanks
Add mIoU metric
State: closed | Author: NielsRogge (CONTRIBUTOR)
Labels: name: enhancement

**Is your feature request related to a problem? Please describe.**
Recently, some semantic segmentation models were added to HuggingFace Transformers, including [SegFormer](https://huggingface.co/transformers/model_doc/segformer.html) and [BEiT](https://huggingface.co/transformers/model_doc/beit.html).

Semantic segmentation (which is the task of labeling every pixel of an image with a corresponding class) is typically evaluated using the Mean Intersection and Union (mIoU). Together with the upcoming Image Feature, adding this metric could be very handy when creating example scripts to fine-tune any Transformer-based model on a semantic segmentation dataset.

An implementation can be found [here](https://github.com/open-mmlab/mmsegmentation/blob/504965184c3e6bc9ec43af54237129ef21981a5f/mmseg/core/evaluation/metrics.py#L132) for instance.


Comments: Resolved via https://github.com/huggingface/datasets/pull/3745.
Add GSM8K dataset
State: closed | Author: NielsRogge (CONTRIBUTOR)
Labels: name: dataset request

## Adding a Dataset
- **Name:** GSM8K (short for Grade School Math 8k)
- **Description:** GSM8K is a dataset of 8.5K high quality linguistically diverse grade school math word problems created by human problem writers.
- **Paper:** https://openai.com/blog/grade-school-math/
- **Data:** https://github.com/openai/grade-school-math
- **Motivation:** The dataset is useful to investigate the reasoning abilities of large Transformer models, such as GPT-3.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Comments: Closed via https://github.com/huggingface/datasets/pull/4103
Multiprocessing filter/map (tests) not working on Windows
State: open | Author: BramVanroy (CONTRIBUTOR)
Labels: name: bug

While running the tests, I found that the multiprocessing examples fail on Windows, or rather they do not complete: they cause a deadlock. I haven't dug deep into it, but they do not seem to work as-is. I currently have no time to tests this in detail but at least the tests seem not to run correctly (deadlocking).

## Steps to reproduce the bug
```shell
pytest tests/test_arrow_dataset.py -k "test_filter_multiprocessing"
pytest tests/test_arrow_dataset.py -k "test_map_multiprocessing"
```

## Expected results
The functionality to work on all platforms.

## Actual results
Deadlock.

## Environment info
- `datasets` version: 1.14.1.dev0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.9.2, also tested with 3.7.9
- PyArrow version: 4.0.1
Dataset viewer issue for '*compguesswhat*'
State: closed | Author: benotti (NONE)
Labels: name: streaming

## Dataset viewer issue for '*compguesswhat*'

**Link:** https://huggingface.co/datasets/compguesswhat

File not found

Am I the one who added this dataset ? No


Comments: ```python
>>> import datasets
>>> dataset = datasets.load_dataset('compguesswhat', name='compguesswhat-original',split='train', streaming=True)
>>> next(iter(dataset))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 497, in __iter__
    for key, example in self._iter():
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 494, in _iter
    yield from ex_iterable
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 87, in __iter__
    yield from self.generate_examples_fn(**self.kwargs)
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/compguesswhat/4d08b9e0a8d1cf036c9626c93be4a759fdd9fcce050ea503ea14b075e830c799/compguesswhat.py", line 251, in _generate_examples
    with gzip.open(filepath) as in_file:
  File "/home/slesage/.pyenv/versions/3.9.6/lib/python3.9/gzip.py", line 58, in open
    binary_file = GzipFile(filename, gz_mode, compresslevel)
  File "/home/slesage/.pyenv/versions/3.9.6/lib/python3.9/gzip.py", line 173, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'zip://compguesswhat-original/0.2.0/compguesswhat.train.jsonl.gz::https://www.dropbox.com/s/l0nc13udml6vs0w/compguesswhat-original.zip?dl=1'
```

It's an issue with the streaming mode. Note that normal mode is used by the dataset viewer when streaming is failing, but only for the smallest datasets. This dataset is above the limit, hence the error.

Same case as https://github.com/huggingface/datasets/issues/3186#issuecomment-1096549774.

cc @huggingface/datasets 

There is an issue with the URLs of their data files: https://www.dropbox.com/s/l0nc13udml6vs0w/compguesswhat-original.zip?dl=1
> Dropbox Error: That didn't work for some reason

Error reported to their repo:
- https://github.com/CompGuessWhat/compguesswhat.github.io/issues/1

Closed by:
- #4968
combination of shuffle and filter results in a bug
State: closed | Author: rabeehk (CONTRIBUTOR)
Labels: name: bug

## Describe the bug
Hi,
I would like to shuffle a dataset, then filter it based on each existing label. however, the combination of `filter`, `shuffle` seems to results in a bug. In the minimal example below, as you see in the filtered results, the filtered labels are not unique, meaning filter has not worked. Any suggestions as a temporary fix is appreciated @lhoestq.

Thanks.
 Best regards
Rabeeh 

## Steps to reproduce the bug
```python
import numpy as np
import datasets 

datasets = datasets.load_dataset('super_glue', 'rte', script_version="master")
shuffled_data = datasets["train"].shuffle(seed=42)
for label in range(2):
    print("label ", label)
    data = shuffled_data.filter(lambda example: int(example['label']) == label)
    print("length ", len(data), np.unique(data['label']))
```

## Expected results
Filtering per label, should only return the data with that specific label.

## Actual results
As you can see, filtered data per label, has still two labels of [0, 1]
```
label  0
length  1249 [0 1]
label  1
length  1241 [0 1]
```
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1 
- Platform: linux 
- Python version: 3.7.11 
- PyArrow version: 5.0.0 


Comments: I cannot reproduce this on master and pyarrow==4.0.1.


Hi ! There was a regression in `datasets` 1.12 that introduced this bug. It has been fixed in #3019 in 1.13

Can you try to update `datasets` and try again ?

Thanks a lot, fixes with 1.13
conll2003 incorrect label explanation
State: closed | Author: BramVanroy (CONTRIBUTOR)
Labels: name: bug

In the [conll2003](https://huggingface.co/datasets/conll2003#data-fields) README, the labels are described as follows

> - `id`: a `string` feature.
> - `tokens`: a `list` of `string` features.
> - `pos_tags`: a `list` of classification labels, with possible values including `"` (0), `''` (1), `#` (2), `$` (3), `(` (4).
> - `chunk_tags`: a `list` of classification labels, with possible values including `O` (0), `B-ADJP` (1), `I-ADJP` (2), `B-ADVP` (3), `I-ADVP` (4).
> - `ner_tags`: a `list` of classification labels, with possible values including `O` (0), `B-PER` (1), `I-PER` (2), `B-ORG` (3), `I-ORG` (4) `B-LOC` (5), `I-LOC` (6) `B-MISC` (7), `I-MISC` (8).

First of all, it would be great if we can get a list of ALL possible pos_tags.

Second, the chunk tags labels cannot be correct. The description says the values go from 0 to 4 whereas the data shows values from at least 11 to 21 and 0.

EDIT: not really a bug, sorry for mistagging.

Comments: Hi @BramVanroy,

since these fields are of type `ClassLabel` (you can check this with `dset.features`), you can inspect the possible values with:
```python
dset.features[field_name].feature.names  # .feature because it's a sequence of labels
```

and to find the mapping between names and integers, use: 
```python
dset.features[field_name].feature.int2str(value_or_values_list)  # map integer value to string value
# or
dset.features[field_name].feature.str2int(value_or_values_list)  # map string value to integer value
```
conll2002 issues
State: closed | Author: BramVanroy (CONTRIBUTOR)
Labels: name: dataset-viewer

**Link:** https://huggingface.co/datasets/conll2002

The dataset viewer throws a server error when trying to preview the dataset. 

```
Message:       Extraction protocol 'train' for file at 'https://raw.githubusercontent.com/teropa/nlp/master/resources/corpora/conll2002/esp.train' is not implemented yet
```

In addition, the "point of contact" has encoding issues and does not work when clicked.

Am I the one who added this dataset ? No, @lhoestq  did

Comments: Hi ! Thanks for reporting :)

This is related to https://github.com/huggingface/datasets/issues/2742, I'm working on it. It should fix the viewer for around 80 datasets.


Ah, hadn't seen that sorry.

The scrambled "point of contact" is a separate issue though, I think.

@lhoestq The "point of contact" is still an issue.

It will be fixed in https://github.com/huggingface/datasets/pull/3274, thanks
Dataset viewer for nli_tr
State: closed | Author: e-budur (CONTRIBUTOR)
Labels: name: streaming

## Dataset viewer issue for '*nli_tr*'

**Link:** https://huggingface.co/datasets/nli_tr

Hello,

Thank you for the new dataset preview feature that will help the users to view the datasets online.

We just noticed that the dataset viewer widget in the `nli_tr` dataset shows the error below. The error must be due to a temporary problem that may have blocked access to the dataset through the dataset viewer. But the dataset is currently accessible through the link in the error message.   May we kindly ask if it would be possible to rerun the job so that it can access the dataset for the dataset viewer function?

Thank you.
Emrah


------------------------------------------
Server Error
Status code:   404
Exception:     FileNotFoundError
Message:       [Errno 2] No such file or directory: 'zip://snli_tr_1.0_train.jsonl::https://tabilab.cmpe.boun.edu.tr/datasets/nli_datasets/snli_tr_1.0.zip
------------------------------------------

Am I the one who added this dataset ? Yes


Comments: It's an issue with the streaming mode:

```python
>>> import datasets
>>> dataset = datasets.load_dataset('nli_tr', name='snli_tr',split='test', streaming=True)
>>> next(iter(dataset))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 497, in __iter__
    for key, example in self._iter():
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 494, in _iter
    yield from ex_iterable
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 87, in __iter__
    yield from self.generate_examples_fn(**self.kwargs)
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/nli_tr/c2ddd0c0a70caddac6a81c2dae5ca7939f00060d517d08f1983927818dba6521/nli_tr.py", line 155, in _generate_examples
    with codecs.open(filepath, encoding="utf-8") as f:
  File "/home/slesage/.pyenv/versions/3.9.6/lib/python3.9/codecs.py", line 905, in open
    file = builtins.open(filename, mode, buffering)
FileNotFoundError: [Errno 2] No such file or directory: 'zip://snli_tr_1.0_test.jsonl::https://tabilab.cmpe.boun.edu.tr/datasets/nli_datasets/snli_tr_1.0.zip'
```

Note that normal mode is used by the dataset viewer when streaming is failing, but only for the smallest datasets. `nli_tr` is above the limit, hence the error.

cc @huggingface/datasets 

Apparently there is an issue with the data source URLs: Server Not Found
- https://tabilab.cmpe.boun.edu.tr/datasets/nli_datasets/snli_tr_1.0.zip

We are contacting the authors to ask them: 
@e-budur you are one of the authors: are you aware of the issue with the URLs of your data ?

Reported to their repo:
- https://github.com/boun-tabi/NLI-TR/issues/9

The server issue was temporary and is now resolved.

Once we have implemented support for streaming, the viewer works: https://huggingface.co/datasets/nli_tr
7z dataset preview not implemented?
State: closed | Author: Kirili4ik (NONE)
Labels: name: dataset-viewer

## Dataset viewer issue for dataset 'samsum'

**Link:** https://huggingface.co/datasets/samsum

Server Error
Status code:   400
Exception:     NotImplementedError
Message:       Extraction protocol '7z' for file at 'https://arxiv.org/src/1911.12237v2/anc/corpus.7z' is not implemented yet


Comments: It's a bug in the dataset viewer: the dataset cannot be downloaded in streaming mode, but since the dataset is relatively small, the dataset viewer should have fallback to normal mode. Working on a fix.

Fixed. https://huggingface.co/datasets/samsum/viewer/samsum/train

<img width="1563" alt="Capture d’écran 2022-04-12 à 13 47 45" src="https://user-images.githubusercontent.com/1676121/162953339-cd8922d7-9037-408b-b896-eac1af0bb54f.png">

Thanks for reporting!