Minimartzz/github-issues huggingface dataset:
Add Multidoc2dial Dataset
State: closed

This PR adds the MultiDoc2Dial dataset introduced in this [paper](https://arxiv.org/pdf/2109.12595v1.pdf )

Comments:
@songfeng cc

Hi @sivasankalpp, thanks for your PR.

There was a bug in TensorFlow/Keras. We have made a temporary fix in our master branch. Please, merge master into your PR branch, so that the CI tests pass.

```
git checkout multidoc2dial
git fetch upstream master
git merge upstream/master
```

Hi @albertvillanova, I have merged master into my PR branch. All tests are passing. 
Please take a look when you get a chance, thanks! 


Thanks for your feedback @lhoestq. We addressed your comments in the latest commit. Let us know if everything looks okay :)
FileNotFoundError for TupleIE dataste
State: closed
Labels: description: Something isn't working
name: bug

Hi,
`dataset = datasets.load_dataset('tuple_ie', 'all')`

returns a FileNotFound error. Is the data not available? 

Many thanks.


Comments:
@mariosasko @lhoestq Could you give me an update on how to load the dataset after the fix?
Thanks.

Hi @arda-vianai,

first, you can try:
```python
import datasets
dataset = datasets.load_dataset('tuple_ie', 'all', revision="master")
```
If this doesn't work, your version of `datasets` is missing some features that are required to run the dataset script, so install the master version with the following command:
```
pip install git+https://github.com/huggingface/datasets.git
```
and then:
```python
import datasets
dataset = datasets.load_dataset('tuple_ie', 'all')
```
should work (even without `revision`).

@mariosasko 
Thanks, it is working now. I actually did that before but I didn't restart the kernel. I restarted it and it works now. My bad!!!
Many thanks and great job!
-arda
Updated: DaNE - updated URL for download
State: closed

It seems that DaNLP has updated their download URLs and it therefore also needs to be updated in here...

Comments:
Actually it looks like the old URL is still working, and it's also the one that is mentioned in https://github.com/alexandrainst/danlp/blob/master/docs/docs/datasets.md

What makes you think we should use the new URL ?

@lhoestq Sorry! I might have jumped to conclusions a bit too fast here... 

I was working in Google Colab and got an error that it was unable to use the URL. I then forked the project, updated the URL, ran it locally and it worked. I therefore assumed that my URL update fixed the issue, however, I see now that it might rather be a Google Colab issue... 

Still - this seems to be the official URL for downloading the dataset, and I think that it will be most beneficial to use. :-) 

It looks like they're using these new urls for their new datasets. Maybe let's change to the new URL in case the old one stops working at one point. Thanks
Add mIoU metric
State: closed
Labels: description: New feature or request
name: enhancement

**Is your feature request related to a problem? Please describe.**
Recently, some semantic segmentation models were added to HuggingFace Transformers, including [SegFormer](https://huggingface.co/transformers/model_doc/segformer.html) and [BEiT](https://huggingface.co/transformers/model_doc/beit.html).

Semantic segmentation (which is the task of labeling every pixel of an image with a corresponding class) is typically evaluated using the Mean Intersection and Union (mIoU). Together with the upcoming Image Feature, adding this metric could be very handy when creating example scripts to fine-tune any Transformer-based model on a semantic segmentation dataset.

An implementation can be found [here](https://github.com/open-mmlab/mmsegmentation/blob/504965184c3e6bc9ec43af54237129ef21981a5f/mmseg/core/evaluation/metrics.py#L132) for instance.


Comments:
Resolved via https://github.com/huggingface/datasets/pull/3745.
Add GSM8K dataset
State: closed
Labels: description: Requesting to add a new dataset
name: dataset request

## Adding a Dataset
- **Name:** GSM8K (short for Grade School Math 8k)
- **Description:** GSM8K is a dataset of 8.5K high quality linguistically diverse grade school math word problems created by human problem writers.
- **Paper:** https://openai.com/blog/grade-school-math/
- **Data:** https://github.com/openai/grade-school-math
- **Motivation:** The dataset is useful to investigate the reasoning abilities of large Transformer models, such as GPT-3.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Comments:
Closed via https://github.com/huggingface/datasets/pull/4103
Multiprocessing filter/map (tests) not working on Windows
State: open
Labels: description: Something isn't working
name: bug

While running the tests, I found that the multiprocessing examples fail on Windows, or rather they do not complete: they cause a deadlock. I haven't dug deep into it, but they do not seem to work as-is. I currently have no time to tests this in detail but at least the tests seem not to run correctly (deadlocking).

## Steps to reproduce the bug
```shell
pytest tests/test_arrow_dataset.py -k "test_filter_multiprocessing"
pytest tests/test_arrow_dataset.py -k "test_map_multiprocessing"
```

## Expected results
The functionality to work on all platforms.

## Actual results
Deadlock.

## Environment info
- `datasets` version: 1.14.1.dev0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.9.2, also tested with 3.7.9
- PyArrow version: 4.0.1



Comments:
Dataset viewer issue for '*compguesswhat*'
State: closed
Labels: name: streaming

## Dataset viewer issue for '*compguesswhat*'

**Link:** https://huggingface.co/datasets/compguesswhat

File not found

Am I the one who added this dataset ? No


Comments:
```python
>>> import datasets
>>> dataset = datasets.load_dataset('compguesswhat', name='compguesswhat-original',split='train', streaming=True)
>>> next(iter(dataset))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 497, in __iter__
    for key, example in self._iter():
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 494, in _iter
    yield from ex_iterable
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 87, in __iter__
    yield from self.generate_examples_fn(**self.kwargs)
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/compguesswhat/4d08b9e0a8d1cf036c9626c93be4a759fdd9fcce050ea503ea14b075e830c799/compguesswhat.py", line 251, in _generate_examples
    with gzip.open(filepath) as in_file:
  File "/home/slesage/.pyenv/versions/3.9.6/lib/python3.9/gzip.py", line 58, in open
    binary_file = GzipFile(filename, gz_mode, compresslevel)
  File "/home/slesage/.pyenv/versions/3.9.6/lib/python3.9/gzip.py", line 173, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'zip://compguesswhat-original/0.2.0/compguesswhat.train.jsonl.gz::https://www.dropbox.com/s/l0nc13udml6vs0w/compguesswhat-original.zip?dl=1'
```

It's an issue with the streaming mode. Note that normal mode is used by the dataset viewer when streaming is failing, but only for the smallest datasets. This dataset is above the limit, hence the error.

Same case as https://github.com/huggingface/datasets/issues/3186#issuecomment-1096549774.

cc @huggingface/datasets 

There is an issue with the URLs of their data files: https://www.dropbox.com/s/l0nc13udml6vs0w/compguesswhat-original.zip?dl=1
> Dropbox Error: That didn't work for some reason

Error reported to their repo:
- https://github.com/CompGuessWhat/compguesswhat.github.io/issues/1

Closed by:
- #4968
combination of shuffle and filter results in a bug
State: closed
Labels: description: Something isn't working
name: bug

## Describe the bug
Hi,
I would like to shuffle a dataset, then filter it based on each existing label. however, the combination of `filter`, `shuffle` seems to results in a bug. In the minimal example below, as you see in the filtered results, the filtered labels are not unique, meaning filter has not worked. Any suggestions as a temporary fix is appreciated @lhoestq.

Thanks.
 Best regards
Rabeeh 

## Steps to reproduce the bug
```python
import numpy as np
import datasets 

datasets = datasets.load_dataset('super_glue', 'rte', script_version="master")
shuffled_data = datasets["train"].shuffle(seed=42)
for label in range(2):
    print("label ", label)
    data = shuffled_data.filter(lambda example: int(example['label']) == label)
    print("length ", len(data), np.unique(data['label']))
```

## Expected results
Filtering per label, should only return the data with that specific label.

## Actual results
As you can see, filtered data per label, has still two labels of [0, 1]
```
label  0
length  1249 [0 1]
label  1
length  1241 [0 1]
```
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1 
- Platform: linux 
- Python version: 3.7.11 
- PyArrow version: 5.0.0 


Comments:
I cannot reproduce this on master and pyarrow==4.0.1.


Hi ! There was a regression in `datasets` 1.12 that introduced this bug. It has been fixed in #3019 in 1.13

Can you try to update `datasets` and try again ?

Thanks a lot, fixes with 1.13
conll2003 incorrect label explanation
State: closed
Labels: description: Something isn't working
name: bug

In the [conll2003](https://huggingface.co/datasets/conll2003#data-fields) README, the labels are described as follows

> - `id`: a `string` feature.
> - `tokens`: a `list` of `string` features.
> - `pos_tags`: a `list` of classification labels, with possible values including `"` (0), `''` (1), `#` (2), `$` (3), `(` (4).
> - `chunk_tags`: a `list` of classification labels, with possible values including `O` (0), `B-ADJP` (1), `I-ADJP` (2), `B-ADVP` (3), `I-ADVP` (4).
> - `ner_tags`: a `list` of classification labels, with possible values including `O` (0), `B-PER` (1), `I-PER` (2), `B-ORG` (3), `I-ORG` (4) `B-LOC` (5), `I-LOC` (6) `B-MISC` (7), `I-MISC` (8).

First of all, it would be great if we can get a list of ALL possible pos_tags.

Second, the chunk tags labels cannot be correct. The description says the values go from 0 to 4 whereas the data shows values from at least 11 to 21 and 0.

EDIT: not really a bug, sorry for mistagging.

Comments:
Hi @BramVanroy,

since these fields are of type `ClassLabel` (you can check this with `dset.features`), you can inspect the possible values with:
```python
dset.features[field_name].feature.names  # .feature because it's a sequence of labels
```

and to find the mapping between names and integers, use: 
```python
dset.features[field_name].feature.int2str(value_or_values_list)  # map integer value to string value
# or
dset.features[field_name].feature.str2int(value_or_values_list)  # map string value to integer value
```
conll2002 issues
State: closed
Labels: description: Related to the dataset viewer on huggingface.co
name: dataset-viewer

**Link:** https://huggingface.co/datasets/conll2002

The dataset viewer throws a server error when trying to preview the dataset. 

```
Message:       Extraction protocol 'train' for file at 'https://raw.githubusercontent.com/teropa/nlp/master/resources/corpora/conll2002/esp.train' is not implemented yet
```

In addition, the "point of contact" has encoding issues and does not work when clicked.

Am I the one who added this dataset ? No, @lhoestq  did

Comments:
Hi ! Thanks for reporting :)

This is related to https://github.com/huggingface/datasets/issues/2742, I'm working on it. It should fix the viewer for around 80 datasets.


Ah, hadn't seen that sorry.

The scrambled "point of contact" is a separate issue though, I think.

@lhoestq The "point of contact" is still an issue.

It will be fixed in https://github.com/huggingface/datasets/pull/3274, thanks
Dataset viewer for nli_tr
State: closed
Labels: name: streaming

## Dataset viewer issue for '*nli_tr*'

**Link:** https://huggingface.co/datasets/nli_tr

Hello,

Thank you for the new dataset preview feature that will help the users to view the datasets online.

We just noticed that the dataset viewer widget in the `nli_tr` dataset shows the error below. The error must be due to a temporary problem that may have blocked access to the dataset through the dataset viewer. But the dataset is currently accessible through the link in the error message.   May we kindly ask if it would be possible to rerun the job so that it can access the dataset for the dataset viewer function?

Thank you.
Emrah


------------------------------------------
Server Error
Status code:   404
Exception:     FileNotFoundError
Message:       [Errno 2] No such file or directory: 'zip://snli_tr_1.0_train.jsonl::https://tabilab.cmpe.boun.edu.tr/datasets/nli_datasets/snli_tr_1.0.zip
------------------------------------------

Am I the one who added this dataset ? Yes


Comments:
It's an issue with the streaming mode:

```python
>>> import datasets
>>> dataset = datasets.load_dataset('nli_tr', name='snli_tr',split='test', streaming=True)
>>> next(iter(dataset))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 497, in __iter__
    for key, example in self._iter():
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 494, in _iter
    yield from ex_iterable
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 87, in __iter__
    yield from self.generate_examples_fn(**self.kwargs)
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/nli_tr/c2ddd0c0a70caddac6a81c2dae5ca7939f00060d517d08f1983927818dba6521/nli_tr.py", line 155, in _generate_examples
    with codecs.open(filepath, encoding="utf-8") as f:
  File "/home/slesage/.pyenv/versions/3.9.6/lib/python3.9/codecs.py", line 905, in open
    file = builtins.open(filename, mode, buffering)
FileNotFoundError: [Errno 2] No such file or directory: 'zip://snli_tr_1.0_test.jsonl::https://tabilab.cmpe.boun.edu.tr/datasets/nli_datasets/snli_tr_1.0.zip'
```

Note that normal mode is used by the dataset viewer when streaming is failing, but only for the smallest datasets. `nli_tr` is above the limit, hence the error.

cc @huggingface/datasets 

Apparently there is an issue with the data source URLs: Server Not Found
- https://tabilab.cmpe.boun.edu.tr/datasets/nli_datasets/snli_tr_1.0.zip

We are contacting the authors to ask them: 
@e-budur you are one of the authors: are you aware of the issue with the URLs of your data ?

Reported to their repo:
- https://github.com/boun-tabi/NLI-TR/issues/9

The server issue was temporary and is now resolved.

Once we have implemented support for streaming, the viewer works: https://huggingface.co/datasets/nli_tr
7z dataset preview not implemented?
State: closed
Labels: description: Related to the dataset viewer on huggingface.co
name: dataset-viewer

## Dataset viewer issue for dataset 'samsum'

**Link:** https://huggingface.co/datasets/samsum

Server Error
Status code:   400
Exception:     NotImplementedError
Message:       Extraction protocol '7z' for file at 'https://arxiv.org/src/1911.12237v2/anc/corpus.7z' is not implemented yet


Comments:
It's a bug in the dataset viewer: the dataset cannot be downloaded in streaming mode, but since the dataset is relatively small, the dataset viewer should have fallback to normal mode. Working on a fix.

Fixed. https://huggingface.co/datasets/samsum/viewer/samsum/train

<img width="1563" alt="Capture d’écran 2022-04-12 à 13 47 45" src="https://user-images.githubusercontent.com/1676121/162953339-cd8922d7-9037-408b-b896-eac1af0bb54f.png">

Thanks for reporting!
RONEC v2
State: closed

Hi, as we've recently finished with the new RONEC (Romanian Named Entity Corpus), we'd like to update the dataset here as well. It's actually essential as links to V1 are no longer valid. 

In reality we'd like to replace completely v1, as v2 is a full re-annotation of v1 with additional data (up to 2x size vs v1).

I've run the make style and all the dummy and real data test, and they passed.

I hope it's okay to merge the new RONEC v2 in the datasets. 

Thanks!

Comments:
@lhoestq Thanks for the review. I totally understand what you are saying. Normally, I would definitely agree with you, but in this particular case, the quality of v1 is poor, and the dataset itself is small (at the time we created v1 it was the only RO NER dataset, and its size was limited by the available resources). 

This is why we worked to build a larger one, with much better inter-annotator agreement. Fact is, models trained on v1 will be of very low quality and I would not recommend to anybody to use/do that. That's why I'd strongly suggest we replace v1 with v2, and kindof make v1 vanish :)  

What do you think? If you insist on having v1 accessible, I'll add the required code. Thanks!



Ok I see ! I think it's fine then, no need to re-add V1
Don't memoize strings when hashing since two identical strings may have different python ids
State: closed

When hashing an object that has several times the same string, the hashing could return a different hash if the identical strings share the same python `id()` or not.

Here is an example code that shows how the issue can affect the caching:
```python
import json
import pyarrow as pa
from datasets.features import Features
from datasets.fingerprint import Hasher

schema = pa.schema([pa.field("some_string", pa.string()), pa.field("another_string", pa.string())])
features_from_schema = Features.from_arrow_schema(schema)
Hasher.hash(features_from_schema) # dffa9dca9a73fd8c

features_dict = json.loads('{"some_string": {"dtype": "string", "id": null, "_type": "Value"}, "another_string": {"dtype": "string", "id": null, "_type": "Value"}}')
features_from_json = Features.from_dict(features_dict)
Hasher.hash(features_from_json) # 3812e76b15e6420e

features_from_schema == features_from_json # True
```

This is because in `features_dict`, some strings like "dtype" are repeated but don't share the same id, contrary to the ones in `features_from_schema`.

I fixed that by disabling memoization for strings.

This could be optimized in the future by implementing a smarter memoization with a special handling for strings.

Comments:
This change slows down the hash computation a little bit but from my tests it doesn't look too impactful. So I think it's fine to merge this.
fix label mapping
State: closed

Fixing label mapping for hlgd.
0 correponds to same event and 1 corresponds to different event
<img width="642" alt="Capture d’écran 2021-10-29 à 10 39 58 AM" src="https://user-images.githubusercontent.com/16107619/139454810-1f225e3d-ad48-44a8-b8b1-9205c9533839.png">
<img width="638" alt="Capture d’écran 2021-10-29 à 10 40 09 AM" src="https://user-images.githubusercontent.com/16107619/139454813-93066a3c-7d33-4f56-b133-2f1a7661e438.png">
nt

Comments:
heck, test failings. moving to draft. will come back to this later today hopefully

Thanks for fixing this :)
I just updated the dataset_infos.json and added the missing `pretty_name` tag to the dataset card

thank you @lhoestq! running around as always it felt through as a lower priority...
More control over TQDM when using map/filter with multiple processes
State: closed
Labels: description: New feature or request
name: enhancement

It would help with the clutter in my terminal if tqdm is only shown for rank 0 when using `num_proces>0` in the map and filter methods of datasets.

```python
dataset.map(lambda examples: tokenize(examples["text"]), batched=True, num_proc=6)
```

The above snippet leads to a lot of TQDM bars and depending on your terminal, these will not overwrite but keep pushing each other down.

```
 #0:   0%|          | 0/13 [00:00<?, ?ba/s]
 #1:   0%|          | 0/13 [00:00<?, ?ba/s]
 #2:   0%|          | 0/13 [00:00<?, ?ba/s]
 #3:   0%|          | 0/13 [00:00<?, ?ba/s]
 #4:   0%|          | 0/13 [00:00<?, ?ba/s]
 #5:   0%|          | 0/13 [00:00<?, ?ba/s]
 #0:   8%|          | 1/13 [00:00<?, ?ba/s]
 #1:   8%|          | 1/13 [00:00<?, ?ba/s]
...
```

Instead, it would be welcome if we had the option to only show the progress of rank 0.

Comments:
Hi,

It's hard to provide an API that would cover all use-cases with tqdm in this project.

However, you can make it work by defining a custom decorator (a bit hacky tho) as follows:
```python
import datasets

def progress_only_on_rank_0(func):
    def wrapper(*args, **kwargs):
        rank = kwargs.get("rank")
        disable_tqdm = kwargs.get("disable_tqdm", False)
        disable_tqdm = True if rank is not None and rank > 0 else disable_tqdm
        kwargs["disable_tqdm"] = disable_tqdm
        return func(*args, **kwargs)
    return wrapper
    
datasets.Dataset._map_single = progress_only_on_rank_0(datasets.Dataset._map_single)
``` 

EDIT: Ups, closed by accident.

Thanks for the provided links. `Trainer` requires this for training in multi-node distributed setting. However, `Dataset.map` doesn't support that yet.

Do you have an API for this in mind? `Dataset.map` is already bloated with the arguments, so IMO it's not a good idea to add a new arg there.



Inspiration may be found at `transformers`.

https://github.com/huggingface/transformers/blob/4a394cf53f05e73ab9bbb4b179a40236a5ffe45a/src/transformers/trainer.py#L1231-L1233

To get unique IDs for each worker, see https://stackoverflow.com/a/10192611/1150683
Add docs for `to_tf_dataset`
State: closed
Labels: description: Improvements or additions to documentation
name: documentation

This PR adds some documentation for new features released in v1.13.0, with the main addition being `to_tf_dataset`:

- Show how to use `to_tf_dataset` in the tutorial, and move `set_format(type='tensorflow'...)` to the Process section (let me know if I'm missing anything @Rocketknight1 😅).
- Add an example for loading dataset from multiple zipped CSV files to the Load section.
- Add an example for removing columns for an `IterableDataset`.
- Add graphic for visualizing streaming.

Comments:
This looks great, thank you!

Thanks !

For some reason the new GIF is 6MB, which is a bit heavy for an image on a website. The previous one was around 200KB though which is perfect. For a good experience we usually expect images to be less than 500KB - otherwise for users with poor connection it takes too long to load. Could you try to reduce its size ? Than I think we can merge :)
Asserts replaced by exceptions (huggingface#3171)
State: closed

I've replaced two asserts with their proper exceptions following the guidelines described in issue #3171 by following the contributing guidelines.

PS: This is one of my first PRs, hoping I don't break anything!

Comments:
Your first PR went smoothly, well done!
You are welcome to continue contributing to this project.
Gràcies, @joseporiolayats! 😉