rIsHu009/github-issues-updated huggingface dataset:
Title: Remove redundant isort module placement
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2021-11-09T13:50:30Z

`isort` can place modules by itself from [version 5.0.0](https://pycqa.github.io/isort/docs/upgrade_guides/5.0.0.html#module-placement-changes-known_third_party-known_first_party-default_section-etc) onwards, making the `known_first_party` and `known_third_party` fields in `setup.cfg` redundant (this is why our CI works, even though we haven't touched these options in a while).

Comments:
Title: Adding ANERcorp-CAMeLLab dataset
Author: vitalyshalumov (NONE)
State: open
Created: 2021-11-09T12:04:04Z
Labels: dataset request


Comments:
Adding ANERcorp dataset

## Adding a Dataset
- **Name:** *ANERcorp-CAMeLLab*
- **Description:** *Since its creation in 2008, the ANERcorp dataset (Benajiba & Rosso, 2008) has been a standard reference used by Arabic named entity recognition researchers around the world. However, over time, this dataset was copied over from user to user, modified slightly here and there, and split in many different configurations that made it hard to compare fairly across papers and systems.

In 2020, a group of researchers from CAMeL Lab (Habash, Alhafni and Oudah), and Mind Lab (Antoun and Baly) met with the creator of the corpus, Yassine Benajiba, to consult with him and collectively agree on an exact split, and accepted minor corrections from the original dataset. Bashar Alhafni from CAMeL Lab working with Nizar Habash implemented the decisions provided in this release.*

- **Paper:** *(a) Benajiba, Yassine, Paolo Rosso, and José Miguel Benedí Ruiz. "Anersys: An Arabic named entity recognition system based on maximum entropy." In International Conference on Intelligent Text Processing and Computational Linguistics, pp. 143-153. Springer, Berlin, Heidelberg, 2007.

(b)Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. "CAMeL Tools: An Open Source Python Toolkit, for Arabic Natural Language Processing." In Proceedings of the Conference on Language Resources and Evaluation (LREC 2020), Marseille, 2020.*
- **Data:** *https://camel.abudhabi.nyu.edu/anercorp/*
- **Motivation:** This is the standard dataset for evaluating NER performance in Arabic*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).
Title: Couldn't reach data file for disaster_response_messages
Author: pandya6988 (NONE)
State: closed
Created: 2021-11-09T09:26:42Z
Labels: dataset bug

## Describe the bug
Following command gives an ConnectionError.

## Steps to reproduce the bug
```python
disaster = load_dataset('disaster_response_messages')
```

## Error
```
ConnectionError: Couldn't reach https://datasets.appen.com/appen_datasets/disaster_response_data/disaster_response_messages_training.csv
```
## Expected results
It should load dataset without an error

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform: Google Colab
- Python version: 3.7
- PyArrow version:

Comments:
It looks like the dataset isn't available anymore on appen.com

The CSV files appear to still be available at https://www.kaggle.com/landlord/multilingual-disaster-response-messages though. It says that the data are under the CC0 license so I guess we can host the dataset elsewhere instead ?
Title: Reuters21578 Couldn't reach 
Author: TingNLP (NONE)
State: closed
Created: 2021-11-09T06:08:56Z
Labels: dataset bug

``## Adding a Dataset
- **Name:** *Reuters21578*
- **Description:** *ConnectionError: Couldn't reach https://kdd.ics.uci.edu/databases/reuters21578/reuters21578.tar.gz*
- **Data:** *https://huggingface.co/datasets/reuters21578*

`from datasets import load_dataset`
`dataset = load_dataset("reuters21578", 'ModLewis')`

ConnectionError: Couldn't reach https://kdd.ics.uci.edu/databases/reuters21578/reuters21578.tar.gz

And I try to request the link as follow:
`import requests`
`requests.head('https://kdd.ics.uci.edu/databases/reuters21578/reuters21578.tar.gz')`

SSLError: HTTPSConnectionPool(host='kdd.ics.uci.edu', port=443): Max retries exceeded with url: /databases/reuters21578/reuters21578.tar.gz (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),))

This problem likes #575
What should I do ?

Comments:
Hi ! The URL works fine on my side today, could you try again ?

thank you @lhoestq 
it works
Title: wikitext description wrong
Author: hongyuanmei (NONE)
State: closed
Created: 2021-11-09T04:06:52Z
Labels: bug

## Describe the bug
Descriptions of the wikitext datasests are wrong. 

## Steps to reproduce the bug
Please see: https://github.com/huggingface/datasets/blob/f6dcafce996f39b6a4bbe3a9833287346f4a4b68/datasets/wikitext/wikitext.py#L50

## Expected results
The descriptions for raw-v1 and v1 should be switched.

Comments:
Hi @hongyuanmei, thanks for reporting.

I'm fixing it.

Duplicate of:
- #795
Title: Addd options to use updated bleurt checkpoints
Author: jaehlee (CONTRIBUTOR)
State: closed
Created: 2021-11-08T18:53:54Z

Adds options to use newer recommended checkpoint (as of 2021/10/8) bleurt-20 and its distilled versions. 

Updated checkpoints are described in https://github.com/google-research/bleurt/blob/master/checkpoints.md#the-recommended-checkpoint-bleurt-20

This change won't affect the default behavior of metrics/bleurt. It only adds option to load newer checkpoints as

`datasets.load_metric('bleurt', 'bleurt-20')`

`bluert-20` generates scores roughly between 0 and 1, which wasn't the case for the previous checkpoints.

Comments:
Title: Avoid PyArrow type optimization if it fails
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2021-11-08T16:10:27Z

Adds a new variable, `DISABLE_PYARROW_TYPES_OPTIMIZATION`, to `config.py` for easier control of the Arrow type optimization.

Fix #2206

Comments:
That's good to have a way to disable this easily :)
I just find it a bit unfortunate that users would have to experience the error once and then do `DISABLE_PYARROW_TYPES_OPTIMIZATION=1`. Do you know if there's a way to simply fallback on disabling it automatically when it fails ?

@lhoestq Actually, I agree a fallback makes more sense. The current approach is not very practical indeed and would require a mention in the docs.


Replaced the env variable with a fallback!

Hmm if the fallback automatically happens without the user knowing it, then I don't think we really need to mention it. But if you really wanted to, I think the [Improve performance](https://huggingface.co/docs/datasets/cache.html#improve-performance) section would be a great place for it! 

Yea I think this could just end up in a note that says that `datasets` automatically picks the most optimized integer precision for your tokenized text data to save you disk space. Maybe later if we have a page on text processing we can add this note, but for now I agree it doesn't fit well into the doc.

In particular in the "Improve performance" section we mention what users can do to speed up their computations, while this behavior is just some internal feature that users don't have control over anyway.
Title: The Xsum datasets seems not able to download.
Author: FYYFU (NONE)
State: closed
Created: 2021-11-08T11:58:54Z
Labels: bug

## Describe the bug
The download Link of the Xsum dataset provided in the repository is [Link](http://bollin.inf.ed.ac.uk/public/direct/XSUM-EMNLP18-Summary-Data-Original.tar.gz). It seems not able to download.

## Steps to reproduce the bug
```python
load_dataset('xsum')
```


## Actual results
``` python
raise ConnectionError("Couldn't reach {}".format(url))
ConnectionError: Couldn't reach http://bollin.inf.ed.ac.uk/public/direct/XSUM-EMNLP18-Summary-Data-Original.tar.gz
```

Comments:
Hi ! On my side the URL is working fine, could you try again ?

> Hi ! On my side the URL is working fine, could you try again ?

I try it again and cannot download the file (might because of my location). Could you please provide another download link(such as google drive)?   :>

I don't know other download links - this is the one provided by the authors of the dataset. Maybe you can try downloading from another location ? There are several solutions: a VPN, a remote VM or Google Colab for example.

> I don't know other download links - this is the one provided by the authors of the dataset. Maybe you can try downloading from another location ? There are several solutions: a VPN, a remote VM or Google Colab for example.

:> ok. Thanks for your reply.
Title: Group tests in multiprocessing workers by test file
Author: albertvillanova (MEMBER)
State: closed
Created: 2021-11-08T08:46:03Z

By grouping tests by test file, we make sure that all the tests in `test_load.py` are sent to the same worker.

Therefore, the fixture `hf_token` will be called only once (and from the same worker).

Related to: #3200.
Fix #3219.

Comments:
Title: Add full tagset to conll2003 README
Author: BramVanroy (CONTRIBUTOR)
State: closed
Created: 2021-11-08T08:06:04Z

Even though it is possible to manually get the tagset list with

```python
dset.features[field_name].feature.names
```

I think it is useful to have an overview of the used tagset on the dataset card. This is particularly useful in light of the **dataset viewer**: the tags are encoded, so it is not immediately obvious what they are for a given sample. Adding a label-int mapping should make it easier for visitors to get a grasp of what they mean.

From user-experience perspective, I would urge the full tagsets to always be available in the README's but I understand that that would take a lot of work, probably. Perhaps it can be automated?

closes #3189

Comments:
I also added the missing `pretty_name` tag in the dataset card to fix the CI
Title: Fix URL in CITATION file
Author: albertvillanova (MEMBER)
State: closed
Created: 2021-11-07T10:04:35Z

Currently the BibTeX citation parsed from the CITATION file has wrong URL (it shows the repo URL instead of the proceedings paper URL):
```
@inproceedings{Lhoest_Datasets_A_Community_2021,
author = {Lhoest, Quentin and Villanova del Moral, Albert and von Platen, Patrick and Wolf, Thomas and Šaško, Mario and Jernite, Yacine and Thakur, Abhishek and Tunstall, Lewis and Patil, Suraj and Drame, Mariama and Chaumond, Julien and Plu, Julien and Davison, Joe and Brandeis, Simon and Sanh, Victor and Le Scao, Teven and Canwen Xu, Kevin and Patry, Nicolas and Liu, Steven and McMillan-Major, Angelina and Schmid, Philipp and Gugger, Sylvain and Raw, Nathan and Lesage, Sylvain and Lozhkov, Anton and Carrigan, Matthew and Matussière, Théo and von Werra, Leandro and Debut, Lysandre and Bekman, Stas and Delangue, Clément},
booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations},
month = {11},
pages = {175--184},
publisher = {Association for Computational Linguistics},
title = {{Datasets: A Community Library for Natural Language Processing}},
url = {https://github.com/huggingface/datasets},
year = {2021}
}
```

Comments:
Title: Update tatoeba to v2021-07-22
Author: KoichiYasuoka (CONTRIBUTOR)
State: closed
Created: 2021-11-06T15:14:31Z

Tatoeba's latest version is v2021-07-22

Comments:
How about this? @lhoestq @abhishekkrthakur 

Hi ! I think it would be nice if people could still be able to load the old version.
Maybe this can be a parameter ? For example to load the old version they could do
```python
load_dataset("tatoeba", lang1="en", lang2="mr", date="v2020-11-09")
```

If it sounds good to you, we can add this parameter to the TatoebaConfig:
```python
class TatoebaConfig(datasets.BuilderConfig):
    def __init__(self, *args, lang1=None, lang2=None, date="v2021-07-22", **kwargs):
    self.date = date
```
and then pass the date to the URL
```python
_BASE_URL = "https://object.pouta.csc.fi/OPUS-Tatoeba/{}/moses/{}-{}.txt.zip"
```
```python
        def _base_url(lang1, lang2, date):
            return _BASE_URL.format(date, lang1, lang2)
```

What do you think ?

`_DATE = "v" + "-".join(s.zfill(2) for s in _VERSION.split("."))` seems rather tricky but works well. How about this? @lhoestq 


The CI is only failing because of the missing sections in the dataset card, and because of an issue with the CER metric that is unrelated to this PR
Title: Add docs for audio processing
Author: stevhliu (MEMBER)
State: closed
Created: 2021-11-05T23:07:59Z
Labels: documentation

This PR adds documentation for the `Audio` feature. It describes:

- The difference between loading `path` and `audio`, as well as use-cases/best practices for each of them.
- Resampling audio files with `cast_column`, and then calling `ds[0]["audio"]` to automatically decode and resample to the desired sampling rate.
- Resampling with `map`.

Preview [here](https://52969-250213286-gh.circle-artifacts.com/0/docs/_build/html/audio_process.html), let me know if I'm missing anything!

Comments:
Nice ! love it this way. I guess you can set this PR to "ready for review" ?

I guess we can merge this one now :)
Title: Add documentation about dataset viewer feature
Author: albertvillanova (MEMBER)
State: open
Created: 2021-11-05T08:11:19Z
Labels: enhancement

dataset-viewer

Add to the docs more details about the dataset viewer feature in the Hub.

CC: @julien-c

Comments:
In particular, include this somewhere in the docs: https://huggingface.co/docs/hub/datasets-viewer#access-the-parquet-files

See https://github.com/huggingface/hub-docs/issues/563
Title: Eventual Invalid Token Error at setup of private datasets
Author: albertvillanova (MEMBER)
State: closed
Created: 2021-11-04T18:50:45Z
Labels: bug

## Describe the bug
From time to time, there appear Invalid Token errors with private datasets:

- https://app.circleci.com/pipelines/github/huggingface/datasets/8520/workflows/d44629f2-4749-40f8-a657-50931d0b3434/jobs/52534
  ```
  ____________ ERROR at setup of test_load_streaming_private_dataset _____________
  ValueError: Invalid token passed!

  ____ ERROR at setup of test_load_streaming_private_dataset_with_zipped_data ____
  ValueError: Invalid token passed!
  
  =========================== short test summary info ============================
  ERROR tests/test_load.py::test_load_streaming_private_dataset - ValueError: I...
  ERROR tests/test_load.py::test_load_streaming_private_dataset_with_zipped_data
  ```

- https://app.circleci.com/pipelines/github/huggingface/datasets/8557/workflows/a8383181-ba6d-4487-9d0a-f750b6dcb936/jobs/52763
  ```
  ____ ERROR at setup of test_load_streaming_private_dataset_with_zipped_data ____
  [gw1] linux -- Python 3.6.15 /home/circleci/.pyenv/versions/3.6.15/bin/python3.6

  hf_api = <huggingface_hub.hf_api.HfApi object at 0x7f4899bab908>
  hf_token = 'vgNbyuaLNEBuGbgCEtSBCOcPjZnngJufHkTaZvHwkXKGkHpjBPwmLQuJVXRxBuaRzNlGjlMpYRPbthfHPFWXaaEDTLiqTTecYENxukRYVAAdpeApIUPxcgsowadkTkPj'
  zip_csv_path = PosixPath('/tmp/pytest-of-circleci/pytest-0/popen-gw1/data16/dataset.csv.zip')

      @pytest.fixture(scope="session")
      def hf_private_dataset_repo_zipped_txt_data_(hf_api: HfApi, hf_token, zip_csv_path):
          repo_name = "repo_zipped_txt_data-{}".format(int(time.time() * 10e3))
          hf_api.create_repo(token=hf_token, name=repo_name, repo_type="dataset", private=True)
          repo_id = f"{USER}/{repo_name}"
          hf_api.upload_file(
              token=hf_token,
              path_or_fileobj=str(zip_csv_path),
              path_in_repo="data.zip",
              repo_id=repo_id,
  >           repo_type="dataset",
          )

  tests/hub_fixtures.py:68:

  ...

  ValueError: Invalid token passed!
  =========================== short test summary info ============================
  ERROR tests/test_load.py::test_load_streaming_private_dataset_with_zipped_data
  ```

Comments:
Title: Fix code quality bug in riddle_sense dataset
Author: albertvillanova (MEMBER)
State: closed
Created: 2021-11-04T17:40:32Z
Labels: bug

## Describe the bug
```
datasets/riddle_sense/riddle_sense.py:36:21: W291 trailing whitespace
```

Comments:
To give more context: https://github.com/psf/black/issues/318. `black` doesn't treat this as a bug, but `flake8` does.
Title: Pin version exclusion for tensorflow incompatible with keras
Author: albertvillanova (MEMBER)
State: closed
Created: 2021-11-04T17:38:06Z

Once `tensorflow` version 2.6.2 is released:
- https://github.com/tensorflow/tensorflow/commit/c1867f3bfdd1042f694df7a9870be51ba80543cb
- https://pypi.org/project/tensorflow/2.6.2/

with the patch:
- tensorflow/tensorflow#52927

we can remove the temporary fix we introduced in:
- #3208

Fix #3209.

Comments:
Title: Add ACAV100M Dataset
Author: nateraw (CONTRIBUTOR)
State: open
Created: 2021-11-04T15:59:58Z
Labels: dataset request

vision

## Adding a Dataset
- **Name:** *ACAV100M*
- **Description:** *contains 100 million videos with high audio-visual correspondence, ideal for self-supervised video representation learning.*
- **Paper:** *https://arxiv.org/abs/2101.10803*
- **Data:** *https://github.com/sangho-vision/acav100m*
- **Motivation:** *The largest dataset (to date) for audio-visual learning.*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Comments:
Title: Sort files before loading
Author: lvwerra (MEMBER)
State: closed
Created: 2021-11-04T11:08:31Z
Labels: enhancement

When loading a dataset that consists of several files (e.g. `my_data/data_001.json`, `my_data/data_002.json` etc.) they are not loaded in order when using `load_dataset("my_data")`.

This could lead to counter-intuitive results if, for example, the data files are sorted by date or similar since they would appear in different order in the `Dataset`.

The straightforward solution is to sort the list of files alphabetically before loading them.

cc @lhoestq

Comments:
This will be fixed by https://github.com/huggingface/datasets/pull/3221
Title: Fix disable_nullable default value to False
Author: lhoestq (MEMBER)
State: closed
Created: 2021-11-04T10:52:06Z

Currently the `disable_nullable` parameter is not consistent across all dataset transforms. For example it is `False` in `map` but `True` in `flatten_indices`.

This creates unexpected behaviors like this
```python
from datasets import Dataset, concatenate_datasets

d1 = Dataset.from_dict({"a": [0, 1, 2, 3]})
d2 = d1.filter(lambda x: x["a"] < 2).flatten_indices()
d1.data.schema == d2.data.schema  # False
```
This can cause issues when concatenating datasets for example.

For consistency I set `disable_nullable` to `False` in `flatten_indices` and I fixed some docstrings

cc @SBrandeis

Comments:
Title: ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.15.1/datasets/wmt16/wmt16.py
Author: xiuzhilu (NONE)
State: closed
Created: 2021-11-04T10:47:26Z
Labels: dataset bug

when I use python examples/pytorch/translation/run_translation.py     --model_name_or_path examples/pytorch/translation/opus-mt-en-ro     --do_train     --do_eval     --source_lang en     --target_lang ro     --dataset_name wmt16     --dataset_config_name ro-en     --output_dir /tmp/tst-translation     --per_device_train_batch_size=4     --per_device_eval_batch_size=4     --overwrite_output_dir     --predict_with_generate to finetune translation model on huggingface, I get the issue"ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.15.1/datasets/wmt16/wmt16.py".But I can open the https://raw.githubusercontent.com/huggingface/datasets/1.15.1/datasets/wmt16/wmt16.py by using website. What should I do to solve the issue?

Comments:
Hi ! Do you have some kind of proxy in your browser that gives you access to internet ?

Maybe you're having this error because you don't have access to this URL from python ?

Hi，do you fixed this error?
I still have this issue when use "use_auth_token=True"

You don't need authentication to access those github hosted files
Please check that you can access this URL from your browser and also from your terminal
Title: CI error: Another metric with the same name already exists in Keras 2.7.0
Author: albertvillanova (MEMBER)
State: closed
Created: 2021-11-04T09:04:11Z
Labels: bug

## Describe the bug
Release of TensorFlow 2.7.0 contains an incompatibility with Keras. See:
- keras-team/keras#15579

This breaks our CI test suite: https://app.circleci.com/pipelines/github/huggingface/datasets/8493/workflows/055c7ae2-43bc-49b4-9f11-8fc71f35a25c/jobs/52363

Comments:
Title: Add Multidoc2dial Dataset
Author: sivasankalpp (CONTRIBUTOR)
State: closed
Created: 2021-11-03T20:48:31Z

This PR adds the MultiDoc2Dial dataset introduced in this [paper](https://arxiv.org/pdf/2109.12595v1.pdf )

Comments:
@songfeng cc

Hi @sivasankalpp, thanks for your PR.

There was a bug in TensorFlow/Keras. We have made a temporary fix in our master branch. Please, merge master into your PR branch, so that the CI tests pass.

```
git checkout multidoc2dial
git fetch upstream master
git merge upstream/master
```

Hi @albertvillanova, I have merged master into my PR branch. All tests are passing. 
Please take a look when you get a chance, thanks! 


Thanks for your feedback @lhoestq. We addressed your comments in the latest commit. Let us know if everything looks okay :)
Title: FileNotFoundError for TupleIE dataste
Author: arda-vianai (NONE)
State: closed
Created: 2021-11-03T14:56:55Z
Labels: bug

Hi,
`dataset = datasets.load_dataset('tuple_ie', 'all')`

returns a FileNotFound error. Is the data not available? 

Many thanks.

Comments:
@mariosasko @lhoestq Could you give me an update on how to load the dataset after the fix?
Thanks.

Hi @arda-vianai,

first, you can try:
```python
import datasets
dataset = datasets.load_dataset('tuple_ie', 'all', revision="master")
```
If this doesn't work, your version of `datasets` is missing some features that are required to run the dataset script, so install the master version with the following command:
```
pip install git+https://github.com/huggingface/datasets.git
```
and then:
```python
import datasets
dataset = datasets.load_dataset('tuple_ie', 'all')
```
should work (even without `revision`).

@mariosasko 
Thanks, it is working now. I actually did that before but I didn't restart the kernel. I restarted it and it works now. My bad!!!
Many thanks and great job!
-arda
Title: Updated: DaNE - updated URL for download
Author: MalteHB (CONTRIBUTOR)
State: closed
Created: 2021-11-03T12:55:13Z

It seems that DaNLP has updated their download URLs and it therefore also needs to be updated in here...

Comments:
Actually it looks like the old URL is still working, and it's also the one that is mentioned in https://github.com/alexandrainst/danlp/blob/master/docs/docs/datasets.md

What makes you think we should use the new URL ?

@lhoestq Sorry! I might have jumped to conclusions a bit too fast here... 

I was working in Google Colab and got an error that it was unable to use the URL. I then forked the project, updated the URL, ran it locally and it worked. I therefore assumed that my URL update fixed the issue, however, I see now that it might rather be a Google Colab issue... 

Still - this seems to be the official URL for downloading the dataset, and I think that it will be most beneficial to use. :-) 

It looks like they're using these new urls for their new datasets. Maybe let's change to the new URL in case the old one stops working at one point. Thanks
Title: Add mIoU metric
Author: NielsRogge (CONTRIBUTOR)
State: closed
Created: 2021-11-03T08:42:32Z
Labels: enhancement

**Is your feature request related to a problem? Please describe.**
Recently, some semantic segmentation models were added to HuggingFace Transformers, including [SegFormer](https://huggingface.co/transformers/model_doc/segformer.html) and [BEiT](https://huggingface.co/transformers/model_doc/beit.html).

Semantic segmentation (which is the task of labeling every pixel of an image with a corresponding class) is typically evaluated using the Mean Intersection and Union (mIoU). Together with the upcoming Image Feature, adding this metric could be very handy when creating example scripts to fine-tune any Transformer-based model on a semantic segmentation dataset.

An implementation can be found [here](https://github.com/open-mmlab/mmsegmentation/blob/504965184c3e6bc9ec43af54237129ef21981a5f/mmseg/core/evaluation/metrics.py#L132) for instance.

Comments:
Resolved via https://github.com/huggingface/datasets/pull/3745.
Title: Add GSM8K dataset
Author: NielsRogge (CONTRIBUTOR)
State: closed
Created: 2021-11-03T08:36:44Z
Labels: dataset request

## Adding a Dataset
- **Name:** GSM8K (short for Grade School Math 8k)
- **Description:** GSM8K is a dataset of 8.5K high quality linguistically diverse grade school math word problems created by human problem writers.
- **Paper:** https://openai.com/blog/grade-school-math/
- **Data:** https://github.com/openai/grade-school-math
- **Motivation:** The dataset is useful to investigate the reasoning abilities of large Transformer models, such as GPT-3.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Comments:
Closed via https://github.com/huggingface/datasets/pull/4103
Title: Fix optimized encoding for arrays
Author: lhoestq (MEMBER)
State: closed
Created: 2021-11-02T15:55:53Z

Hi !

#3124 introduced a regression that made the benchmarks CI fail because of a bad array comparison when checking the first encoded element. This PR fixes this by making sure that encoding is applied on all sequence types except lists.

cc @eladsegal fyi (no big deal)

Comments:
Title: Multiprocessing filter/map (tests) not working on Windows
Author: BramVanroy (CONTRIBUTOR)
State: open
Created: 2021-11-01T15:36:08Z
Labels: bug

While running the tests, I found that the multiprocessing examples fail on Windows, or rather they do not complete: they cause a deadlock. I haven't dug deep into it, but they do not seem to work as-is. I currently have no time to tests this in detail but at least the tests seem not to run correctly (deadlocking).

## Steps to reproduce the bug
```shell
pytest tests/test_arrow_dataset.py -k "test_filter_multiprocessing"
pytest tests/test_arrow_dataset.py -k "test_map_multiprocessing"
```

## Expected results
The functionality to work on all platforms.

## Actual results
Deadlock.

## Environment info
- `datasets` version: 1.14.1.dev0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.9.2, also tested with 3.7.9
- PyArrow version: 4.0.1

Comments:
Title: Dataset viewer issue for '*compguesswhat*'
Author: benotti (NONE)
State: closed
Created: 2021-11-01T14:16:49Z
Labels: streaming

## Dataset viewer issue for '*compguesswhat*'

**Link:** https://huggingface.co/datasets/compguesswhat

File not found

Am I the one who added this dataset ? No

Comments:
```python
>>> import datasets
>>> dataset = datasets.load_dataset('compguesswhat', name='compguesswhat-original',split='train', streaming=True)
>>> next(iter(dataset))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 497, in __iter__
    for key, example in self._iter():
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 494, in _iter
    yield from ex_iterable
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.9/site-packages/datasets/iterable_dataset.py", line 87, in __iter__
    yield from self.generate_examples_fn(**self.kwargs)
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/compguesswhat/4d08b9e0a8d1cf036c9626c93be4a759fdd9fcce050ea503ea14b075e830c799/compguesswhat.py", line 251, in _generate_examples
    with gzip.open(filepath) as in_file:
  File "/home/slesage/.pyenv/versions/3.9.6/lib/python3.9/gzip.py", line 58, in open
    binary_file = GzipFile(filename, gz_mode, compresslevel)
  File "/home/slesage/.pyenv/versions/3.9.6/lib/python3.9/gzip.py", line 173, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'zip://compguesswhat-original/0.2.0/compguesswhat.train.jsonl.gz::https://www.dropbox.com/s/l0nc13udml6vs0w/compguesswhat-original.zip?dl=1'
```

It's an issue with the streaming mode. Note that normal mode is used by the dataset viewer when streaming is failing, but only for the smallest datasets. This dataset is above the limit, hence the error.

Same case as https://github.com/huggingface/datasets/issues/3186#issuecomment-1096549774.

cc @huggingface/datasets 

There is an issue with the URLs of their data files: https://www.dropbox.com/s/l0nc13udml6vs0w/compguesswhat-original.zip?dl=1
> Dropbox Error: That didn't work for some reason

Error reported to their repo:
- https://github.com/CompGuessWhat/compguesswhat.github.io/issues/1

Closed by:
- #4968
Title: combination of shuffle and filter results in a bug
Author: rabeehk (CONTRIBUTOR)
State: closed
Created: 2021-11-01T13:07:29Z
Labels: bug

## Describe the bug
Hi,
I would like to shuffle a dataset, then filter it based on each existing label. however, the combination of `filter`, `shuffle` seems to results in a bug. In the minimal example below, as you see in the filtered results, the filtered labels are not unique, meaning filter has not worked. Any suggestions as a temporary fix is appreciated @lhoestq.

Thanks.
 Best regards
Rabeeh 

## Steps to reproduce the bug
```python
import numpy as np
import datasets 

datasets = datasets.load_dataset('super_glue', 'rte', script_version="master")
shuffled_data = datasets["train"].shuffle(seed=42)
for label in range(2):
    print("label ", label)
    data = shuffled_data.filter(lambda example: int(example['label']) == label)
    print("length ", len(data), np.unique(data['label']))
```

## Expected results
Filtering per label, should only return the data with that specific label.

## Actual results
As you can see, filtered data per label, has still two labels of [0, 1]
```
label  0
length  1249 [0 1]
label  1
length  1241 [0 1]
```
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1 
- Platform: linux 
- Python version: 3.7.11 
- PyArrow version: 5.0.0

Comments:
I cannot reproduce this on master and pyarrow==4.0.1.


Hi ! There was a regression in `datasets` 1.12 that introduced this bug. It has been fixed in #3019 in 1.13

Can you try to update `datasets` and try again ?

Thanks a lot, fixes with 1.13
Title: conll2003 incorrect label explanation
Author: BramVanroy (CONTRIBUTOR)
State: closed
Created: 2021-11-01T11:03:30Z
Labels: bug

In the [conll2003](https://huggingface.co/datasets/conll2003#data-fields) README, the labels are described as follows

> - `id`: a `string` feature.
> - `tokens`: a `list` of `string` features.
> - `pos_tags`: a `list` of classification labels, with possible values including `"` (0), `''` (1), `#` (2), `$` (3), `(` (4).
> - `chunk_tags`: a `list` of classification labels, with possible values including `O` (0), `B-ADJP` (1), `I-ADJP` (2), `B-ADVP` (3), `I-ADVP` (4).
> - `ner_tags`: a `list` of classification labels, with possible values including `O` (0), `B-PER` (1), `I-PER` (2), `B-ORG` (3), `I-ORG` (4) `B-LOC` (5), `I-LOC` (6) `B-MISC` (7), `I-MISC` (8).

First of all, it would be great if we can get a list of ALL possible pos_tags.

Second, the chunk tags labels cannot be correct. The description says the values go from 0 to 4 whereas the data shows values from at least 11 to 21 and 0.

EDIT: not really a bug, sorry for mistagging.

Comments:
Hi @BramVanroy,

since these fields are of type `ClassLabel` (you can check this with `dset.features`), you can inspect the possible values with:
```python
dset.features[field_name].feature.names  # .feature because it's a sequence of labels
```

and to find the mapping between names and integers, use: 
```python
dset.features[field_name].feature.int2str(value_or_values_list)  # map integer value to string value
# or
dset.features[field_name].feature.str2int(value_or_values_list)  # map string value to integer value
```
Title: conll2002 issues
Author: BramVanroy (CONTRIBUTOR)
State: closed
Created: 2021-11-01T09:49:24Z
Labels: dataset-viewer

**Link:** https://huggingface.co/datasets/conll2002

The dataset viewer throws a server error when trying to preview the dataset. 

```
Message:       Extraction protocol 'train' for file at 'https://raw.githubusercontent.com/teropa/nlp/master/resources/corpora/conll2002/esp.train' is not implemented yet
```

In addition, the "point of contact" has encoding issues and does not work when clicked.

Am I the one who added this dataset ? No, @lhoestq  did

Comments:
Hi ! Thanks for reporting :)

This is related to https://github.com/huggingface/datasets/issues/2742, I'm working on it. It should fix the viewer for around 80 datasets.


Ah, hadn't seen that sorry.

The scrambled "point of contact" is a separate issue though, I think.

@lhoestq The "point of contact" is still an issue.

It will be fixed in https://github.com/huggingface/datasets/pull/3274, thanks
Title: 7z dataset preview not implemented?
Author: Kirili4ik (NONE)
State: closed
Created: 2021-10-30T20:18:27Z
Labels: dataset-viewer

## Dataset viewer issue for dataset 'samsum'

**Link:** https://huggingface.co/datasets/samsum

Server Error
Status code:   400
Exception:     NotImplementedError
Message:       Extraction protocol '7z' for file at 'https://arxiv.org/src/1911.12237v2/anc/corpus.7z' is not implemented yet

Comments:
It's a bug in the dataset viewer: the dataset cannot be downloaded in streaming mode, but since the dataset is relatively small, the dataset viewer should have fallback to normal mode. Working on a fix.

Fixed. https://huggingface.co/datasets/samsum/viewer/samsum/train

<img width="1563" alt="Capture d’écran 2022-04-12 à 13 47 45" src="https://user-images.githubusercontent.com/1676121/162953339-cd8922d7-9037-408b-b896-eac1af0bb54f.png">

Thanks for reporting!
Title: RONEC v2
Author: dumitrescustefan (CONTRIBUTOR)
State: closed
Created: 2021-10-30T10:50:03Z

Hi, as we've recently finished with the new RONEC (Romanian Named Entity Corpus), we'd like to update the dataset here as well. It's actually essential as links to V1 are no longer valid. 

In reality we'd like to replace completely v1, as v2 is a full re-annotation of v1 with additional data (up to 2x size vs v1).

I've run the make style and all the dummy and real data test, and they passed.

I hope it's okay to merge the new RONEC v2 in the datasets. 

Thanks!

Comments:
@lhoestq Thanks for the review. I totally understand what you are saying. Normally, I would definitely agree with you, but in this particular case, the quality of v1 is poor, and the dataset itself is small (at the time we created v1 it was the only RO NER dataset, and its size was limited by the available resources). 

This is why we worked to build a larger one, with much better inter-annotator agreement. Fact is, models trained on v1 will be of very low quality and I would not recommend to anybody to use/do that. That's why I'd strongly suggest we replace v1 with v2, and kindof make v1 vanish :)  

What do you think? If you insist on having v1 accessible, I'll add the required code. Thanks!



Ok I see ! I think it's fine then, no need to re-add V1
Title: Don't memoize strings when hashing since two identical strings may have different python ids
Author: lhoestq (MEMBER)
State: closed
Created: 2021-10-29T16:26:17Z

When hashing an object that has several times the same string, the hashing could return a different hash if the identical strings share the same python `id()` or not.

Here is an example code that shows how the issue can affect the caching:
```python
import json
import pyarrow as pa
from datasets.features import Features
from datasets.fingerprint import Hasher

schema = pa.schema([pa.field("some_string", pa.string()), pa.field("another_string", pa.string())])
features_from_schema = Features.from_arrow_schema(schema)
Hasher.hash(features_from_schema) # dffa9dca9a73fd8c

features_dict = json.loads('{"some_string": {"dtype": "string", "id": null, "_type": "Value"}, "another_string": {"dtype": "string", "id": null, "_type": "Value"}}')
features_from_json = Features.from_dict(features_dict)
Hasher.hash(features_from_json) # 3812e76b15e6420e

features_from_schema == features_from_json # True
```

This is because in `features_dict`, some strings like "dtype" are repeated but don't share the same id, contrary to the ones in `features_from_schema`.

I fixed that by disabling memoization for strings.

This could be optimized in the future by implementing a smarter memoization with a special handling for strings.

Comments:
This change slows down the hash computation a little bit but from my tests it doesn't look too impactful. So I think it's fine to merge this.
Title: fix label mapping
Author: VictorSanh (CONTRIBUTOR)
State: closed
Created: 2021-10-29T14:42:24Z

Fixing label mapping for hlgd.
0 correponds to same event and 1 corresponds to different event
<img width="642" alt="Capture d’écran 2021-10-29 à 10 39 58 AM" src="https://user-images.githubusercontent.com/16107619/139454810-1f225e3d-ad48-44a8-b8b1-9205c9533839.png">
<img width="638" alt="Capture d’écran 2021-10-29 à 10 40 09 AM" src="https://user-images.githubusercontent.com/16107619/139454813-93066a3c-7d33-4f56-b133-2f1a7661e438.png">
nt

Comments:
heck, test failings. moving to draft. will come back to this later today hopefully

Thanks for fixing this :)
I just updated the dataset_infos.json and added the missing `pretty_name` tag to the dataset card

thank you @lhoestq! running around as always it felt through as a lower priority...
Title: More control over TQDM when using map/filter with multiple processes
Author: BramVanroy (CONTRIBUTOR)
State: closed
Created: 2021-10-29T11:56:16Z
Labels: enhancement

It would help with the clutter in my terminal if tqdm is only shown for rank 0 when using `num_proces>0` in the map and filter methods of datasets.

```python
dataset.map(lambda examples: tokenize(examples["text"]), batched=True, num_proc=6)
```

The above snippet leads to a lot of TQDM bars and depending on your terminal, these will not overwrite but keep pushing each other down.

```
 #0:   0%|          | 0/13 [00:00<?, ?ba/s]
 #1:   0%|          | 0/13 [00:00<?, ?ba/s]
 #2:   0%|          | 0/13 [00:00<?, ?ba/s]
 #3:   0%|          | 0/13 [00:00<?, ?ba/s]
 #4:   0%|          | 0/13 [00:00<?, ?ba/s]
 #5:   0%|          | 0/13 [00:00<?, ?ba/s]
 #0:   8%|          | 1/13 [00:00<?, ?ba/s]
 #1:   8%|          | 1/13 [00:00<?, ?ba/s]
...
```

Instead, it would be welcome if we had the option to only show the progress of rank 0.

Comments:
Hi,

It's hard to provide an API that would cover all use-cases with tqdm in this project.

However, you can make it work by defining a custom decorator (a bit hacky tho) as follows:
```python
import datasets

def progress_only_on_rank_0(func):
    def wrapper(*args, **kwargs):
        rank = kwargs.get("rank")
        disable_tqdm = kwargs.get("disable_tqdm", False)
        disable_tqdm = True if rank is not None and rank > 0 else disable_tqdm
        kwargs["disable_tqdm"] = disable_tqdm
        return func(*args, **kwargs)
    return wrapper
    
datasets.Dataset._map_single = progress_only_on_rank_0(datasets.Dataset._map_single)
``` 

EDIT: Ups, closed by accident.

Thanks for the provided links. `Trainer` requires this for training in multi-node distributed setting. However, `Dataset.map` doesn't support that yet.

Do you have an API for this in mind? `Dataset.map` is already bloated with the arguments, so IMO it's not a good idea to add a new arg there.



Inspiration may be found at `transformers`.

https://github.com/huggingface/transformers/blob/4a394cf53f05e73ab9bbb4b179a40236a5ffe45a/src/transformers/trainer.py#L1231-L1233

To get unique IDs for each worker, see https://stackoverflow.com/a/10192611/1150683
Title: OpenSLR dataset: update generate_examples to properly extract data for SLR83
Author: tyrius02 (CONTRIBUTOR)
State: closed
Created: 2021-10-29T00:59:27Z

Fixed #3168.

The SLR38 indices are CSV files and there wasn't any code in openslr.py to process these files properly. The end result was an empty table.

I've added code to properly process these CSV files.

Comments:
Also fix #3125.
Title: Add docs for `to_tf_dataset`
Author: stevhliu (MEMBER)
State: closed
Created: 2021-10-28T20:55:22Z
Labels: documentation

This PR adds some documentation for new features released in v1.13.0, with the main addition being `to_tf_dataset`:

- Show how to use `to_tf_dataset` in the tutorial, and move `set_format(type='tensorflow'...)` to the Process section (let me know if I'm missing anything @Rocketknight1 😅).
- Add an example for loading dataset from multiple zipped CSV files to the Load section.
- Add an example for removing columns for an `IterableDataset`.
- Add graphic for visualizing streaming.

Comments:
This looks great, thank you!

Thanks !

For some reason the new GIF is 6MB, which is a bit heavy for an image on a website. The previous one was around 200KB though which is perfect. For a good experience we usually expect images to be less than 500KB - otherwise for users with poor connection it takes too long to load. Could you try to reduce its size ? Than I think we can merge :)
Title: Asserts replaced by exceptions (huggingface#3171)
Author: joseporiolayats (CONTRIBUTOR)
State: closed
Created: 2021-10-28T11:55:45Z

I've replaced two asserts with their proper exceptions following the guidelines described in issue #3171 by following the contributing guidelines.

PS: This is one of my first PRs, hoping I don't break anything!

Comments:
Your first PR went smoothly, well done!
You are welcome to continue contributing to this project.
Gràcies, @joseporiolayats! 😉
Title: Raise exceptions instead of using assertions for control flow
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2021-10-27T18:26:52Z
Labels: good first issue

Motivated by https://github.com/huggingface/transformers/issues/12789 in Transformers, one welcoming change would be replacing assertions with proper exceptions. The only type of assertions we should keep are those used as sanity checks.

Currently, there is a total of 87 files with the `assert` statements (located under `datasets` and `src/datasets`), so when working on this, to manage the PR size, only modify 4-5 files at most before submitting a PR.

Comments:
Adding the remaining tasks for this issue to help new code contributors.  
$ cd src/datasets && ack assert -lc 
- [x] commands/convert.py:1
- [x] arrow_reader.py:3
- [x] load.py:7
- [x] utils/py_utils.py:2
- [x] features/features.py:9
- [x] arrow_writer.py:7
- [x] search.py:6
- [x] table.py:1
- [x] metric.py:3
- [x] tasks/image_classification.py:1
- [x] arrow_dataset.py:17
- [x] fingerprint.py:6
- [x] io/json.py:1
- [x] io/csv.py:1

Hi all,
I am interested in taking up `fingerprint.py`, `search.py`, `arrow_writer.py` and `metric.py`. Will raise a PR soon!

Let me look into `arrow_dataset.py`, `table.py`, `data_files.py` & `features.py` 

All the tasks are completed for this issue. This can be closed.
Title: OpenSLR/83 is empty
Author: tyrius02 (CONTRIBUTOR)
State: closed
Created: 2021-10-26T19:42:21Z
Labels: bug

## Describe the bug
As the summary says, openslr / SLR83 / train is empty.

The dataset returned after loading indicates there are **zero** rows. The correct number should be **17877**.
## Steps to reproduce the bug
```python
import datasets


datasets.load_dataset('openslr', 'SLR83')
```

## Expected results
```
DatasetDict({
    train: Dataset({
        features: ['path', 'audio', 'sentence'],
        num_rows: 17877
    })
})
```
## Actual results
```
DatasetDict({
    train: Dataset({
        features: ['path', 'audio', 'sentence'],
        num_rows: 0
    })
})
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.14.1.dev0 (master HEAD)
- Platform: Ubuntu 20.04
- Python version: 3.7.10
- PyArrow version: 3.0.0

Comments:
Hi @tyrius02, thanks for reporting. I see you self-assigned this issue: are you working on this?

@albertvillanova Yes. Figured I introduced the broken config, I should fix it too.

I've got it working, but I'm struggling with one of the tests. I've started a PR so I/we can work through it.

Looks like the tests all passed on the PR.
Title: bookcorpusopen no longer works
Author: lucadiliello (CONTRIBUTOR)
State: closed
Created: 2021-10-26T16:06:15Z
Labels: bug

## Describe the bug

When using the latest version of datasets (1.14.0), I cannot use the `bookcorpusopen` dataset. The process blocks always around `9924 examples [00:06, 1439.61 examples/s]` when preparing the dataset. I also noticed that after half an hour the process is automatically killed because of the RAM usage (the machine has 1TB of RAM...).

This did not happen with 1.4.1.
I tried also `rm -rf ~/.cache/huggingface` but did not help.
Changing python version between 3.7, 3.8 and 3.9 did not help too.

## Steps to reproduce the bug
```python
import datasets
d = datasets.load_dataset('bookcorpusopen')
```

## Expected results
A clear and concise description of the expected results.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.14.0
- Platform: Linux-5.4.0-1054-aws-x86_64-with-glibc2.27
- Python version: 3.9.7
- PyArrow version: 4.0.1

Comments:
Hi ! Thanks for reporting :) I think #3280 should fix this

I tried with the latest changes from #3280 on google colab and it worked fine :)
We'll do a new release soon, in the meantime you can use the updated version with:
```python
load_dataset("bookcorpusopen", revision="master")
```

Fixed by #3280.
Title: Deprecate prepare_module
Author: albertvillanova (MEMBER)
State: closed
Created: 2021-10-26T15:28:24Z

In version 1.13, `prepare_module` was deprecated.

This PR adds a deprecation warning and removes it from all the library, using `dataset_module_factory` or `metric_module_factory` instead.

Fix #3165.

Comments:
Sounds good, thanks !