rIsHu009/github-issues-updated huggingface dataset:
Title: Use standard open-domain validation split in nq_open
Author: craffel (CONTRIBUTOR)
State: closed
Created: 2021-10-05T14:19:27Z

The nq_open dataset originally drew the validation set from this file:
https://github.com/google-research-datasets/natural-questions/blob/master/nq_open/NQ-open.efficientqa.dev.1.1.sample.jsonl
However, that's the dev set used specifically and only for the efficientqa competition, and it's not the same dev set as is used in every open-domain question answering paper (including the Lee et al paper that introduced the open-domain variant of NQ, cited at the top of the dataset file). This PR changes nq_open to use the standard validation split and bumps the version to 2.0.0 since this is a breaking change.

Comments:
I had to run datasets-cli with --ignore_verifications the first time since it was complaining about a missing file, but now it runs without that flag fine. I moved dummy_data.zip to the new folder, but also had to modify the filename of the test file in the zip (should I not have done that?). Finally, I added the pretty name tag.

Great, thanks for the help.
Title: Properly install ruamel-yaml for windows CI
Author: lhoestq (MEMBER)
State: closed
Created: 2021-10-05T11:51:15Z


Comments:
@lhoestq I would say this does not "properly" install `ruamel-yaml`, but the contrary, you overwrite the previous version without desinstalling it first.

According to `pip` docs:
>  This can break your system if the existing package is of a different version or was installed with a different package manager!

Note that our case fulfills both conditions:
- the installing version (`0.17.16`) is different from the existing one (`0.15.87`)
- you are installing using `pip` (`setuptools`), whereas the exisitng version was installed using `distutils`

That is why I did not fix the issue this way, made a hotfix pinning `huggingface_hub` (#3025), while looking for a permanent solution for the issue.

Yea I did this because we need the latest version of `huggingface_hub` for #2986 and because I didn't want to ssh to the windows worker x)
Maybe it can be fixed by installing it with conda - let me try

Oh yea it may not work since it was first installed with distutils
Title: Resolve data_files by split name
Author: lhoestq (MEMBER)
State: closed
Created: 2021-10-05T10:24:36Z

This issue is about discussing the default behavior when someone loads a dataset that consists in data files. For example:
```python
load_dataset("lhoestq/demo1")
```
should return two splits "train" and "test" since the dataset repostiory is like
```
data/
├── train.csv
└── test.csv
```
Currently it returns only one split "train" which contains the data of both files


I started playing with this idea on this branch btw: `resolve-data_files-by-split-name`
Basically the idea is that if you named you data files after split names then the default pattern is
```python
{
    "train": ["*train*"],
    "test": ["*test*"],
    "validation": ["*dev*", "valid"],
}
```
otherwise it's
```python
{
    "train": ["*"]
}
```

Let me know what you think !
cc @albertvillanova @LysandreJik @vblagoje

Comments:
Awesome @lhoestq I like the proposal and it works great on my JSON community dataset. Here is the [log](https://gist.github.com/vblagoje/714babc325bcbdd5de579fd8e1648892). 

From my discussion with @borisdayma it would be more general the files match if their paths contains the split name - not only if the filename contains the split name. For example for a dataset like this:
```
train/
└── data.csv
test/
└── data.csv
```

But IMO the default should be 
```
data/
├── train.csv
└── test.csv
```
because it allows people to have other directories if they have different subsets of their data (different configurations, not splits)

I just created a PR for this at https://github.com/huggingface/datasets/pull/3221, let me know what you think :)
Title: Windows test suite fails
Author: albertvillanova (MEMBER)
State: closed
Created: 2021-10-05T08:46:46Z
Labels: bug

## Describe the bug

There is an error during installation of tests dependencies for Windows: https://app.circleci.com/pipelines/github/huggingface/datasets/7981/workflows/9b6a0114-2b8e-4069-94e5-e844dbbdba4e/jobs/49206

```
ERROR: Cannot uninstall 'ruamel-yaml'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
```

Comments:
Title: MeDAL dataset: Add further description and update download URL
Author: xhluca (NONE)
State: closed
Created: 2021-10-05T00:13:28Z

Added more details in the following sections:
* Dataset Structure
* Data Instances
* Data Splits
* Source Data
* Annotations
* Discussions of Biases
* LIcensing Information

Comments:
@lhoestq I'm a bit confused by the error message. I haven't touched the YAML code at all - do you have any insight on that?

I just added the missing `pretty_name` tag in the YAML - sorry about that ;)

Thanks! Seems like it did the trick since the tests are passing. Let me know if there's anything else I can do in this PR!

It's all good thank you :)

merging !
Title: Add a metric for the MATH dataset (competition_math).
Author: hacobe (CONTRIBUTOR)
State: closed
Created: 2021-10-04T16:52:16Z

This metric computes accuracy for the MATH dataset (https://arxiv.org/abs/2103.03874) after canonicalizing the prediction and the reference (e.g., converting "1/2" to "\\\\frac{1}{2}").

Comments:
I believe the only failed test related to this PR is tests/test_metric_common.py::LocalMetricTest::test_load_metric_competition_math. It gives the following error:

ImportError: To be able to use this dataset, you need to install the following dependencies['math_equivalence'] using 'pip install git+https://github.com/hendrycks/math.git' for instance'

It fails along with (these fail with ImportError as well):
test_load_metric_bertscore
test_load_metric_bleurt
test_load_metric_comet
test_load_metric_coval

Let me know if there is anything I need to change.

Hi ! The script looks all good thanks :)

To fix the CI you just need to merge `master` into your branch
```
git fetch upstream/master
git merge upstream/master
```

Then you also need to add `math_equivalence` to the list of git packages installed for the tests in `additional-tests-requirements.txt`
https://github.com/huggingface/datasets/blob/ba831e4bcd175ae3d52afbf7d12c4f625bf541b0/additional-tests-requirements.txt#L1-L3

I ran:

git fetch upstream
git merge upstream/master

And I also added  math_equivalence to the list of git packages installed for the tests in additional-tests-requirements.txt

tests/test_metric_common.py fails with the same errors as before. tests/test_dataset_cards.py also fails, but it doesn't look related to this PR (it's an issue datasets/ami/README.md).

@lhoestq Anything else I can do? I re-merged again and am getting the same test failures as described in the previous comment.
Title: Fix filter leaking
Author: lhoestq (MEMBER)
State: closed
Created: 2021-10-04T15:42:58Z

If filter is called after using a first transform `shuffle`, `select`, `shard`, `train_test_split`, or `filter`, then it could not work as expected and return examples from before the first transform. This is because the indices mapping was not taken into account when saving the indices to keep when doing the filtering

Affected versions: 1.12.0 and 1.12.1

This should fix #3010

Comments:
Title: Support multiple zipped CSV data files
Author: albertvillanova (MEMBER)
State: open
Created: 2021-10-04T15:16:59Z
Labels: enhancement

As requested by @lewtun, support loading multiple zipped CSV data files.

```python
from datasets import load_dataset

url = "https://domain.org/filename.zip"
data_files = {"train": "train_filename.csv", "test": "test_filename.csv"}
dataset = load_dataset("csv", data_dir=url, data_files=data_files)
```

Comments:
@lhoestq I would like to draw your attention to the proposed API by @lewtun, using `data_dir` to pass the ZIP URL.

I'm not totally convinced with this... What do you think?

Maybe we could discuss other approaches...

One brainstorming idea: what about using URL chaining with the hop operator in `data_files`?

`data_dir` is currently exclusively used for manually downloaded data.

Maybe we can have an API that only uses data_files as you are suggesting, using URL chaining ?
```python
from datasets import load_dataset

url = "https://domain.org/filename.zip"
data_files = {"train": "zip://train_filename.csv::" + url, "test": "zip://test_filename.csv::" + url}
dataset = load_dataset("csv", data_files=data_files)
```

URL chaining is used by `fsspec` to get access to files in nested filesystems of any kind. Since `fsspec` is being used by `pandas`, `dask` and also extensively by `datasets` I think it would be nice to use it here too

URL chaining sounds super nice to me! And it's also a nice way to leverage the same concepts we currently have in the docs around `fsspec` :)
Title: Improve `get_dataset_infos`?
Author: severo (COLLABORATOR)
State: closed
Created: 2021-10-04T09:47:04Z
Labels: question

dataset-viewer

Using the dedicated function `get_dataset_infos` on a dataset that has no dataset-info.json file returns an empty info:

```
>>> from datasets import get_dataset_infos
>>> get_dataset_infos('wit')
{}
```

While it's totally possible to get it (regenerate it) with:

```
>>> from datasets import load_dataset_builder
>>> builder = load_dataset_builder('wit')
>>> builder.info
DatasetInfo(description='Wikipedia-based Image Text (WIT) Dataset is a large multimodal multilingual dataset. WIT is composed of a curated set\n of 37.6 million entity rich image-text examples with 11.5 million unique images across 108 Wikipedia languages. Its\n size enables WIT to be used as a pretraining dataset for multimodal machine learning models.\n', citation='@article{srinivasan2021wit,\n  title={WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning},\n  author={Srinivasan, Krishna and Raman, Karthik and Chen, Jiecao and Bendersky, Michael and Najork, Marc},\n  journal={arXiv preprint arXiv:2103.01913},\n  year={2021}\n}\n', homepage='https://github.com/google-research-datasets/wit', license='', features={'b64_bytes': Value(dtype='string', id=None), 'embedding': Sequence(feature=Value(dtype='float64', id=None), length=-1, id=None), 'image_url': Value(dtype='string', id=None), 'metadata_url': Value(dtype='string', id=None), 'original_height': Value(dtype='int32', id=None), 'original_width': Value(dtype='int32', id=None), 'mime_type': Value(dtype='string', id=None), 'caption_attribution_description': Value(dtype='string', id=None), 'wit_features': Sequence(feature={'language': Value(dtype='string', id=None), 'page_url': Value(dtype='string', id=None), 'attribution_passes_lang_id': Value(dtype='string', id=None), 'caption_alt_text_description': Value(dtype='string', id=None), 'caption_reference_description': Value(dtype='string', id=None), 'caption_title_and_reference_description': Value(dtype='string', id=None), 'context_page_description': Value(dtype='string', id=None), 'context_section_description': Value(dtype='string', id=None), 'hierarchical_section_title': Value(dtype='string', id=None), 'is_main_image': Value(dtype='string', id=None), 'page_changed_recently': Value(dtype='string', id=None), 'page_title': Value(dtype='string', id=None), 'section_title': Value(dtype='string', id=None)}, length=-1, id=None)}, post_processed=None, supervised_keys=None, task_templates=None, builder_name='wit', config_name='default', version=0.0.0, splits=None, download_checksums=None, download_size=None, post_processing_size=None, dataset_size=None, size_in_bytes=None)
```

Should we test if info is empty, and in that case regenerate it? Or always generate it?

Comments:
To keeps things simple maybe we should use `load_dataset_builder` in `get_dataset_infos`.
`load_dataset_builder` instantiates a builder and runs the _infos() method in order to give you the most up-to-date infos, even if the dataset_infos.json is outdated or missing.
Title: Replace item with float in metrics
Author: albertvillanova (MEMBER)
State: closed
Created: 2021-10-04T09:45:28Z

As pointed out by @mariosasko in #3001, calling `float()` instad of `.item()` is faster.

Moreover, it might avoid potential issues if any of the third-party functions eventually returns a `float` instead of an `np.float64`.

Related to #3001.

Comments:
Title: load_dataset_builder should error if "name" does not exist?
Author: severo (COLLABORATOR)
State: open
Created: 2021-10-04T09:20:46Z
Labels: bug

dataset-viewer

```
import datasets as ds
builder = ds.load_dataset_builder('sent_comp', name="doesnotexist")
builder.info.config_name
```

returns

```
'doesnotexist'
```

Shouldn't it raise an error instead?

For this dataset, the only valid values for `name` should be: `"default"` or `None` (ie. argument not passed)

Comments:
Yes I think it should raise an error. Currently it looks like it instantiates a custom configuration with the name given by the user:
https://github.com/huggingface/datasets/blob/ba27ce33bf568374cf23a07669fdd875b5718bc2/src/datasets/builder.py#L391-L397
Title: DatasetDict.filter and Dataset.filter crashes with any "fn_kwargs" argument
Author: DrMatters (NONE)
State: closed
Created: 2021-10-04T00:49:29Z
Labels: bug

## Describe the bug
The ".filter" method of DatasetDict or Dataset objects fails when passing any "fn_kwargs" argument

## Steps to reproduce the bug
```python
import datasets

example_dataset = datasets.Dataset.from_dict({"a": {1, 2, 3, 4}})

def filter_value(example, value):
    return example['a'] == value

filtered = example_dataset.filter(filter_value, fn_kwargs={'value': 3})
```

## Expected results
`filtered` is a dataset containing {"a": {3}}

## Actual results

> Traceback (most recent call last):
>   File "C:\Users\qsemi\Documents\git\nlp_experiments\gpt_celebrity\src\test_faulty_filter.py", line 8, in <module>
>     filtered = example_dataset.filter(filter_value, fn_kwargs={'value': 3})
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 185, in wrapper
>     out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\fingerprint.py", line 398, in wrapper
>     out = func(self, *args, **kwargs)
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 2169, in filter
>     indices = self.map(
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 1686, in map
>     return self._map_single(
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 185, in wrapper
>     out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\fingerprint.py", line 398, in wrapper
>     out = func(self, *args, **kwargs)
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 2048, in _map_single
>     batch = apply_function_on_filtered_inputs(
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 1939, in apply_function_on_filtered_inputs
>     function(*fn_args, effective_indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
> TypeError: get_indices_from_mask_function() got an unexpected keyword argument 'value'

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Windows-10-10.0.19042-SP0
- Python version: 3.9.7
- PyArrow version: 5.0.0

Comments:
Hi @DrMatters, thanks for reporting.

This issue was fixed 14 days ago: #2950.

Currently, the fix is only in the master branch and will be made available in our next library release.

In the meantime, you can incorporate the fix by installing datasets from the master branch:
```shell
pip install -U git+ssh://git@github.com/huggingface/datasets.git@master#egg=datasest
```
or
```shell
pip install -U git+https://github.com/huggingface/datasets.git@master#egg=datasets
```

Thanks, sorry for bothering
Title: LexGLUE: A Benchmark Dataset for Legal Language Understanding in English.
Author: iliaschalkidis (CONTRIBUTOR)
State: closed
Created: 2021-10-03T10:03:25Z

Inspired by the recent widespread use of the GLUE multi-task benchmark NLP dataset (Wang et al., 2018), the subsequent more difficult SuperGLUE (Wang et al., 2019), other previous multi-task NLP benchmarks (Conneau and Kiela, 2018; McCann et al., 2018), and similar initiatives in other domains (Peng et al., 2019), we introduce the Legal General Language Understanding Evaluation (LexGLUE) benchmark, a benchmark dataset to evaluate the performance of NLP methods in legal tasks. LexGLUE is based on seven existing legal NLP datasets, selected using criteria largely from SuperGLUE.

As in GLUE and SuperGLUE (Wang et al., 2019b,a), one of our goals is to push towards generic (or ‘foundation’) models that can cope with multiple NLP tasks, in our case legal NLP tasks possibly with limited task-specific fine-tuning. Another goal is to provide a convenient and informative entry point for NLP researchers and practitioners wishing to explore or develop methods for legalNLP. Having these goals in mind, the datasets we include in LexGLUE and the tasks they address have been simplified in several ways to make it easier for newcomers and generic models to address all tasks.

LexGLUE benchmark is accompanied by experimental infrastructure that relies on Hugging Face Transformers library and resides at: https://github.com/coastalcph/lex-glue.

Comments:
Please wait until Tuesday. Arxiv pre-print is pending. 🤗 

Hi @lhoestq, I updated the README with the Arxiv publication info and now the tests are not passing.

It seems that the error is completely irrelevant to my code:

```
 Attempting uninstall: ruamel.yaml
    Found existing installation: ruamel-yaml 0.15.87
ERROR: Cannot uninstall 'ruamel-yaml'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
```

Hi ! Thanks for adding this one. Amazing work :o 

Yea you can ignore the `ruamel-yaml` error, it's unrelated to your PR and fixed on `master`

Hi @lhoestq, 

- I fetched and merged the upstream master to get the `ruamel-yaml` fix.
- I updated the README.md file including more information for the inputs and labels, while I also turned all tables in HTML format hoping that they will render nicely in the dataset card in the HF website.
- I also simplified the CaseHOLD input, excl. the unused `question` field and the `context` replicas, as suggested.
Title: Remove a reference to the open Arrow file when deleting a TF dataset created with to_tf_dataset
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2021-10-02T17:44:09Z

This [comment](https://github.com/huggingface/datasets/issues/2934#issuecomment-922970919) explains the issue. This PR fixes that with a `weakref` callback, and additionally:
* renames `TensorflowDatasetMixIn` to `TensorflowDatasetMixin` for consistency
* correctly indents `TensorflowDatasetMixin`'s docstring
* replaces `tf.data.AUTOTUNE` with `tf.data.experimental.AUTOTUNE` (we support TF>=2.2 according to the [setup.py](https://github.com/huggingface/datasets/blob/fc46bba66ba4f432cc10501c16a677112e13984c/setup.py#L188) and `AUTOTUNE` has been moved to the experimental part of `tf.data` in 1.X if I'm not mistaken)

Fixes #2934

Comments:
@lhoestq The test passes even without the try/except block!

Hey, I'm a little late because I was caught up in the course work, but I double-checked this and it looks great. Thanks for fixing!
Title: Fix cast to Python scalar in Matthews Correlation metric
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2021-10-02T11:44:59Z

This PR is motivated by issue #2964.

The Matthews Correlation metric relies on sklearn's `matthews_corrcoef` function to compute the result. This function returns either `float` or `np.float64` (see the [source](https://github.com/scikit-learn/scikit-learn/blob/844b4be24d20fc42cc13b957374c718956a0db39/sklearn/metrics/_classification.py#L906-L909)). Obviously, calling `.item()` on the float value will fail, so I'm fixing this with the built-in `float()` function, which covers both cases. Surprisingly, on my machine, casting `np.float64` to a Python scalar with `float()` is even faster than with the `.item()` method.

Comments:
Title: Fix json loader when conversion not implemented
Author: lhoestq (MEMBER)
State: closed
Created: 2021-10-01T17:47:22Z

Sometimes the arrow json parser fails if the `block_size` is too small and returns an `ArrowNotImplementedError: JSON conversion to struct...` error.

By increasing the block size it makes it work again.

Hopefully it should help with https://github.com/huggingface/datasets/issues/2799

I tried with the file mentioned in the issue and it worked for me
cc @lewtun can you try again from this branch ?

Comments:
And we're already at PR number 3,000 ! ^^

Thank you so much for fixing this @lhoestq 😍 ! I just tested the branch out and it works like a charm!
Title: cannot shuffle dataset loaded from disk
Author: pya25 (NONE)
State: open
Created: 2021-10-01T13:49:52Z
Labels: bug

## Describe the bug
dataset loaded from disk cannot be shuffled.  

## Steps to reproduce the bug
```
my_dataset = load_from_disk('s3://my_file/validate', fs=s3)
sample = my_dataset.select(range(100)).shuffle(seed=1234)
```

## Actual results
```
sample = my_dataset .select(range(100)).shuffle(seed=1234)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 185, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/fingerprint.py", line 398, in wrapper
    out = func(self, *args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 2494, in shuffle
    new_fingerprint=new_fingerprint,
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 185, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/fingerprint.py", line 398, in wrapper
    out = func(self, *args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 2303, in select
    tmp_file = tempfile.NamedTemporaryFile("wb", dir=os.path.dirname(indices_cache_file_name), delete=False)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/tempfile.py", line 547, in NamedTemporaryFile
    (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/tempfile.py", line 258, in _mkstemp_inner
    fd = _os.open(file, flags, 0o600)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpnnu5uhnx/my_file/validate/tmpy76d70g4'
```
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1

- Python version: 3.7
- PyArrow version: 5.0.0

Comments:
Title: Dataset has incorrect labels
Author: heiko-hotz (NONE)
State: closed
Created: 2021-10-01T12:09:06Z

The dataset https://huggingface.co/datasets/turkish_product_reviews has incorrect labels - all reviews are labelled with "1" (positive sentiment). None of the reviews is labelled with "0". See screenshot attached:

![Capture](https://user-images.githubusercontent.com/63367770/135617428-14ce0b27-5208-4e66-a3ee-71542e3257b4.PNG)

Comments:
Hi @marshmellow77, thanks for reporting.

That issue is fixed since `datasets` version 1.9.0 (see 16bc665f2753677c765011ef79c84e55486d4347).

Please, update `datasets` with: `pip install -U datasets`

Thanks. Please note that the dataset explorer (https://huggingface.co/datasets/viewer/?dataset=turkish_product_reviews) still shows the incorrect state. The sentiment for the first few customer reviews is actually negative and should be labelled with "0", see screenshot:

![Capture1](https://user-images.githubusercontent.com/63367770/135637150-93d9b09b-f1dd-4701-97a5-5cb2672ec0c7.PNG)




Thanks @marshmellow77, good catch! I'm transferring this issue to https://github.com/huggingface/datasets-viewer.
Title: Fix loading compressed CSV without streaming
Author: albertvillanova (MEMBER)
State: closed
Created: 2021-10-01T07:28:59Z

When implementing support to stream CSV files (https://github.com/huggingface/datasets/commit/ad489d4597381fc2d12c77841642cbeaecf7a2e0#diff-6f60f8d0552b75be8b3bfd09994480fd60dcd4e7eb08d02f721218c3acdd2782), a regression was introduced preventing loading compressed CSV files in non-streaming mode.

This PR fixes it, allowing loading compressed/uncompressed CSV files in streaming/non-streaming mode.

Fix #2977.

Comments:
Title: add docmentation for the `Unix style pattern` matching feature that can be leverage for `data_files` into `load_dataset`
Author: SaulLu (CONTRIBUTOR)
State: open
Created: 2021-09-30T13:22:01Z
Labels: enhancement

Unless I'm mistaken, it seems that in the new documentation it is no longer mentioned that you can use Unix style pattern matching in the  `data_files` argument of the `load_dataset` method.

This feature was mentioned [here](https://huggingface.co/docs/datasets/loading_datasets.html#from-a-community-dataset-on-the-hugging-face-hub) in the previous documentation.

I'd love to hear your opinion @lhoestq , @albertvillanova and @stevhliu

Comments:
Title: Add CommonLanguage
Author: anton-l (MEMBER)
State: closed
Created: 2021-09-29T17:21:30Z

This PR adds the Common Language dataset (https://zenodo.org/record/5036977)
The dataset is intended for language-identification speech classifiers and is already used by models on the Hub:
* https://huggingface.co/speechbrain/lang-id-commonlanguage_ecapa
* https://huggingface.co/anton-l/wav2vec2-base-langid

cc @patrickvonplaten

Comments:
Title: add new dataset kan_hope
Author: adeepH (CONTRIBUTOR)
State: closed
Created: 2021-09-29T05:20:28Z

## Adding a Dataset
- **Name:** *KanHope*
- **Description:** *A code-mixed English-Kannada dataset for Hope speech detection*
- **Task:** *Binary Text Classification*
- **Paper:** *https://arxiv.org/abs/2108.04616* 
- **Data:** *https://github.com/adeepH/kan_hope/tree/main/dataset*
- **Motivation:** *The dataset is amongst the very few resources available for code-mixed low-resourced Dravidian languages of India*

Comments:
Title: add wit dataset
Author: nateraw (CONTRIBUTOR)
State: closed
Created: 2021-09-28T16:34:49Z

Resolves #2902 based on conversation there - would also close #2810. Open to suggestions/help 😀

CC @hassiahk @lhoestq @yjernite

Comments:
Opening this up - Here's whats left to make it really shine:
- [ ] Update dataset card (this is being worked on by the author)
- [ ] Add `dataset_info.json`. Requires downloading the entire dataset. I believe @lhoestq mentioned he may have a machine he's using for this sort of thing.

I think both of these _could_ be done in separate PRs, but I'm not picky either way.

Also, we may find that users (and/or the authors) would like additional configurations for languages, image only, embeddings only, etc. I think we can wait and see what the community says on this and address in different PR.

Finally got around to this PR, sorry for the delay!

I added some minimal info to the dataset card so we can at least have it on the hub and work more on it later.

Unfortunately, this re-triggered the tests and it looks like we need a rebase? cc @nateraw 

Yea we need to rebase. Also I think it would be best to update the features to all be strings to match the original datasets format. I ran into a lot of issues with incorrect data types deep into the Dataset info Json process because it couldn't cast correctly. 

I mentioned this to @mariosasko - he might pick up this PR if I don't have bandwidth to finish soon. 

> I think this is a clear occasion to start supporting "canonical" datasets:
> - without dataset_infos.json file
> - without dummy data

I agree we can support datasets without them. However the dataset_infos.json file can at least be **highly encouraged** since it has very nice info about the size of the dataset in MB and in number of examples, and it also helps guaranteeing the integrity of the dataset when it's downloaded (verification of the file checksums and the split sizes).

This dataset now lives here: https://huggingface.co/datasets/wikimedia/wit_base. @nateraw @yjernite Thanks for the initial work.
Title: OpenSLR 25: ASR data for Amharic, Swahili and Wolof
Author: cdleong (CONTRIBUTOR)
State: open
Created: 2021-09-28T15:04:36Z
Labels: dataset request

## Adding a Dataset
- **Name:** *SLR25*
- **Description:** *Subset 25 from OpenSLR. Other subsets have been added to https://huggingface.co/datasets/openslr, 25 covers Amharic, Swahili and Wolof data*
- **Paper:** *https://www.openslr.org/25/ has citations for each of the three subsubsets. *
- **Data:** *Currently the three links to the .tar.bz2 files can be found a thttps://www.openslr.org/25/*
- **Motivation:** *Increase ASR data for underrepresented African languages. Also, other subsets of OpenSLR speech recognition have been uploaded, so this would be easy.*


https://github.com/huggingface/datasets/blob/master/datasets/openslr/openslr.py already has been created for various other OpenSLR subsets, this should be relatively straightforward to do.

Comments:
Whoever handles this just needs to: 

- [ ] fork the HuggingFace Datasets repo
- [ ] update the [existing dataset script](https://github.com/huggingface/datasets/blob/master/datasets/openslr/openslr.py) to add SLR25. Lots of copypasting from other sections of the script should make that easy. 
Amharic URL: https://www.openslr.org/resources/25/data_readspeech_am.tar.bz2. 
Swahili URL: https://www.openslr.org/resources/25/data_broadcastnews_sw.tar.bz2, 
Wolof URL: https://www.openslr.org/resources/25/data_readspeech_wo.tar.bz2
- [ ] update the [data card](https://github.com/huggingface/datasets/blob/master/datasets/openslr/README.md) to include information about SLR25. There's lots of other examples to draw from. 
- [ ] add the appropriate language tags to the data card as well. https://www.w3.org/International/questions/qa-choosing-language-tags, or just use `sw`, `am`, and `wo` for consistency. 
- [ ] make a pull request to merge your changes back into HuggingFace's repo

... also the example in "use in datasets library" should be updated. It currently says 
![image](https://user-images.githubusercontent.com/4109253/135115980-8583a44a-cae6-4121-b699-00667020849f.png)
But you actually have to specify a subset, e.g. 
```python
dataset = load_dataset("openslr", "SLR32")
```

![image](https://user-images.githubusercontent.com/4109253/135116070-82d4e732-b7b3-4c5b-bd4e-a40d8ccabb0e.png)
Title: ValueError when computing f1 metric with average None
Author: asofiaoliveira (CONTRIBUTOR)
State: closed
Created: 2021-09-28T11:34:53Z
Labels: bug

## Describe the bug
When I try to compute the f1 score for each class in a multiclass classification problem, I get a ValueError. The same happens with recall and precision. I traced the error to the `.item()` in these scripts, which is probably there for the other averages. E.g. from f1.py:
```python 
return {
            "f1": f1_score(
                references,
                predictions,
                labels=labels,
                pos_label=pos_label,
                average=average,
                sample_weight=sample_weight,
            ).item(),
        }
```
Since the result is an array with more than one item, the `.item()` throws the error. I didn't submit a PR because this might be needed for the other averages, I'm not very familiar with the library

## Steps to reproduce the bug
```python
from datasets import load_metric
metric = load_metric("f1")
metric.add_batch(predictions=[2,34,1,34,1,2,3], references=[23,52,1,3,523,5,8])
metric.compute(average=None)
```

## Expected results
`array([0.66666667, 0.        , 0.        , 0.        , 0.        ,
       0.        , 0.        , 0.        , 0.        ])`

## Actual results
ValueError: can only convert an array of size 1 to a Python scalar

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.9.5
- PyArrow version: 5.0.0

Comments:
Hi @asofiaoliveira, thanks for reporting.

I'm fixing it.
Title: Run CI tests against non-production server
Author: albertvillanova (MEMBER)
State: open
Created: 2021-09-28T09:41:26Z

Currently, the CI test suite performs requests to the HF production server.

As discussed with @elishowk, we should refactor our tests to use the HF staging server instead, like `huggingface_hub` and `transformers`.

Comments:
Hey @albertvillanova could you provide more context, including extracts from the discussion we had ?

Let's ping @Pierrci @julien-c and @n1t0 for their opinion about that

@julien-c increased the huggingface.co production workers in order to see if it solve [the 502 you had this morning](https://app.circleci.com/pipelines/github/huggingface/datasets/7843/workflows/fc83fa32-18f5-4dc3-9e2f-ba277ae1af74)

For the decision process: be aware that moon-staging does not have persistent repos (they are deleted regularly). as a consequence, **if the moon-staging solution is validated**, you should consider a way to keep the repository that are loaded in tests. These are the ones I found : https://github.com/huggingface/datasets/blob/d488db2f64f312f88f72bbc57a09b7eddb329182/tests/test_load.py and https://github.com/huggingface/datasets/blob/40773111c3e7db8a992fa1c48af32d900a1018d6/tests/test_streaming_download_manager.
Title: Impossible to load compressed csv
Author: Valahaar (CONTRIBUTOR)
State: closed
Created: 2021-09-28T07:18:54Z
Labels: bug

## Describe the bug
It is not possible to load from a compressed csv anymore.

## Steps to reproduce the bug
```python
load_dataset('csv', data_files=['/path/to/csv.bz2'])
```

## Problem and possible solution
This used to work, but the commit that broke it is [this one](https://github.com/huggingface/datasets/commit/ad489d4597381fc2d12c77841642cbeaecf7a2e0#diff-6f60f8d0552b75be8b3bfd09994480fd60dcd4e7eb08d02f721218c3acdd2782).

`pandas` usually gets the compression information from the filename itself (which was previously directly passed). Now, since it gets a file descriptor, it might be good to auto-infer the compression or let the user pass the `compression` kwarg to `load_dataset` (or maybe warn the user if the file ends with a commonly known compression scheme?).

## Environment info
- `datasets` version: 1.10.0 (and over)
- Platform: Linux-5.8.0-45-generic-x86_64-with-glibc2.17
- Python version: 3.8.10
- PyArrow version: 3.0.0

Comments:
Hi @Valahaar, thanks for reporting and for your investigation about the source cause.

You are right and that commit prevents `pandas` from inferring the compression. On the other hand, @lhoestq did that change to support loading that dataset in streaming mode. 

I'm fixing it.
Title: Can't load dataset
Author: mskovalova (NONE)
State: closed
Created: 2021-09-27T21:38:14Z
Labels: bug

I'm trying to load a wikitext dataset

```
from datasets import load_dataset
raw_datasets = load_dataset("wikitext")
```

ValueError: Config name is missing.
Please pick one among the available configs: ['wikitext-103-raw-v1', 'wikitext-2-raw-v1', 'wikitext-103-v1', 'wikitext-2-v1']
Example of usage:
	`load_dataset('wikitext', 'wikitext-103-raw-v1')`.

If I try
```
from datasets import load_dataset
raw_datasets = load_dataset("wikitext-2-v1")
```

FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/1.12.1/datasets/wikitext-2-v1/wikitext-2-v1.py

#### Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic (colab)
- Python version: 3.7.12
- PyArrow version: 3.0.0

Comments:
Hi @mskovalova, 

Some datasets have multiple configurations. Therefore, in order to load them, you have to specify both the *dataset name* and the *configuration name*.

In the error message you got, you have a usage example:
- To load the 'wikitext-103-raw-v1' configuration of the 'wikitext' dataset, you should use: 
  ```python
  load_dataset('wikitext', 'wikitext-103-raw-v1')
  ```

In your case, if you would like to load the 'wikitext-2-v1' configuration of the 'wikitext' dataset, please use:
```python
raw_datasets = load_dataset("wikitext", "wikitext-2-v1")
```

Hi, if I want to load the dataset from local file, then how to specify the configuration name?

> 嗨，如果我想从本地文件加载数据集，那么如何指定配置名称？

请问你知道怎么从本地加载了吗 谢谢

`raw_datasets = load_dataset("/local/path", "wikitext-2-v1", ...)`
I use datasets==2.18.0
Title: masakhaner dataset load problem
Author: huu4ontocord (CONTRIBUTOR)
State: closed
Created: 2021-09-27T04:59:07Z
Labels: bug

## Describe the bug
Masakhaner dataset is not loading

## Steps to reproduce the bug
```
from datasets import load_dataset
dataset = load_dataset("masakhaner",'amh')
```

## Expected results
Expected the return of a dataset

## Actual results

```

NonMatchingSplitsSizesError               Traceback (most recent call last)
<ipython-input-3-a6abc1161d4c> in <module>()
      1 from datasets import load_dataset
      2 
----> 3 dataset = load_dataset("masakhaner",'amh')



3 frames
/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py
 in verify_splits(expected_splits, recorded_splits)
     72     ]
     73     if len(bad_splits) > 0:
---> 74         raise NonMatchingSplitsSizesError(str(bad_splits))
     75     logger.info("All the splits matched successfully.")
     76 

NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=639927, num_examples=1751, dataset_name='masakhaner'), 'recorded': SplitInfo(name='train', num_bytes=639911, num_examples=1750, dataset_name='masakhaner')}, {'expected': SplitInfo(name='validation', num_bytes=92768, num_examples=251, dataset_name='masakhaner'), 'recorded': SplitInfo(name='validation', num_bytes=92753, num_examples=250, dataset_name='masakhaner')}, {'expected': SplitInfo(name='test', num_bytes=184286, num_examples=501, dataset_name='masakhaner'), 'recorded': SplitInfo(name='test', num_bytes=184271, num_examples=500, dataset_name='masakhaner')}]
```

## Environment info
Google Colab

Comments:
Thanks for reporting, @ontocord. We are fixing the wrong metadata.
Title: Magnet’s
Author: rcacho172 (NONE)
State: closed
Created: 2021-09-26T09:50:29Z
Labels: dataset request

## Adding a Dataset
- **Name:** *name of the dataset*
- **Description:** *short description of the dataset (or link to social media or blog post)*
- **Paper:** *link to the dataset paper if available*
- **Data:** *link to the Github repository or current dataset location*
- **Motivation:** *what are some good reasons to have this dataset*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Comments:
Title: medical-dialog error
Author: smeyerhot (NONE)
State: closed
Created: 2021-09-25T23:08:44Z
Labels: bug

## Describe the bug
A clear and concise description of what the bug is.
When I attempt to download the huggingface datatset medical_dialog it errors out midway through
## Steps to reproduce the bug
```python
raw_datasets = load_dataset("medical_dialog", "en", split="train", download_mode="force_redownload", data_dir="./Medical-Dialogue-Dataset-English")
```

## Expected results
A clear and concise description of the expected results.
No error
## Actual results

```
3 frames
/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py in verify_splits(expected_splits, recorded_splits)
     72     ]
     73     if len(bad_splits) > 0:
---> 74         raise NonMatchingSplitsSizesError(str(bad_splits))
     75     logger.info("All the splits matched successfully.")
     76 

NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=0, num_examples=0, dataset_name='medical_dialog'), 'recorded': SplitInfo(name='train', num_bytes=295097913, num_examples=229674, dataset_name='medical_dialog')}]
```
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.21.1
- Platform: colab
- Python version: colab 3.7
- PyArrow version: N/A

Comments:
Hi @smeyerhot, thanks for reporting.

You are right: there is an issue with the dataset metadata. I'm fixing it.

In the meantime, you can circumvent the issue by passing `ignore_verifications=True`:
```python
raw_datasets = load_dataset("medical_dialog", "en", split="train", download_mode="force_redownload", data_dir="./Medical-Dialogue-Dataset-English", ignore_verifications=True)
```

Hi @albertvillanova -- I see you made changes to the wikitext dataset a few hours ago.
I'm getting an error similar to the one here : https://github.com/huggingface/datasets/issues/215
After your changes. 

I deleted the original dataset cache after encountering that error and then rerun with the following command :

`traindata = load_dataset('wikitext', 'wikitext-2-raw-v1', split='train', ignore_verifications=True, download_mode='force_redownload')
    testdata = load_dataset('wikitext', 'wikitext-2-raw-v1', split='test', ignore_verifications=True)`

But this is currently hanging.  Any ideas ? Thanks

@ldery, you can find the answer in the corresponding Discussion on the Hub: https://huggingface.co/datasets/wikitext/discussions/9

You need to update the `datasets` library:
```shell
pip install -U datasets
```
Title: Adding vision-and-language datasets (e.g., VQA, VCR) to Datasets
Author: WadeYin9712 (NONE)
State: closed
Created: 2021-09-25T20:58:15Z
Labels: enhancement

**Is your feature request related to a problem? Please describe.**
Would you like to add any vision-and-language datasets (e.g., VQA, VCR) to Huggingface Datasets?

**Describe the solution you'd like**
N/A

**Describe alternatives you've considered**
N/A

**Additional context**
This is Da Yin at UCLA. Recently, we have published an EMNLP 2021 paper about geo-diverse visual commonsense reasoning (https://arxiv.org/abs/2109.06860). We propose a new dataset called GD-VCR, a vision-and-language dataset to evaluate how well V&L models perform on scenarios involving geo-location-specific commonsense. We hope to have our V&L dataset incorporated into Huggingface to further promote our project, but I haven't seen much V&L datasets in the current package. Is it  possible to add V&L datasets, and if so, how should we prepare for the loading? Thank you very much!

Comments:
Title: Invalid download URL of WMT17 `zh-en` data 
Author: Ririkoo (NONE)
State: closed
Created: 2021-09-25T13:17:32Z
Labels: bug

dataset bug

## Describe the bug
Partial data (wmt17 zh-en) cannot be downloaded due to an invalid URL.

## Steps to reproduce the bug
```python
from datasets import load_dataset
 dataset = load_dataset('wmt17','zh-en')
```

## Expected results
ConnectionError: Couldn't reach ftp://cwmt-wmt:cwmt-wmt@datasets.nju.edu.cn/parallel/casia2015.zip

Comments:
Fixed in the current release. Close this issue.
Title:     raise TypeError( TypeError: Provided `function` which is applied to all elements of table returns a variable of type <class 'list'>. Make sure provided `function` returns a variable of type `dict` to update the dataset or `None` if you are only interested in side effects.
Author: keloemma (NONE)
State: closed
Created: 2021-09-24T15:35:11Z
Labels: bug

## Describe the bug
A clear and concise description of what the bug is.
I am trying to use Dataset to load my file in order to use Bert embeddings model baut when I finished loading using dataset and I want to pass to the tokenizer using the function map; I get the following error : raise TypeError(
TypeError: Provided `function` which is applied to all elements of table returns a variable of type <class 'list'>. Make sure provided `function` returns a variable of type `dict` to update the dataset or `None` if you are only interested in side effects.

I was able to load my file using dataset before but since this morning , I keep getting this erreor.
## Steps to reproduce the bug
```python
# Xtrain, ytrain, filename, len_labels = read_file_2(fic)
	# Xtrain, lge_size = get_flaubert_layer(Xtrain, path_to_model_lge)

	data_preprocessed = make_new_traindata(Xtrain)
	
	my_dict = {"verbatim": data_preprocessed[1], "label": ytrain} # lemme avec conjonction
	dataset = Dataset.from_dict(my_dict)
```

## Expected results
A clear and concise description of the expected results.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform:
- Python version:
- PyArrow version:

Comments:
Title: Enable splits during streaming the dataset
Author: merveenoyan (CONTRIBUTOR)
State: open
Created: 2021-09-24T15:01:29Z
Labels: enhancement

## Describe the Problem

I'd like to stream only a specific percentage or part of the dataset. 
I want to do splitting when I'm streaming dataset as well.

## Solution
Enabling splits when `streaming = True` as well.
`e.g. dataset = load_dataset('dataset', split='train[:100]', streaming = True)`

## Alternatives
Below is the alternative of doing it.
`dataset = load_dataset("dataset", split='train', streaming = True).take(100)`

Comments:
Title: Fix CI doc build
Author: albertvillanova (MEMBER)
State: closed
Created: 2021-09-24T13:13:28Z

Pin `fsspec`.

Before the issue: 'fsspec-2021.8.1', 's3fs-2021.8.1'
Generating the issue: 'fsspec-2021.9.0', 's3fs-0.5.1'

Comments:
Title: Support pandas 1.3 new `read_csv` parameters
Author: SBrandeis (CONTRIBUTOR)
State: closed
Created: 2021-09-24T08:37:24Z

Support two new arguments introduced in pandas v1.3.0:
- `encoding_errors`
- `on_bad_lines` 

`read_csv` reference: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html

Comments:
Title: Added computer vision tasks
Author: merveenoyan (CONTRIBUTOR)
State: closed
Created: 2021-09-23T15:07:27Z

Added various image processing/computer vision tasks.

Comments:
Looks great, thanks ! If the 3d ones are really rare we can remove them for now.

And I can see that `object-detection` and `semantic-segmentation` are both task categories (top-level) and task ids (bottom-level). Maybe there's a way to group them and have less granularity for the task categories. For example `speech-processing` is a high level task category. What do you think ?

We can still update the list of tasks later if needed when we have more vision datasets


@lhoestq  @osanseviero I used the categories (there were main ones and subcategories) in the paperswithcode, I got rid of some of them that could be too granular. I can put it there if you'd like (I'll wait for your reply before committing it again)

We can ignore the ones that are too granular IMO. What we did for audio tasks is to have them all under "audio-processing". Maybe we can do the same here for now until we have more comprehensive tasks/applications ?

Following the discussion in (private) https://github.com/huggingface/moon-landing/issues/2020, what do you think of aligning the top level tasks list with the model tasks taxonomy ?

* Image Classification
* Object Detection
* Image Segmentation
* Text-to-Image
* Image-to-Text


I moved it to [a branch](https://github.com/huggingface/datasets/pull/3800) for ease.