rIsHu009/github-issues-updated huggingface dataset:
Title: [Imagefolder] Docs + Don't infer labels from file names when there are metadata + Error messages when metadata and images aren't linked correctly
Author: lhoestq (MEMBER)
State: closed
Created: 2022-05-10T15:52:15Z

I updated the `docs/source/image_process.mdx` documentation and added an example for image captioning and object detection using `ImageFolder`.

While doing so I also improved a few aspects:
- we don't need to infer labels from file names when there are metadata - they can just be in the metadata if necessary
- raise informative error messages when metadata and images aren't linked correctly:
  - when an image is missing a metadata file
  - when a metadata file is missing an image

I added some tests for these changes as well

cc @mariosasko

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Merging this one since mario is off, I took care of adding some tests to make sure everything is fine. Will do the release after it
Title: Fixes FrugalScore
Author: moussaKam (CONTRIBUTOR)
State: open
Created: 2022-05-10T12:44:06Z
Labels: transfer-to-evaluate

There are two minor modifications in this PR:
1) `predictions` and `references` are swapped. Basically Frugalscore is commutative, however some tiny differences can occur if we swap the references and the predictions. I decided to swap them just to obtain the exact results as reported in the paper.
2) I switched to dynamic padding that was was used in the training, forcing the padding to `max_length`  introduces errors for some reason that I ignore.

@lhoestq

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4305). All of your documentation changes will be reflected on that endpoint.

> predictions and references are swapped. Basically Frugalscore is commutative, however some tiny differences can occur if we swap the references and the predictions. I decided to swap them just to obtain the exact results as reported in the paper.

What is the order of magnitude of the difference ? Do you know what causes this ?

> I switched to dynamic padding that was was used in the training, forcing the padding to max_length introduces errors for some reason that I ignore.

What error ?
Title: Language code search does direct matches
Author: leondz (CONTRIBUTOR)
State: open
Created: 2022-05-10T11:59:16Z
Labels: bug

## Describe the bug

Hi. Searching for bcp47 tags that are just the language prefix (e.g. `sq` or `da`) excludes datasets that have added extra information in their language metadata (e.g. `sq-AL` or `da-bornholm`). The example codes given in the [tagging app](https://huggingface.co/spaces/huggingface/datasets-tagging) encourages addition of the additional codes ("_expected format is BCP47 tags separated for ';' e.g. 'en-US;fr-FR'_") but this would lead to those datasets being hidden in datasets search.

## Steps to reproduce the bug
1. Add a dataset using a variant tag (e.g. [`sq-AL`](https://huggingface.co/datasets?languages=languages:sq-AL))
2. Look for datasets using the full code 
3. Note that they're missing when just the language is searched for (e.g. [`sq`](https://huggingface.co/datasets?languages=languages:sq))

Some datasets are already affected by this - e.g. `AmazonScience/massive` is listed under `sq-AL` but not `sq`.

One workaround is for dataset creators to add an additional root language tag to dataset YAML metadata, but it's unclear how to communicate this. It might be possible to index the search on `languagecode.split('-')[0]` but I wanted to float this issue before trying to write any code :)

## Expected results
Datasets using longer bcp47 tags also appear under searches for just the language code; e.g. Quebecois datasets (`fr-CA`) would come up when looking for French datasets with no region specification (`fr`), or US English (`en-US`) datasets would come up when searching for English datasets (`en`).

## Actual results
The language codes seem to be directly string matched, excluding datasets with specific language tags from non-specific searches.

## Environment info
(web app)

Comments:
Thanks for reporting ! I forwarded the issue to the front-end team :)

Will keep you posted !

I also changed the tagging app to suggest two letters code for now.
Title: Remove hacking license tags when mirroring datasets on the Hub
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-05-10T05:52:46Z

Currently, when mirroring datasets on the Hub, the license tags are hacked: removed of characters "." and "$". On the contrary, this hacking is not applied to community datasets on the Hub. This generates multiple variants of the same tag on the Hub. 

I guess this hacking is no longer necessary:
- it is not applied to community datasets
- all canonical datasets are validated by maintainers before being merged: CI + maintainers make sure license tags are the right ones

Fix #4298.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The Hub doesn't allow these characters in the YAML tags, and git push fails if you want to push a dataset card containing these characters.

Ok, let me rename the bad config names :) I think I can also keep backward compatibility with a warning

Almost done with it btw, will submit a PR that shows all the configuration name changes (from a bit more than 20 datasets)

Please, let me know when the renaming of configs is done. If not enough bandwidth, I can take care of it...

Will focus on this this afternoon ;)

I realized when renaming all the configurations with dots in https://github.com/huggingface/datasets/pull/4365 that it's not ideal for certain cases. For example:
- many configurations have a version like "1.0.0" in their names
- to avoid breaking changes we need to replace dots with underscores in the user input and show a warning, which hurts the experience
- our second most downloaded dataset at the moment is affected: `newsgroup`
- if we disallow dots, then we'll never be able to make the [allenai/c4](https://huggingface.co/datasets/allenai/c4) work with its different configurations since they contain dots, and we can't rename them because they are the official download links

I was thinking of other alternatives:
1. just stop separating tags per config name completely, and have a single flat YAML for all configurations. Dataset search doesn't use this info anyway
2. use another YAML structure to avoid having config names as keys, such as
```yaml
languages:
- config: 20220301_en
  values:
  - en
```

I'm down for 1, to keep things simple

@lhoestq I agree:
- better not changing config names (so that we do not introduce any braking change)
- therefore, we should not use them as keys

In relation with the proposed solutions, I have no strong opinion:
- option 1 is simpler and aligns better with current usage on the Hub (configs are ignored)
- however:
  - we will lose all the information per config we already have (for those datasets containing config keys; contributors made an effort to put that information per config)
  - and this information might be useful on the Hub in the future, in case we would like to enrich the search feature with more granularity; this is only applicable if this feature could eventually make sense

So, no strong opinion...

Closing in favor of https://github.com/huggingface/datasets/pull/4367
Title: Add ImageNet-Sketch dataset
Author: nateraw (CONTRIBUTOR)
State: closed
Created: 2022-05-09T23:38:45Z

This PR adds the ImageNet-Sketch dataset and resolves #3953 .

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I think you can go ahead with uploading the data, and also ping the author in parallel. I think the images may subject to copyright anyway (scrapped from google image) so the dataset author is not allowed to set a license to the data.

I think it's fine to upload the dataset as soon as we mention explicitly that the images may be subject to copyright.
Title: Add API code examples for loading methods
Author: stevhliu (MEMBER)
State: closed
Created: 2022-05-09T21:30:26Z
Labels: documentation

This PR adds API code examples for loading methods, let me know if I've missed any important parameters we should showcase :)

I was a bit confused about `inspect_dataset` and `inspect_metric`. The `path` parameter says it will accept a dataset identifier from the Hub. But when I try the identifier `rotten_tomatoes`, it gives me:

```py
from datasets import inspect_dataset
inspect_dataset('rotten_tomatoes', local_path='/content/rotten_tomatoes')

FileNotFoundError: Couldn't find a dataset script at /content/rotten_tomatoes/rotten_tomatoes.py or any data file in the same directory.
```

Does the user need to have an existing copy of `rotten_tomatoes.py` on their local drive (in which case, it seems like the same option as the first option in `path`)?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Remove manual download from imagenet-1k
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2022-05-09T20:49:18Z

Remove the manual download code from `imagenet-1k` to make it a regular dataset.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks for the reviews @apsdehal and @lhoestq! As suggested by @lhoestq, I'll separate the train/val/test splits, apply the validation split fixes and shuffle the images files to simplify the script and make streaming faster.

@apsdehal I dismissed your review as it's no longer relevant after the data files changes suggested by @lhoestq.
Title: Normalise license names
Author: leondz (CONTRIBUTOR)
State: closed
Created: 2022-05-09T13:51:32Z
Labels: enhancement

**Is your feature request related to a problem? Please describe.**
When browsing datasets, the Licenses tag cloud (bottom left of e.g. https://huggingface.co/datasets) has multiple variants of the same license. This means the options exclude datasets arbitrarily, giving users artificially low recall. The cause of the dupes is probably due to a bit of variation in metadata.

**Describe the solution you'd like**
I'd like the licenses in metadata to follow the same standard as much as possible, to remove this problem. I'd like to go ahead and normalise the dataset metadata to follow the format & values given in [src/datasets/utils/resources/licenses.json](https://github.com/huggingface/datasets/blob/master/src/datasets/utils/resources/licenses.json) .

**Describe alternatives you've considered**
None

**Additional context**
None

**Priority** 
Low

Comments:
we'll add the same server-side metadata validation system as for hf.co/models soon-ish

(you can check on hf.co/models that licenses are "clean")

Fixed by #4367.
Title: Datasets YAML tagging space is down
Author: leondz (CONTRIBUTOR)
State: closed
Created: 2022-05-09T13:45:05Z
Labels: bug

## Describe the bug
The neat hf spaces app for generating YAML tags for dataset `README.md`s is down

## Steps to reproduce the bug
1. Visit https://huggingface.co/spaces/huggingface/datasets-tagging

## Expected results
There'll be a HF spaces web app for generating dataset metadata YAML

## Actual results
There's an error message; here's the step where it breaks:

```
Step 18/29 : RUN pip install -r requirements.txt
 ---> Running in e88bfe7e7e0c
Defaulting to user installation because normal site-packages is not writeable
Collecting git+https://github.com/huggingface/datasets.git@update-task-list (from -r requirements.txt (line 4))
  Cloning https://github.com/huggingface/datasets.git (to revision update-task-list) to /tmp/pip-req-build-bm8t0r0k
  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/datasets.git /tmp/pip-req-build-bm8t0r0k
  WARNING: Did not find branch or tag 'update-task-list', assuming revision or ref.
  Running command git checkout -q update-task-list
  error: pathspec 'update-task-list' did not match any file(s) known to git
  error: subprocess-exited-with-error
  
  × git checkout -q update-task-list did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× git checkout -q update-task-list did not run successfully.
│ exit code: 1
╰─> See above for output.
```

## Environment info

- Platform: Linux / Brave

Comments:
@lhoestq @albertvillanova `update-task-list` branch does not exist anymore, should point to `main` now i guess

Thanks for reporting, fixing it now

It's up again :)
Title: Add API code examples for remaining main classes
Author: stevhliu (MEMBER)
State: closed
Created: 2022-05-06T18:15:31Z
Labels: documentation

This PR adds API code examples for the remaining functions in the Main classes. I wasn't too familiar with some of the functions (`decode_batch`, `decode_column`, `decode_example`, etc.) so please feel free to add an example of usage and I can fill in the rest :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Dataset Viewer issue for strombergnlp/ipm_nel : preview is empty, no error message
Author: leondz (CONTRIBUTOR)
State: closed
Created: 2022-05-06T12:03:27Z
Labels: dataset-viewer

### Link

https://huggingface.co/datasets/strombergnlp/ipm_nel/viewer/ipm_nel/train

### Description

The viewer is blank. I tried my best to emulate a dataset with a working viewer, but this one just doesn't seem to want to come up. What did I miss?

### Owner

Yes

Comments:
Hi @leondz, thanks for reporting.

Indeed, the dataset viewer relies on the dataset being streamable (passing `streaming=True` to `load_dataset`). Whereas most of the datastes are streamable out of the box (thanks to our implementation of streaming), there are still some exceptions.

In particular, in your case, that is due to the data file being TAR. This format is not streamable out of the box (it does not allow random access to the archived files), but we use a trick to allow streaming: using `dl_manager.iter_archive`.

Let me know if you need some help: I could push a commit to your repo with the fix.

Ah, right! The preview is working now, but this explanation is good to know, thank you. I'll prefer formats with random file access supported in datasets.utils.extract in future, and try out this fix for the tarfiles :)
Title: Fix filesystem docstring
Author: stevhliu (MEMBER)
State: closed
Created: 2022-05-04T17:42:42Z

This PR untangles the `S3FileSystem` docstring so the [parameters](https://huggingface.co/docs/datasets/master/en/package_reference/main_classes#parameters) are properly displayed.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Don't do unnecessary list type casting to avoid replacing None values by empty lists
Author: lhoestq (MEMBER)
State: closed
Created: 2022-05-04T16:37:01Z

In certain cases, `None` values are replaced by empty lists when casting feature types.

It happens every time you cast an array of nested lists like [None, [0, 1, 2, 3]] to a different type (to change the integer precision for example). In this case you'd get [[], [0, 1, 2, 3]] for example. This issue comes from PyArrow, see the discussion in https://github.com/huggingface/datasets/issues/3676

This issue also happens when no type casting is needed, because casting is supposed to be a no-op in this case. But as https://github.com/huggingface/datasets/issues/3676 shown, it's not the case and `None` are replaced by empty lists even if we cast to the exact same type.

In this PR I just workaround this bug in the case where no type casting is needed. In particular, I only call `pa.ListArray.from_arrays` only when necessary.

I also added a warning when some `None` are effectively replaced by empty lists. I wanted to raise an error in this case, but maybe we should wait a major update to do so

This PR fixes this particular case, that is occurring in `run_qa.py` in `transformers`:
```python
from datasets import Dataset

ds = Dataset.from_dict({"a": range(4)})
ds = ds.map(lambda x: {"b": [[None, [0]]]}, batched=True, batch_size=1, remove_columns=["a"])
print(ds.to_pandas())
# before:
#              b
# 0  [None, [0]]
# 1    [[], [0]]
# 2    [[], [0]]
# 3    [[], [0]]
#
# now:
#              b
# 0  [None, [0]]
# 1  [None, [0]]
# 2  [None, [0]]
# 3  [None, [0]]
```

cc @sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Quick question about the message in the warning. You say "will be fixed in a future major version" but don't you mean "will raise an error in a future major version"?

Right ! Good catch, thanks, I updated the message to say "will raise an error in a future major version"
Title: Remove a copy-paste sentence in dataset cards
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-05-04T15:41:55Z

Remove the following copy-paste sentence from dataset cards:
```
We show detailed information for up to 5 configurations of the dataset.
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The non-passing tests have nothing to do with this PR.
Title: Add missing features to commonsense_qa dataset
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-05-04T14:24:26Z

Fix partially #4275.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@albertvillanova it adds question_concept and id which is great. I suppose we'll talk about staying true to the format on another PR. 

Yes, let's merge this PR as it is: it adds missing features.

A subsequent PR may address the request on changing the dataset feature structure.
Title: Add missing features to openbookqa dataset for additional config
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-05-04T09:22:50Z

Fix partially #4276.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Let's merge this PR as it is: it adds missing features.

A subsequent PR may address the request on changing the data feature structure.
Title: Enable label alignment for token classification datasets
Author: lewtun (MEMBER)
State: closed
Created: 2022-05-04T07:15:16Z

This PR extends the `Dataset.align_labels_with_mapping()` method to support alignment of label mappings between datasets and models for token classification (e.g. NER).

Example of usage:

```python
from datasets import load_dataset

ner_ds = load_dataset("conll2003", split="train")
# returns [3, 0, 7, 0, 0, 0, 7, 0, 0]
ner_ds[0]["ner_tags"]
# hypothetical model mapping with O <--> B-LOC
label2id = {
    "B-LOC": "0",
    "B-MISC": "7",
    "B-ORG": "3",
    "B-PER": "1",
    "I-LOC": "6",
    "I-MISC": "8",
    "I-ORG": "4",
    "I-PER": "2",
    "O": "5"
  }
ner_aligned_ds = ner_ds.align_labels_with_mapping(label2id, "ner_tags")
# returns [3, 5, 7, 5, 5, 5, 7, 5, 5]
ner_aligned_ds[0]["ner_tags"]
```

Context: we need this in AutoTrain to automatically align datasets / models during evaluation. cc @abhishekkrthakur

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hmm, not sure why the Windows tests are failing with:

```
Did not find path entry C:\tools\miniconda3\bin
C:\tools\miniconda3\envs\py37\python.exe: No module named pytest
```

Edit: running the CI again fixed the problem 🙃 

> One last nit and we can merge then

Thanks, done!
Title: CommonSenseQA has missing and inconsistent field names
Author: vblagoje (CONTRIBUTOR)
State: open
Created: 2022-05-04T05:38:59Z
Labels: dataset bug

## Describe the bug
In short, CommonSenseQA implementation is inconsistent with the original dataset.

More precisely, we need to:

1. Add the dataset matching "id" field. The current dataset, instead, regenerates monotonically increasing id.  
2. The [“question”][“stem”] field is flattened into "question". We should match the original dataset and unflatten it
3. Add the missing "question_concept" field in the question tree node
4. Anything else? Go over the data structure of the newly repaired CommonSenseQA and make sure it matches the original

## Expected results
Every data item of the CommonSenseQA should structurally and data-wise match the original CommonSenseQA dataset.

## Actual results
TBD

## Environment info
- `datasets` version: 2.1.0
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.13
- PyArrow version: 7.0.0
- Pandas version: 1.4.2

Comments:
Thanks for reporting, @vblagoje.

I'm opening a PR to address this.
Title: Fix typo in logging docs
Author: stevhliu (MEMBER)
State: closed
Created: 2022-05-03T20:47:57Z

This PR fixes #4271.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> This PR fixes #4271.

Things have not changed when searching "tqdm" in the Dataset document. The second result still performs as "Enable".

Hi @jiangwy99, the fix will appear on the `main` version of the docs:

![Screen Shot 2022-05-04 at 8 38 29 AM](https://user-images.githubusercontent.com/59462357/166718225-6848ab91-87d1-4572-9912-40a909af6cb9.png)


Fixed now, thanks.
Title: A typo in docs of datasets.disable_progress_bar
Author: jiangwangyi (NONE)
State: closed
Created: 2022-05-03T17:44:56Z
Labels: bug

## Describe the bug
in the docs of V2.1.0 datasets.disable_progress_bar, we should replace "enable" with "disable".

Comments:
Hi! Thanks for catching and reporting the typo, a PR has been opened to fix it :)
Title: Add HF Speech Bench to Librispeech Dataset Card
Author: sanchit-gandhi (CONTRIBUTOR)
State: closed
Created: 2022-05-02T16:59:31Z

Adds the HF Speech Bench to Librispeech Dataset Card in place of the Papers With Code Leaderboard. Should improve usage and visibility of this leaderboard! Wondering whether this can also be done for [Common Voice 7](https://huggingface.co/datasets/mozilla-foundation/common_voice_7_0) and [8](https://huggingface.co/datasets/mozilla-foundation/common_voice_8_0) through someone with permissions? 

cc @patrickvonplaten: more leaderboard promotion!

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Rename imagenet2012 -> imagenet-1k
Author: lhoestq (MEMBER)
State: closed
Created: 2022-05-02T10:26:21Z

On the Hugging Face Hub, users refer to imagenet2012 (from #4178 ) as imagenet-1k in their model tags.

To correctly link models to imagenet, we should rename this dataset `imagenet-1k`.

Later we can add `imagenet-21k` as a new dataset if we want.

Once this one is merged we can delete the `imagenet2012` dataset repository on the Hub.

EDIT: to complete the rationale on why we should name it `imagenet-1k`:
If users specifically added the tag `imagenet-1k` , then it could be for two reasons (not sure which one is predominant), either they
- wanted to make it explicit that it’s not 21k -> the distinction is important for the community
- or they have been following this convention from other models -> the convention implicitly exists already

Comments:
> Later we can add imagenet-21k as a new dataset if we want.

isn't it what models refer to as `imagenet` already?

> isn't it what models refer to as imagenet already?

I wasn't sure, but it looks like it indeed. Therefore having a dataset `imagenet` for ImageNet 21k makes sense actually.

EDIT: actually not all `imagenet` tag refer to ImageNet 21k - we will need to correct some of them

_The documentation is not available anymore as the PR was closed or merged._

should we remove the repo mirror on the hub side or will you do it?
Title: Add YAML tags to Dataset Card rotten tomatoes
Author: mo6zes (CONTRIBUTOR)
State: closed
Created: 2022-05-01T11:59:08Z

The dataset card for the rotten tomatoes / MR movie review dataset had some missing YAML tags. Hopefully, this also improves the visibility of this dataset now that paperswithcode and huggingface link to eachother.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Add mr_polarity movie review sentiment classification
Author: mo6zes (CONTRIBUTOR)
State: closed
Created: 2022-04-30T13:19:33Z

Add the MR (Movie Review) dataset. The original dataset contains sentences from Rotten Tomatoes labeled as either "positive" or  "negative". 

Homepage: [https://www.cs.cornell.edu/people/pabo/movie-review-data/](https://www.cs.cornell.edu/people/pabo/movie-review-data/)
paperswithcode: [https://paperswithcode.com/dataset/mr](https://paperswithcode.com/dataset/mr)

- [ ] I was not able to generate dummy data, the original dataset files have ".pos" and ".neg" as file extensions so the auto-generator does not work. Is it fine like this or should dummy data be added?

Comments:
whoops just found https://huggingface.co/datasets/rotten_tomatoes
Title: Fix bug in choices labels in openbookqa dataset
Author: manandey (CONTRIBUTOR)
State: closed
Created: 2022-04-30T07:41:39Z

This PR fixes the Bug in the openbookqa dataset as mentioned in this issue #3550.

Fix #3550.

cc. @lhoestq @mariosasko

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Fix/start token mask issue and update documentation
Author: TristanThrush (CONTRIBUTOR)
State: closed
Created: 2022-04-29T22:42:44Z

This pr fixes a couple bugs:

1) the perplexity was calculated with a 0 in the attention mask for the start token, which was causing high perplexity scores that were not correct
2) the documentation was not updated

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Good catch ! Thanks :)
> 
> Next time can you describe your fix in the Pull Request description please ?

Thanks. Also whoops, sorry about not being very descriptive. I updated the pull request description, and will keep this in mind for future PRs.
Title: No google drive URL for pubmed_qa
Author: lhoestq (MEMBER)
State: closed
Created: 2022-04-29T15:55:46Z

I hosted the data files in https://huggingface.co/datasets/pubmed_qa. This is allowed because the data is under the MIT license.

cc @stas00

Comments:
_The documentation is not available anymore as the PR was closed or merged._

CI is failing because some sections are missing in the dataset card, but this is unrelated to this PR - Merging !
Title: Replace data URL in SAMSum dataset and support streaming
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-04-29T08:21:43Z

This PR replaces data URL in SAMSum dataset:
- original host (arxiv.org) does not allow HTTP Range requests
- we have hosted the data on the Hub (license: CC BY-NC-ND 4.0)

Moreover, it implements support for streaming.

Fix #4146.
Related to: #4236.

CC: @severo

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Bump PyArrow Version to 6
Author: dnaveenr (CONTRIBUTOR)
State: closed
Created: 2022-04-28T18:10:50Z

Fixes #4152 

This PR updates the PyArrow version to 6 in setup.py, CI job files .circleci/config.yaml and .github/workflows/benchmarks.yaml files.
This will fix ArrayND error which exists in pyarrow 5.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Updated meta.yaml as well. Thanks.

I'm OK with bumping PyArrow to version 6 to match the version in Colab, but maybe a better solution would be to stop using extension types in our codebase to avoid similar issues.

> but maybe a better solution would be to stop using extension types in our codebase to avoid similar issues.

I agree, not much attention has been payed to extension arrays in the latest developments of Arrow anyway.

Let's not use them more that what we do right now, and try to remove them at one point
Title: The data preview of XGLUE
Author: czq1999 (NONE)
State: closed
Created: 2022-04-28T07:30:50Z

It seems that something wrong with the data previvew of XGLUE

Comments:
![image](https://user-images.githubusercontent.com/49108847/165700611-915b4343-766f-4b81-bdaa-b31950250f06.png)


Thanks for reporting @czq1999.

Note that the dataset viewer uses the dataset in streaming mode and that not all datasets support streaming yet.

That is the case for XGLUE dataset (as the error message points out): this must be refactored to support streaming. 

Fixed, thanks @albertvillanova !

https://huggingface.co/datasets/xglue

<img width="824" alt="Capture d’écran 2022-04-29 à 10 23 14" src="https://user-images.githubusercontent.com/1676121/165909391-9f98d98a-665a-4e57-822d-8baa2dc9b7c9.png">
Title: Support to load dataset with TSV files by passing only dataset name
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-04-28T07:30:15Z

This PR implements support to load a dataset (w/o script) containing TSV files by passing only the dataset name (no need to pass `sep='\t'`):
```python
ds = load_dataset("dataset/name")
```

The refactoring allows for future builder kwargs customizations based on file extension.

Related to #4238.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: WIP: Initial shades loading script and readme
Author: shayne-longpre (NONE)
State: closed
Created: 2022-04-27T17:45:43Z
Labels: dataset contribution


Comments:
Thanks for your contribution, @shayne-longpre.

Are you still interested in adding this dataset? As we are transferring the dataset scripts from this GitHub repo, we would recommend you to add this to the Hugging Face Hub: https://huggingface.co/datasets
Title: Fix yield for crd3
Author: shanyas10 (CONTRIBUTOR)
State: closed
Created: 2022-04-27T12:31:36Z

Modified the `_generate_examples` function to consider all the turns for a chunk id as a single example
Modified the features accordingly

```
"turns": [
                        {
                            "names": datasets.features.Sequence(datasets.Value("string")),
                            "utterances": datasets.features.Sequence(datasets.Value("string")),
                            "number": datasets.Value("int32"),
                        }
                    ],
                }
```

I wasn't able to run `datasets-cli dummy_data datasets` command. Is there a workaround for this?

Comments:
I don't think you need to generate new dummy data, since they're in the same format as the original data.

The CI is failing because of this error:
```python
>                       turn["names"] = turn["NAMES"]
E                       TypeError: tuple indices must be integers or slices, not str
```

Do you know what could cause this ? If I understand correctly, `turn` is supposed to be a list of dictionaries right ?

> ```  
>   
> Do you know what could cause this ? If I understand correctly, turn is supposed to be a list of dictionaries right ?
> ```

This is strange. Let me look into this. As per https://github.com/RevanthRameshkumar/CRD3/blob/master/data/aligned%20data/c%3D2/C1E001_2_0.json TURNS is a list of dictionaries. So when we iterate over `row["TURNS]"` each `turn` is essentially a dictionary. Not sure why it's being considered a tuple here.
Title: Replace data URL in big_patent dataset and support streaming
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-04-27T10:01:13Z

This PR replaces the Google Drive URL with our Hub one, once the data owners have approved to host their data on the Hub.

Moreover, this PR makes the dataset streamable.

Fix #4217.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I first uploaded the data files to the Hub: I think it is a good option because we have git lfs to track versions and changes. Moreover people will be able to make PRs to propose updates on the data files.
- I would have preferred to upload it it to the "data" org namespace, but it is already taken (although not used): might be possible to take it?

As an alternative (and to be consistent with previous datasets), I also uploaded the data files to our AWS bucket.

We should decide which to use (now and for future datasets) and set it here before merging. We should remove the data files for the non-chosen option.

CC: @lhoestq @mariosasko @polinaeterna 

Would it make sense to make the dataset a community one (so, create an organization for it) and store the script and the data in a single repository? Just as it is for most of the datasets. That way we can also access the data using a relative path inside the repo (that's not the point though). The point is that to me it seems a bit more straightforward to store everything in one place. 

I guess the strong argument against this logic is that in this case the canonical version won't work... But maybe there is some redirecting mechanism I don't know about? :)

Anyway, I'm in favor of hosting data on the Hub instead of AWS :) 

I also think storing everything in one place/single repository is the best option.

@polinaeterna Canonical datasets also support data files (see the [`red_caps` repo](https://huggingface.co/datasets/red_caps/tree/main) for instance) 

Thanks @polinaeterna and @mariosasko for your comments.

Yes, definitely it is much better to have everything in the same repo. 

I'm transferring their data files to the Hub under "big_patent" and deleting them from the other repo and AWS.
Title: How to load VERY LARGE dataset?
Author: CaoYiqingT (NONE)
State: closed
Created: 2022-04-27T07:50:13Z
Labels: bug

### System Info

```shell
I am using transformer trainer while meeting the issue.
The trainer requests torch.utils.data.Dataset as input, which loads the whole dataset into the memory at once. Therefore, when the dataset is too large to load, there's nothing I can do except using IterDataset, which loads samples of data seperately, and results in low efficiency. 
I wonder if there are any tricks like Sharding in huggingface trainer.
Looking forward to your reply.
```


### Who can help?

Trainer: @sgugger

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

None

### Expected behavior

```shell
I wonder if there are any tricks like fairseq Sharding very large datasets https://fairseq.readthedocs.io/en/latest/getting_started.html.
Thanks a lot!
```

Comments:
The `Trainer` support `IterableDataset`, not just datasets.
Title: Autoeval config
Author: nazneenrajani (NONE)
State: closed
Created: 2022-04-27T05:32:10Z

Added autoeval config to imdb as pilot

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Related to: https://github.com/huggingface/autonlp-backend/issues/414 and https://github.com/huggingface/autonlp-backend/issues/424

The tests are failing due to the changed metadata:

```
got an unexpected keyword argument 'train-eval-index'
```

I think you can fix this by updating the `DatasetMetadata` class and implementing an appropriate `validate_train_eval_index()` function

@lhoestq we are working with an arbitrary set of tags for `autoeval config`. See https://github.com/huggingface/autonlp-backend/issues/414
I need to add a validator function though for the tests to pass. Our set is not well-defined as in the rest https://github.com/huggingface/datasets/tree/master/src/datasets/utils/resources. What's a workaround for this?

On the question of validating the `train-eval-index` metadata, I think the simplest approach would be to validate that the required fields exist and not worry about their values (which are open-ended).

For me, the required fields include:

* `config`
* `task`
* `task_id`
* `splits` (train / validation / eval)
* `col_mapping`
* `metrics` (checking that each one has `type`, `name`) 

Here I'm using the spec defined in https://github.com/huggingface/autonlp-backend/issues/414 as a guide.

WDYT @lhoestq ?

Makes sense ! Currently the metadata type validator doesn't support subfields - let me open a PR to add it

I ended up improving the metadata validation in this PR x)

In particular:
- I added support YAML keys with dashes instead of underscores for `train-eval-index`
- I added `train-eval-index` validation  with `validate_train_eval_index`. It does nothing fancy, it just checks that it is a list if it exists in the YAML, but feel free to improve it if you want

Let me know if it sounds good to you ! I think we can improve `validate_train_eval_index` in another PR

Come on windows... I didn't do anything advanced...

Anyway, will try to fix this when I get back home x)

> Come on windows... I didn't do anything advanced...
> 
> Anyway, will try to fix this when I get back home x)

Hehe, thanks!

Thanks, @lhoestq this is great! 

Did I just fix it for windows and now it fails on linux ? xD

> Did I just fix it for windows and now it fails on linux ? xD

Looks like the Heisenberg uncertainty principle is at play here - you cannot simultaneously have unit tests passing in both Linux and Windows 😅 

The worst is that the tests pass locally both on my windows and my linux x)

Ok fixed it, the issue came from python 3.6 that doesn't return the right `__origin__` for Dict and List types

> Alright thanks for adding the first Autoeval config ! :D

Woohoo! Thank you so much 🤗 

This is cool!
Title: Fix invalid url to CC-Aligned dataset
Author: juntang-zhuang (CONTRIBUTOR)
State: closed
Created: 2022-04-27T01:07:01Z

The CC-Aligned dataset url has changed to  https://data.statmt.org/cc-aligned/, the old address http://www.statmt.org/cc-aligned/ is no longer valid

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Why the `conll2003` dataset on huggingface only contains the `en` subset? Where is the  German data?
Author: beyondguo (NONE)
State: closed
Created: 2022-04-27T00:53:52Z
Labels: enhancement

![image](https://user-images.githubusercontent.com/37113676/165416606-96b5db18-b16c-4b6b-928c-de8620fd943e.png)

But on huggingface datasets:
![image](https://user-images.githubusercontent.com/37113676/165416649-8fd77980-ca0d-43f0-935e-f398ba8323a4.png)

Where is the  German data?

Comments:
Thanks for reporting @beyondguo.

Indeed, we generate this dataset from this raw data file URL: https://data.deepai.org/conll2003.zip
And that URL only contains the English version.

The German data requires payment

The [original task page](https://www.clips.uantwerpen.be/conll2003/ner/) states "The German data is a collection of articles from the Frankfurter Rundschau. The named entities have been annotated by people of the University of Antwerp. Only the annotations are available here. In order to build these data sets you need access to the ECI Multilingual Text Corpus. It can be ordered from the Linguistic Data Consortium (2003 non-member price: US$ 35.00)."

Inflation since 2003 has also affected LDC's prices, and today the dataset [LDC94T5](https://catalog.ldc.upenn.edu/LDC94T5) is available under license for $75 a copy. The [license](https://catalog.ldc.upenn.edu/license/eci-slash-mci-user-agreement.pdf) includes a non-distribution condition, which is probably why the data has not turned up openly.

The ACL hold copyright of this data; I'll mail them and anyone I can find at ECI to see if they'll open this up now. After all, it worked with Microsoft 3DMM, why not here too, after 28 years? :)


Closing this issue as we are not allowed to share publicly the German subset.
Title: Add pearsonr mc, update functionality to match the original docs
Author: emibaylor (CONTRIBUTOR)
State: closed
Created: 2022-04-26T18:30:46Z

- adds pearsonr metric card
- adds ability to return p-value
    - p-value was mentioned in the original docs as a return value, but there was no option to return it. I updated the _compute function slightly to have an option to return the p-value.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

thank you @lhoestq!! :hugs:
Title: Fix description links in dataset cards
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-04-26T14:36:25Z

I noticed many links were not properly displayed (only text, no link) on the Hub because of wrong syntax, e.g.: https://huggingface.co/datasets/big_patent

This PR fixes all description links in dataset cards.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Non passing tests are due to other pre-existing errors in dataset cards: not related to this PR.
Title: Dictionary Feature
Author: jordiae (NONE)
State: closed
Created: 2022-04-26T12:50:18Z
Labels: question

Hi, I'm trying to create the loading script for a dataset in which one feature is a list of dictionaries, which afaik doesn't fit very well the values and structures supported by Value and Sequence. Is there any suggested workaround, am I missing something?

Thank you in advance.

Comments:
Hi @jordiae,

Instead of the `Sequence` feature, you can use just a regular list: put the dict between `[` and `]`:
```python
"list_of_dict_feature": [
    {
        "key1_in_dict": datasets.Value("string"),
        "key2_in_dict": datasets.Value("int32"),
        ...
    }
],
```

Feel free to re-open this issue if that does not work for your use case.

> Hi @jordiae,
> 
> Instead of the `Sequence` feature, you can use just a regular list: put the dict between `[` and `]`:
> 
> ```python
> "list_of_dict_feature": [
>     {
>         "key1_in_dict": datasets.Value("string"),
>         "key2_in_dict": datasets.Value("int32"),
>         ...
>     }
> ],
> ```
> 
> Feel free to re-open this issue if that does not work for your use case.

Thank you
Title: Altered faiss installation comment
Author: vishalsrao (CONTRIBUTOR)
State: closed
Created: 2022-04-26T01:20:43Z


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi ! Can you explain why this change is needed ?

Facebook recommends installing FAISS using conda (https://github.com/facebookresearch/faiss/blob/main/INSTALL.md). pip does not seem to have the latest version of FAISS. The latest version of faiss is 1.7.2 (https://anaconda.org/conda-forge/faiss), but the latest one available through pip is 1.5.3 (https://pypi.org/project/faiss/).
Title: Big_Patent dataset broken
Author: Matthew-Larsen (NONE)
State: closed
Created: 2022-04-25T15:31:45Z
Labels: hosted-on-google-drive

## Dataset viewer issue for '*big_patent*'

**Link:** *[link to the dataset viewer page](https://huggingface.co/datasets/big_patent/viewer/all/train)*

*Unable to view because it says FileNotFound, also cannot download it through the python API*

Am I the one who added this dataset ? No

Comments:
Thanks for reporting. The issue seems not to be directly related to the dataset viewer or the `datasets` library, but instead to it being hosted on Google Drive.

See related issues: https://github.com/huggingface/datasets/issues?q=is%3Aissue+is%3Aopen+drive.google.com

To quote [@lhoestq](https://github.com/huggingface/datasets/issues/4075#issuecomment-1087362551):

> PS: if possible, please try to not use Google Drive links in your dataset script, since Google Drive has download quotas and is not always reliable.



We should find out if the dataset license allows redistribution and contact the data owners to propose them to host their data on our Hub.

The data owners have agreed on hosting their data on the Hub.
Title: Avoid recursion error in map if example is returned as dict value
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2022-04-25T14:40:32Z

I noticed this bug while answering [this question](https://discuss.huggingface.co/t/correct-way-to-create-a-dataset-from-a-csv-file/15686/11?u=mariosasko). 

This code replicates the bug:
```python
from datasets import Dataset
dset = Dataset.from_dict({"en": ["aa", "bb"], "fr": ["cc", "dd"]})
dset.map(lambda ex: {"translation": ex})
```
and this is the fix for it (before this PR):
```python
from datasets import Dataset
dset = Dataset.from_dict({"en": ["aa", "bb"], "fr": ["cc", "dd"]})
dset.map(lambda ex: {"translation": dict(ex)})
```

Internally, this can be fixed by merging two dicts via dict unpacking (instead of `dict.update) `in `Dataset.map`, which avoids creating recursive dictionaries.

P.S. `{**a, **b}` is slightly more performant than `a.update(b)` in my bencmarks.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Add `drop_last_batch` to `IterableDataset.map`
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2022-04-25T14:15:19Z

Addresses this comment: https://github.com/huggingface/datasets/pull/3801#pullrequestreview-901736921

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Skip checksum computation in Imagefolder by default
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2022-04-25T14:10:41Z

Avoids having to set `ignore_verifications=True` in `load_dataset("imagefolder", ...)` to skip checksum verification and speed up loading.

The user can still pass `DownloadConfig(record_checksums=True)` to not skip this part.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: [Common Voice] Make sure bytes are correctly deleted if `path` exists
Author: patrickvonplaten (CONTRIBUTOR)
State: closed
Created: 2022-04-25T13:18:26Z

`path` should be set to local path inside audio feature if exist so that bytes can correctly be deleted.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

cool that you noticed that we store unnecessary bytes again :D
Title: Add Nerval Metric
Author: maridda (NONE)
State: closed
Created: 2022-04-22T19:45:00Z
Labels: transfer-to-evaluate

This PR adds readme.md and ner_val.py to metrics.
Nerval is a python package that helps evaluate NER models. It creates classification report and confusion matrix at entity level.

Comments:
Metrics are deprecated in `datasets` and `evaluate` should be used instead: https://github.com/huggingface/evaluate
Title: Fix `convert_file_size_to_int` for kilobits and megabits
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2022-04-22T14:56:21Z

Minor change to fully align this function with the recent change in Transformers (https://github.com/huggingface/transformers/pull/16891)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Add Recall Metric Card
Author: emibaylor (CONTRIBUTOR)
State: closed
Created: 2022-04-22T14:24:26Z

What this PR mainly does:
- add metric card for recall metric
- update docs in recall python file

Note: I've also included a .json file with all of the metric card information. I've started compiling the relevant information in this type of .json files, and then using a script I wrote to generate the formatted metric card, as well as the docs to go in the .py file. I figured I'd upload the .json because it could be useful, especially if I also make a PR with the script I'm using (let me know if that's something you think would be beneficial!)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

This looks good to me!
Title: Add Precision Metric Card
Author: emibaylor (CONTRIBUTOR)
State: closed
Created: 2022-04-22T14:23:48Z

What this PR mainly does:
- add metric card for precision metric
- update docs in precision python file

Note: I've also included a .json file with all of the metric card information. I've started compiling the relevant information in this type of .json files, and then using a script I wrote to generate the formatted metric card, as well as the docs to go in the .py file. I figured I'd upload the .json because it could be useful, especially if I also make a PR with the script I'm using (let me know if that's something you think would be beneficial!)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Update GH template for dataset viewer issues
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-04-22T09:34:44Z

Update template to use new issue forms instead.

With this PR we can check if this new feature is useful for us.

Once validated, we can update the other templates.

CC: @severo

Comments:
_The documentation is not available anymore as the PR was closed or merged._

You can see rendering at: https://github.com/huggingface/datasets/blob/6b48fedbdafe12a42c7b6edcecc32820af1a4822/.github/ISSUE_TEMPLATE/dataset-viewer.yml