konrud/llm_course_chapter5_own_dataset_github_issues huggingface dataset:
Add long answer candidates to natural questions dataset

This is a modification of the Natural Questions dataset to include missing information specifically related to long answer candidates. (See here: https://github.com/google-research-datasets/natural-questions#long-answer-candidates). This information is important to ensure consistent comparison with prior work. It does not disturb the rest of the format . @lhoestq @albertvillanova 

_The documentation is not available anymore as the PR was closed or merged._

Once we have added `long_answer_candidates` maybe it would be worth to also add the missing `candidate_index` (inside `long_answer`). What do you think, @seirasto ?

Also note the "Data Fields" section in the README is missing the `long_answer` field.

Moreover, there is no instance example in "Data Instances" section.

We could either make these fixes in this PR or in a subsequent PR.

@albertvillanova I've added the missing fields and updated the README to include a data instance and some other things. 

Great! I've made the updates to align the README. Please let me know if I missed anything.

As there were many minor little fixes, I thought it would be easier to fix them directly.

I think the loading script is OK now. If it is also validated by another datasets maintainer, I could run the generation of the pre-processed data and then merge this PR into master (once all the tests are green).

CC: @lhoestq 

It looks good to me, thanks @seirasto !

I have merged the master branch, so that we include all the fixes on Apache Beam + Google Dataflow.

Pre-processing is running!

Already finished for "dev" config:
```python
In [2]: ds = load_dataset("datasets/natural_questions", "dev")

In [3]: ds
Out[3]: 
DatasetDict({
    validation: Dataset({
        features: ['id', 'document', 'question', 'long_answer_candidates', 'annotations'],
        num_rows: 7830
    })
})
```

There is an issue while running the preprocessing for the "default" (train+dev) config. Train data files are larger than than dev ones and workers run out of memory.

I'm opening a separate issue to handle this problem: #4525

@seirasto is proposing uploading their preprocessed data files to our Datasets bucket.

I think @lhoestq can give a more informed answer about authentication requirements.

Now that the data fiels are uploaded, can you merge the `main` branch into yours to re-trigger the CI @seirasto please ? :) Then I think we can merge if it's good for you @albertvillanova 

Merge is done! I think someone needs to approve the CI to run :) 

Can you run `make style` to fix the code formatting required by the CI please ?

Thanks @albertvillanova! I've committed all your suggestions.

The CI is green. I'm merging this PR.
Remove config names as yaml keys

Many datasets have dots in their config names. However it causes issues with the YAML tags of the dataset cards since we can't have dots in YAML keys. 

I fix this, I removed the tags separations per config name completely, and have a single flat YAML for all configurations. Dataset search doesn't use this info anyway. I removed all the config names used as YAML keys, and I moved them in under a new `config:` key.

This is related to https://github.com/huggingface/datasets/pull/2362 (internal https://github.com/huggingface/moon-landing/issues/946).

Also removing the dots in the YAML keys would allow us to do as in https://github.com/huggingface/datasets/pull/4302 which removes a hack that replaces all the dots by underscores in the YAML tags.

I also added a test in the CI that checks that all the YAML tags to make sure that:

- they can be parsed using a YAML parser
- they contain only valid YAML tags like languages or task_ids

I included the change from https://github.com/huggingface/datasets/pull/4302 directly in this PR, this way the datasets will be updated right away in the CI (the CI is only triggered when a dataset card is changed)

_The documentation is not available anymore as the PR was closed or merged._

Alright it's ready now :)

Here is an example for the `ade_corpus_v2` dataset card. Notice the new `configs` key:

https://github.com/huggingface/datasets/blob/76d9a141740a03f6836feb251f6059894b8d8046/datasets/ade_corpus_v2/README.md#L1-L78

CI failures are only related to dataset cards missing some content.
TypeError: __init__() missing 1 required positional argument: 'scheme'

  "name" : "node-1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "",
  "version" : {
    "number" : "7.5.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "",
    "build_date" : "2019-11-26T01:06:52.518245Z",
    "build_snapshot" : false,
    "lucene_version" : "8.3.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
 
when I run the order:
nohup python3 custom_service.pyc > service.log 2>&1&

the log:
nohup: 忽略输入
Traceback (most recent call last):
  File "/home/xfz/p3_custom_test/custom_service.py", line 55, in <module>
  File "/home/xfz/p3_custom_test/custom_service.py", line 48, in doInitialize
  File "custom_impl.py", line 286, in custom_setup
  File "custom_impl.py", line 127, in create_es_index
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/_sync/client/__init__.py", line 345, in __init__
    ssl_show_warn=ssl_show_warn,
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/_sync/client/utils.py", line 105, in client_node_configs
    node_configs = hosts_to_node_configs(hosts)
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/_sync/client/utils.py", line 154, in hosts_to_node_configs
    node_configs.append(host_mapping_to_node_config(host))
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/_sync/client/utils.py", line 221, in host_mapping_to_node_config
    return NodeConfig(**options)  # type: ignore
TypeError: __init__() missing 1 required positional argument: 'scheme'
[1]+  退出 1                nohup python3 custom_service.pyc > service.log 2>&1

custom_service_pyc can't running



Duplicate of:
- #3956

I think you should report that issue to `elasticsearch` library: https://github.com/elastic/elasticsearch-py
Remove dots in config names

20+ datasets have dots in their config names. However it causes issues with the YAML tags of the dataset cards since we can't have dots in YAML keys.

This is related to https://github.com/huggingface/datasets/pull/2362 (internal https://github.com/huggingface/moon-landing/issues/946).

Also removing the dots in the config names would allow us to merge https://github.com/huggingface/datasets/pull/4302 which removes a hack that replaces all the dots by underscores in the YAML tags.

I also added a test in the CI that checks that all the YAML tags to make sure that:
- they can be parsed using a YAML parser
- they contain only valid YAML tags like `languages` or `task_ids`
- they contain valid config names (no invalid characters `<>:/\|?*.`)

_The documentation is not available anymore as the PR was closed or merged._

Closing in favor of https://github.com/huggingface/datasets/pull/4367
Support complex feature types as `features` in packaged loaders 

This PR adds `table_cast` to the packaged loaders to fix casting to the `Image`/`Audio`, `ArrayND` and `ClassLabel` types. If these types are not present in the `builder.config.features` dictionary, the built-in `pa.Table.cast` is used for better performance. Additionally, this PR adds `cast_storage` to `ClassLabel` to support the string to int conversion in `table_cast` and ensure that integer labels are in a valid range.

Fix https://github.com/huggingface/datasets/issues/4210

This PR is also a solution for these (popular) discussions: https://discuss.huggingface.co/t/converting-string-label-to-int/2816 and https://discuss.huggingface.co/t/class-labels-for-custom-datasets/15130/2

TODO:
* [x] tests

_The documentation is not available anymore as the PR was closed or merged._
The dataset preview is not available for this split.

I have uploaded the corpus developed by our lab in the speech domain to huggingface [datasets](https://huggingface.co/datasets/Roh/ryanspeech). You can read about the companion paper accepted in interspeech 2021 [here](https://arxiv.org/abs/2106.08468). The dataset works fine but I can't make the dataset preview work. It gives me the following error that I don't understand. Can you help me to begin debugging it?

```
Status code:   400
Exception:     AttributeError
Message:       'NoneType' object has no attribute 'split'
``` 

Hi!  A dataset has to be streamable to work with the viewer. I did a quick test, and yours is, so this might be a bug in the viewer. cc @severo 


Looking at it. The message is now:

```
Message:       cannot cache function '__shear_dense': no locator available for file '/src/services/worker/.venv/lib/python3.9/site-packages/librosa/util/utils.py'
```

so possibly it's related to the libraries versions?


Maybe this SO thread can help: https://stackoverflow.com/questions/59290386/runtimeerror-at-cannot-cache-function-shear-dense-no-locator-available-fo

Same error for https://huggingface.co/datasets/LIUM/tedlium/viewer/release1/test. cc @sanchit-gandhi . I'm on it

Fixed in the datasets viewer, by setting the `NUMBA_CACHE_DIR` env var to a writable directory.

https://huggingface.co/datasets/Roh/ryanspeech/viewer/male/train

<img width="1538" alt="Capture d’écran 2022-06-08 à 11 30 08" src="https://user-images.githubusercontent.com/1676121/172583285-4cd49a0f-5715-423b-95dd-5f6ace3b2416.png">


https://huggingface.co/datasets/LIUM/tedlium/viewer/

<img width="1538" alt="Capture d’écran 2022-06-08 à 14 31 52" src="https://user-images.githubusercontent.com/1676121/172616897-fbcb7df7-0308-4d09-a17d-48826bc91374.png">
Update dataset_infos for UDHN/udhr dataset

Checksum update to `udhr` for issue #4361

_The documentation is not available anymore as the PR was closed or merged._

Thanks for contributing @leondz.

The checksums of the files have changed because more languages have been added:
- the new language codes need to be added to the dataset card (README file)
- I think the dataset version number should also be increased, so that users who had previously cached it, get a new dataset download (with the additional languages)

Yep! All done (also fixed the language tags in the README which were iso639-3 instead of the expected bcp47)

I guess the language code CI failure is due to languages.json being a subset of bcp47 (see issue #4304), happy to contribute a solution here, e.g. autogeneration of the lang list from the relevant isos and the ietf bcp47 subtag register or full code for validation

> Thanks again for your contribution, @leondz.
> 
> Yes, I think it is OK to set version 1.0.0 (as previous was 0.0.0).
> 
> One of the CI failures is related to dummy data: once you have updated the dataset version, the dummy_data ZIP file should be moved from "dummy/0.0.0/dummy_data.zip" to "dummy/1.0.0/dummy_data.zip".

Oh, thanks, I missed that one


> Other CI failure is related to missing languages in our resources file. This has been addressed in this PR:
> 
> * #4371
> 
> You should merge master branch into your feature branch to incorporate that fix.

Yeah, I saw this :) I already have the merge, thanks. I'm talking about the longer-term picture: every time another language code comes up (e.g. da-bornholm or es-VE), the json will need updating, because the current approach is non-exhaustive manual whitelisting instead of relying on the established bcp standard.
Fix example in opus_ubuntu, Add license info

This PR 
* fixes a typo in the example for the`opus_ubuntu` dataset where it's mistakenly referred to as `ubuntu`
* adds the declared license info for this corpus' origin
* adds an example instance
* updates the data origin type

CI seems to fail due to languages incorrectly being flagged as invalid, I guess that's related to the currently-broken bcp47 validation (see #4304)

_The documentation is not available anymore as the PR was closed or merged._
Fix Version equality

I think `Version` equality should align with other similar cases in Python, like:
```python
In [1]: "a" == 5, "a" == None
Out[1]: (False, False)

In [2]: "a" != 5, "a" != None
Out[2]: (True, True)
```

With this PR, we will get:
```python
In [3]: Version("1.0.0") == 5, Version("1.0.0") == None
Out[3]: (False, False)

In [4]: Version("1.0.0") != 5, Version("1.0.0") != None
Out[4]: (True, True)
```

Note I found this issue when `doc-builder` tried to compare:
```python
if param.default != inspect._empty
```
where `param.default` is an instance of `Version`.

_The documentation is not available anymore as the PR was closed or merged._
Missing dataset tags and sections in some dataset cards

Summary of CircleCI errors for different dataset metadata:

- **BoolQ**: missing 8 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', 'source_datasets', 'task_categories', and 'task_ids'
- **Conllpp**: expected some content in section `Citation Information` but it is empty.
- **GLUE**:   'annotations_creators', 'language_creators', 'source_datasets' :['unknown'] are not registered tags
- **ConLL2003**:  field 'task_ids':  ['part-of-speech-tagging'] are not registered tags for 'task_ids'
-  **Hate_speech18:**   Expected some content in section `Data Instances` but it is empty, Expected some content in section `Data Splits` but it is empty
- **Jjigsaw_toxicity_pred**:  `Citation Information` but it is empty.
- **LIAR** :  `Data Instances`,`Data Fields`,  `Data Splits`,  `Citation Information` are empty.
- **MSRA NER** : Dataset Summary`, `Data Instances`, `Data Fields`,  `Data Splits`, `Citation Information` are empty.
- **sem_eval_2010_task_8**: missing 8 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', 'source_datasets', 'task_categories', and 'task_ids'
- **sms_spam**: `Data Instances` and`Data Splits` are empty.
- **Quora** : Expected some content in section `Citation Information` but it is empty,  missing 8 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', 'source_datasets', 'task_categories', and 'task_ids'
- **sentiment140**:  missing 8 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', 'source_datasets', 'task_categories', and 'task_ids'

@lhoestq I can take this issue. Please can you point out to me where I can find the other positional arguments?

Hi @RohitRathore1 :)

You can find all the YAML tags in the tagging app here: https://hf.co/spaces/huggingface/datasets-tagging). They're all passed as arguments to a DatasetMetadata object used to validate the tags.
Fix dataset builder default version

Currently, when using a custom config (subclass of `BuilderConfig`), default version set at the builder level is ignored: we must set default version in the custom config class.

However, when loading a dataset with `config_kwargs` (for a configuration not present in `BUILDER_CONFIGS`), the default version set in the custom config is ignored and "0.0.0" is used instead:
```python
ds = load_dataset("wikipedia", language="co", date="20220501", beam_runner="DirectRunner")
```
generates the following config:
```python
WikipediaConfig(name='20220501.co', version=0.0.0, data_dir=None, data_files=None, description='Wikipedia dataset for co, parsed from 20220501 dump.')
```
with version "0.0.0" instead of "2.0.0".

See as a counter-example, when the config is present in `BUILDER_CONFIGS`:
```python
ds = load_dataset("wikipedia", "20220301.fr", beam_runner="DirectRunner")
```
generates the following config:
```python
WikipediaConfig(name='20220301.fr', version=2.0.0, data_dir=None, data_files=None, description='Wikipedia dataset for fr, parsed from 20220301 dump.')
```
with correct version "2.0.0", as set in the custom config class.


The reason for this is that `DatasetBuilder` has a default VERSION ("0.0.0") that overwrites the default version set at the custom config class.

This PR:
- Removes the default VERSION at `DatasetBuilder` (set to None, so that the class attribute exists but it does not override the custom config default version).
- Note that the `BuilderConfig` class already sets a default version = "0.0.0"; no need to pass this from the builder.

_The documentation is not available anymore as the PR was closed or merged._

This PR requires one of these other PRs being merged first:
- #4359 
- huggingface/doc-builder#211
Add optional progress bar for .save_to_disk(..) and .load_from_disk(..) when working with remote filesystems

**Is your feature request related to a problem? Please describe.**
When working with large datasets stored on remote filesystems(such as s3), the process of uploading a dataset could take really long time. For instance: I was uploading a re-processed version of wmt17 en-ru to my s3 bucket and it took like 35 minutes(and that's given that I have a fiber optic connection). The only output during that process was a progress bar for flattening indices and then ~35 minutes of complete silence.

**Describe the solution you'd like**
I want to be able to enable a progress bar when calling .save_to_disk(..) and .load_from_disk(..), it would track either amount of bytes sent/received or amount of records written/loaded, and will give some ETA. Basically just tqdm. 

**Describe alternatives you've considered**
- Save dataset to tmp folder at the disk and then upload it using custom wrapper over botocore, which will work with progress bar, like [this](https://alexwlchan.net/2021/04/s3-progress-bars/).

Hi! I like this idea. For consistency with `load_dataset`, we can use `fsspec`'s `TqdmCallback` in  `.load_from_disk` to monitor the number of bytes downloaded, and in `.save_to_disk`, we can track the number of saved shards for consistency with `push_to_hub` (after we implement https://github.com/huggingface/datasets/issues/4196).
Add a new metric: CTC_Consistency

Add CTC_Consistency metric
Do I also need to modify the `test_metric_common.py` file to make it run on test?

Thanks for your contribution, @YEdenZ.

Please note that our old `metrics` module is in the process of being incorporated to a separate library called `evaluate`: https://github.com/huggingface/evaluate

Therefore, I would ask you to transfer your PR to that repository. Thank you.
`inspect` functions can't fetch dataset script from the Hub

The `inspect_dataset` and `inspect_metric` functions are unable to retrieve a dataset or metric script from the Hub and store it locally at the specified `local_path`:

```py
>>> from datasets import inspect_dataset
>>> inspect_dataset('rotten_tomatoes', local_path='path/to/my/local/folder')

FileNotFoundError: Couldn't find a dataset script at /content/rotten_tomatoes/rotten_tomatoes.py or any data file in the same directory.
```

Hi, thanks for reporting! `git bisect` points to #2986 as the PR that introduced the bug. Since then, there have been some additional changes to the loading logic, and in the current state, `force_local_path` (set via `local_path`) forbids pulling a script from the internet instead of downloading it: https://github.com/huggingface/datasets/blob/cfae0545b2ba05452e16136cacc7d370b4b186a1/src/datasets/inspect.py#L89-L91

cc @lhoestq: `force_local_path` is only used in `inspect_dataset` and `inspect_metric`. Is it OK if we revert the behavior to match the old one?

Good catch ! Yea I think it's fine :)
Support remote cache_dir

This PR implements complete support for remote `cache_dir`. Before, the support was just partial.

This is useful to create datasets using Apache Beam (parallel data processing) builder with `cache_dir` in a remote bucket, e.g., for Wikipedia dataset.

_The documentation is not available anymore as the PR was closed or merged._

@lhoestq thanks for your review.

Please note that `xjoin` cannot be used in this context, as it always returns a POSIX path string and this is not suitable on Windows machines.


`xjoin` returns windows paths (not posix) on windows, since it just extends`os.path.join` 


Actually you are right.

https://github.com/huggingface/datasets/blob/08ec04ccb59630a3029b2ecd8a14d327bddd0c4a/src/datasets/utils/streaming_download_manager.py#L104-L105

Though this is not an issue because posix paths (as returned by Path().as_posix()) work on windows. That's why we can replace `os.path.join` with `xjoin` in streaming mode. They look like `c:/Program Files/` or something (can't confirm right now, I don't have a windows with me)

Until now, we have always replaced "/" in paths with `os.path.join` (`os.sep`,...) in order to support Windows paths (that contain r"\\").

Now, you suggest ignoring this and work with POSIX strings (with "/").

As an example, when passing `cache_dir=r"C:\Users\Username\.mycache"`:
- Until now, it results in `self._cache_downloaded_dir = r"C:\Users\Username\.mycache\downloads"`
- If we use `xjoin`, it will give `self._cache_downloaded_dir = "C:/Users/Username/.mycache/downloads"`

You say this is OK and we don't care if we work with POSIX strings on Windows machines.

I'm incorporating your suggested changes then...

Also note that using `xjoin`, if we pass `cache_dir="C:\\Users\\Username\\.mycache"`, we get:
- `self._cache_dir_root = "C:\\Users\\Username\\.mycache"`
- `self._cache_downloaded_dir = "C:/Users/Username/.mycache/downloads"`

It looks like it broke the CI on windows :/ maybe this was not a good idea, sorry
Fix never ending GH Action to build documentation

There was an unclosed code block introduced by:
- #4313 

https://github.com/huggingface/datasets/pull/4313/files#diff-f933ce41f71c6c0d1ce658e27de62cbe0b45d777e9e68056dd012ac3eb9324f7R538 

This causes the "Make documentation" step in the "Build documentation" workflow to never finish.
- I think this issue should also be addressed in the `doc-builder` lib.


Fix #4346.

_The documentation is not available anymore as the PR was closed or merged._
Metrics documentation is not accessible in the datasets doc UI

**Is your feature request related to a problem? Please describe.**
Search for a metric name like "seqeval" yields no results on https://huggingface.co/docs/datasets/master/en/index . One needs to go look in `datasets/metrics/README.md` to find the doc. Even in the `README.md`, it can be hard to understand what the metric expects as an input, for example for `squad` there is a [key `id`](https://github.com/huggingface/datasets/blob/1a4c185663a6958f48ec69624473fdc154a36a9d/metrics/squad/squad.py#L42) documented only in the function doc but not in the `README.md`, and one needs to go look into the code to understand what the metric expects.

**Describe the solution you'd like**
Have the documentation for metrics appear as well in the doc UI, e.g. this https://github.com/huggingface/datasets/blob/1a4c185663a6958f48ec69624473fdc154a36a9d/metrics/squad/squad.py#L21-L63

I know there are plans to migrate metrics to the evaluate library, but just pointing this out.


Hey @fxmarty :) Yes we are working on showing the docs of all the metrics on the Hugging face website. If you want to follow the advancements you can check the [evaluate](https://github.com/huggingface/evaluate) repository cc @lvwerra @sashavor
Failing CI on Windows for sari and wiki_split metrics

## Describe the bug
Our CI is failing from yesterday on Windows for metrics: sari and wiki_split
```
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_sari - ...
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_wiki_split
```

See: https://app.circleci.com/pipelines/github/huggingface/datasets/11928/workflows/79daa5e7-65c9-4e85-829b-00d2bfbd076a/jobs/71594
Eval metadata Batch 4: Tweet Eval, Tweets Hate Speech Detection, VCTK, Weibo NER, Wisesight Sentiment, XSum, Yahoo Answers Topics, Yelp Polarity, Yelp Review Full

Adding evaluation metadata for:

- Tweet Eval
- Tweets Hate Speech Detection
- VCTK
- Weibo NER
- Wisesight Sentiment
- XSum
- Yahoo Answers Topics
- Yelp Polarity
- Yelp Review Full

Summary of CircleCI errors:

- **XSum**: missing 6 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', and 'source_datasets'
- **Yelp_polarity**: missing 8 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', 'source_datasets', 'task_categories', and 'task_ids'

_The documentation is not available anymore as the PR was closed or merged._
Eval metadata batch 3: Reddit, Rotten Tomatoes, SemEval 2010, Sentiment 140, SMS Spam, Snips, SQuAD, SQuAD v2, Timit ASR

Adding evaluation metadata for:
- Reddit
- Rotten Tomatoes
- SemEval 2010
- Sentiment 140
- SMS Spam
- Snips
- SQuAD
- SQuAD v2
- Timit ASR

Summary of CircleCI errors:

- **sem_eval_2010_task_8**: missing 8 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', 'source_datasets', 'task_categories', and 'task_ids'
- **sms_spam**: `Data Instances` and`Data Splits` are empty.
- **Quora** : Expected some content in section `Citation Information` but it is empty,  missing 8 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', 'source_datasets', 'task_categories', and 'task_ids'
- **sentiment140**:  missing 8 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', 'source_datasets', 'task_categories', and 'task_ids'

There are also some timeout errors, I don't really understand the source though :confused: 

_The documentation is not available anymore as the PR was closed or merged._
Eval metadata batch 2 : Health Fact, Jigsaw Toxicity, LIAR, LJ Speech, MSRA NER, Multi News, NCBI Disease, Poem Sentiment

Adding evaluation metadata for : 
- Health Fact
- Jigsaw Toxicity
- LIAR
- LJ Speech
- MSRA NER
- Multi News
- NCBI Diseas
- Poem Sentiment

Summary of CircleCI errors:
- **Jjigsaw_toxicity_pred**:  `Citation Information` but it is empty.
- **LIAR** :  `Data Instances`,`Data Fields`,  `Data Splits`,  `Citation Information` are empty.
- **MSRA NER** : Dataset Summary`, `Data Instances`, `Data Fields`,  `Data Splits`, `Citation Information` are empty.


The CI errors about missing content in the dataset cards can be ignored in this PR btw

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4336). All of your documentation changes will be reflected on that endpoint.
Eval metadata batch 1: BillSum, CoNLL2003, CoNLLPP, CUAD, Emotion, GigaWord, GLUE, Hate Speech 18,  Hate Speech

Adding evaluation metadata for:
- BillSum
- CoNLL2003
- CoNLLPP
- CUAD
- Emotion
- GigaWord
- GLUE
- Hate Speech 18 
- Hate Speech Offensive

Summary of CircleCI errors:
- **BoolQ**: missing 8 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', 'source_datasets', 'task_categories', and 'task_ids'
- **Conllpp**: expected some content in section `Citation Information` but it is empty.
- **GLUE**:   'annotations_creators', 'language_creators', 'source_datasets' :['unknown'] are not registered tags
- **ConLL2003**:  field 'task_ids':  ['part-of-speech-tagging'] are not registered tags for 'task_ids'
-  **Hate_speech18:**   Expected some content in section `Data Instances` but it is empty, Expected some content in section `Data Splits` but it is empty

And yes we can ignore all the CI errors related to missing content in the dataset cards, these issues can be fixed in other PRs

_The documentation is not available anymore as the PR was closed or merged._
`wikipedia` pre-processed datasets

## Describe the bug
[Wikipedia](https://huggingface.co/datasets/wikipedia) dataset readme says that certain subsets are preprocessed. However it seems like they are not available. When I try to load them it takes a really long time, and it seems like it's processing them.

## Steps to reproduce the bug
```python
from datasets import load_dataset

load_dataset("wikipedia", "20220301.en")
```

## Expected results
To load the dataset

## Actual results
Takes a very long time to load (after downloading)

After `Downloading data files: 100%`. It takes hours and gets killed.
Tried `wikipedia.simple` and it got processed after ~30mins.

Hi @vpj, thanks for reporting.

I'm sorry, but I can't reproduce your bug: I load "20220301.simple"in 9 seconds:
```shell
time python -c "from datasets import load_dataset; load_dataset('wikipedia', '20220301.simple')"

Downloading and preparing dataset wikipedia/20220301.simple (download: 228.58 MiB, generated: 224.18 MiB, post-processed: Unknown size, total: 452.76 MiB) to .../.cache/huggingface/datasets/wikipedia/20220301.simple/2.0.0/aa542ed919df55cc5d3347f42dd4521d05ca68751f50dbc32bae2a7f1e167559...
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.66k/1.66k [00:00<00:00, 1.02MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 235M/235M [00:02<00:00, 82.8MB/s]
Dataset wikipedia downloaded and prepared to .../.cache/huggingface/datasets/wikipedia/20220301.simple/2.0.0/aa542ed919df55cc5d3347f42dd4521d05ca68751f50dbc32bae2a7f1e167559. Subsequent calls will reuse this data.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 290.75it/s]

real	0m9.693s
user	0m6.002s
sys	0m3.260s
```

Could you please check your environment info, as requested when opening this issue?
```
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform:
- Python version:
- PyArrow version:
```
Maybe you are using an old version of `datasets`...

Downloading and processing `wikipedia simple` dataset completed in under 11sec on M1 Mac. Could you please check `dataset` version as mentioned by @albertvillanova? Also check system specs, if system is under load processing could take some time I guess.
Fix type hint and documentation for `new_fingerprint`

Currently, there are no type hints nor `Optional` for the argument `new_fingerprint` in several methods of `datasets.arrow_dataset.Dataset`.

There was some documentation missing as well.

Note that pylance is happy with the type hints, but pyright does not detect that `new_fingerprint` is set within the decorator.

The modifications in this PR are fine since here https://github.com/huggingface/datasets/blob/aa743886221d76afb409d263e1b136e7a71fe2b4/src/datasets/fingerprint.py#L446-L454

for the non-inplace case we make sure to auto-generate a new fingerprint (as indicated in the doc).

_The documentation is not available anymore as the PR was closed or merged._
Dataset Viewer issue for strombergnlp/offenseval_2020, strombergnlp/polstance

### Link

https://huggingface.co/datasets/strombergnlp/offenseval_2020/viewer/ar/train

### Description

The viewer isn't running for these two datasets. I left it overnight because a wait sometimes helps things get loaded, and the error messages have all gone, but the datasets are still turning up blank in viewer. Maybe it needs a bit more time.

* https://huggingface.co/datasets/strombergnlp/polstance/viewer/PolStance/train

* https://huggingface.co/datasets/strombergnlp/offenseval_2020/viewer/ar/train

While offenseval_2020 is gated w. prompt, the other gated previews I have run fine in Viewer, e.g. https://huggingface.co/datasets/strombergnlp/shaj , so I'm a bit stumped!

### Owner

Yes

Not sure if it's related... I was going to raise an issue for https://huggingface.co/datasets/domenicrosati/TruthfulQA which also has the same issue... https://huggingface.co/datasets/domenicrosati/TruthfulQA/viewer/domenicrosati--TruthfulQA/train 



Yes, it's related. The backend behind the dataset viewer is currently under too much load, and these datasets are still in the jobs queue. We're actively working on this issue, and we expect to fix the issue permanently soon. Thanks for your patience 🙏  

Thanks @severo and no worries! - a suggestion for a UI usability thing maybe is to indicate that the dataset processing is in the job queue (rather than no data?)

Thanks, these are working great now (including @domenicrosati 's, afaics!)
Support >1 PWC dataset per dataset card

**Is your feature request related to a problem? Please describe.**
Some datasets cover more than one dataset on PapersWithCode. For example, the OffensEval 2020 challenge involved five languages, and there's one dataset to cover all five datasets, [`strombergnlp/offenseval_2020`](https://huggingface.co/datasets/strombergnlp/offenseval_2020). However, the yaml `paperswithcode_id:` dataset card entry only supports one value; when multiple are added, the PWC link disappears from the dataset page.

Because the link from a PapersWithCode dataset to a Hugging Face Hub entry can't be entered manually and seems to be scraped, this means end users don't have a way of getting a dataset reader link to appear on all the PWC datasets supported by one HF Hub Dataset reader.

It's not super unusual to have papers introduce multiple parallel variants of a dataset and would be handy to reflect this, so e.g. dataset maintainers can DRY, and so dataset users can keep what they're doing simple.

**Describe the solution you'd like**
I'd like `paperswithcode_id:` to support lists and be able to connect with multiple PWC datasets.

**Describe alternatives you've considered**
De-normalising the datasets on HF Hub to create multiple readers for each variation on a task, i.e. instead of a single `offenseval_2020`, having `offenseval_2020_ar`, `offenseval_2020_da`, `offenseval_2020_gr`, ...

**Additional context**
Hope that's enough

**Priority**
Low

Hi @leondz, I agree it would be nice. We'll see what we can do ;)
Multi-news dataset loader attempts to strip wrong character from beginning of summaries

## Describe the bug

The `multi_news.py` data loader has [a line which attempts to strip `"- "` from the beginning of summaries](https://github.com/huggingface/datasets/blob/aa743886221d76afb409d263e1b136e7a71fe2b4/datasets/multi_news/multi_news.py#L97). The actual character in the multi-news dataset, however, is `"– "`, which is different, e.g. `"– " != "- "`.

I would have just opened a PR to fix the mistake, but I am wondering what the motivation for stripping this character is? AFAICT most approaches just leave it in, e.g. the current SOTA on this dataset, [PRIMERA](https://huggingface.co/allenai/PRIMERA-multinews) (you can see its in the generated summaries of the model in their [example notebook](https://github.com/allenai/PRIMER/blob/main/Evaluation_Example.ipynb)).

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.2.0
- Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.13
- PyArrow version: 6.0.1
- Pandas version: 1.3.5


Hi ! Thanks for reporting :)

This dataset was simply converted from [tensorflow datasets](https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/summarization/multi_news.py)

I think we can just remove the `.strip("- ")` and keep this character

Cool! I made a PR.
Fix cnn_dailymail (dm stories were ignored)

https://github.com/huggingface/datasets/pull/4188 introduced a bug in `datasets` 2.2.0: DailyMail stories are ignored when generating the dataset.

I fixed that, and removed the google drive link (it has annoying quota limitations issues)

We can do a patch release after this is merged

_The documentation is not available anymore as the PR was closed or merged._
[Imagefolder] Docs + Don't infer labels from file names when there are metadata + Error messages when metadata and images aren't linked correctly

I updated the `docs/source/image_process.mdx` documentation and added an example for image captioning and object detection using `ImageFolder`.

While doing so I also improved a few aspects:
- we don't need to infer labels from file names when there are metadata - they can just be in the metadata if necessary
- raise informative error messages when metadata and images aren't linked correctly:
  - when an image is missing a metadata file
  - when a metadata file is missing an image

I added some tests for these changes as well

cc @mariosasko 

_The documentation is not available anymore as the PR was closed or merged._

Merging this one since mario is off, I took care of adding some tests to make sure everything is fine. Will do the release after it
Fixes FrugalScore

There are two minor modifications in this PR:
1) `predictions` and `references` are swapped. Basically Frugalscore is commutative, however some tiny differences can occur if we swap the references and the predictions. I decided to swap them just to obtain the exact results as reported in the paper.
2) I switched to dynamic padding that was was used in the training, forcing the padding to `max_length`  introduces errors for some reason that I ignore.

@lhoestq 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4305). All of your documentation changes will be reflected on that endpoint.

> predictions and references are swapped. Basically Frugalscore is commutative, however some tiny differences can occur if we swap the references and the predictions. I decided to swap them just to obtain the exact results as reported in the paper.

What is the order of magnitude of the difference ? Do you know what causes this ?

> I switched to dynamic padding that was was used in the training, forcing the padding to max_length introduces errors for some reason that I ignore.

What error ?
Language code search does direct matches

## Describe the bug

Hi. Searching for bcp47 tags that are just the language prefix (e.g. `sq` or `da`) excludes datasets that have added extra information in their language metadata (e.g. `sq-AL` or `da-bornholm`). The example codes given in the [tagging app](https://huggingface.co/spaces/huggingface/datasets-tagging) encourages addition of the additional codes ("_expected format is BCP47 tags separated for ';' e.g. 'en-US;fr-FR'_") but this would lead to those datasets being hidden in datasets search.

## Steps to reproduce the bug
1. Add a dataset using a variant tag (e.g. [`sq-AL`](https://huggingface.co/datasets?languages=languages:sq-AL))
2. Look for datasets using the full code 
3. Note that they're missing when just the language is searched for (e.g. [`sq`](https://huggingface.co/datasets?languages=languages:sq))

Some datasets are already affected by this - e.g. `AmazonScience/massive` is listed under `sq-AL` but not `sq`.

One workaround is for dataset creators to add an additional root language tag to dataset YAML metadata, but it's unclear how to communicate this. It might be possible to index the search on `languagecode.split('-')[0]` but I wanted to float this issue before trying to write any code :)

## Expected results
Datasets using longer bcp47 tags also appear under searches for just the language code; e.g. Quebecois datasets (`fr-CA`) would come up when looking for French datasets with no region specification (`fr`), or US English (`en-US`) datasets would come up when searching for English datasets (`en`).

## Actual results
The language codes seem to be directly string matched, excluding datasets with specific language tags from non-specific searches.

## Environment info
(web app)

Thanks for reporting ! I forwarded the issue to the front-end team :)

Will keep you posted !

I also changed the tagging app to suggest two letters code for now.
Remove hacking license tags when mirroring datasets on the Hub

Currently, when mirroring datasets on the Hub, the license tags are hacked: removed of characters "." and "$". On the contrary, this hacking is not applied to community datasets on the Hub. This generates multiple variants of the same tag on the Hub. 

I guess this hacking is no longer necessary:
- it is not applied to community datasets
- all canonical datasets are validated by maintainers before being merged: CI + maintainers make sure license tags are the right ones

Fix #4298.

_The documentation is not available anymore as the PR was closed or merged._

The Hub doesn't allow these characters in the YAML tags, and git push fails if you want to push a dataset card containing these characters.

Ok, let me rename the bad config names :) I think I can also keep backward compatibility with a warning

Almost done with it btw, will submit a PR that shows all the configuration name changes (from a bit more than 20 datasets)

Please, let me know when the renaming of configs is done. If not enough bandwidth, I can take care of it...

Will focus on this this afternoon ;)

I realized when renaming all the configurations with dots in https://github.com/huggingface/datasets/pull/4365 that it's not ideal for certain cases. For example:
- many configurations have a version like "1.0.0" in their names
- to avoid breaking changes we need to replace dots with underscores in the user input and show a warning, which hurts the experience
- our second most downloaded dataset at the moment is affected: `newsgroup`
- if we disallow dots, then we'll never be able to make the [allenai/c4](https://huggingface.co/datasets/allenai/c4) work with its different configurations since they contain dots, and we can't rename them because they are the official download links

I was thinking of other alternatives:
1. just stop separating tags per config name completely, and have a single flat YAML for all configurations. Dataset search doesn't use this info anyway
2. use another YAML structure to avoid having config names as keys, such as
```yaml
languages:
- config: 20220301_en
  values:
  - en
```

I'm down for 1, to keep things simple

@lhoestq I agree:
- better not changing config names (so that we do not introduce any braking change)
- therefore, we should not use them as keys

In relation with the proposed solutions, I have no strong opinion:
- option 1 is simpler and aligns better with current usage on the Hub (configs are ignored)
- however:
  - we will lose all the information per config we already have (for those datasets containing config keys; contributors made an effort to put that information per config)
  - and this information might be useful on the Hub in the future, in case we would like to enrich the search feature with more granularity; this is only applicable if this feature could eventually make sense

So, no strong opinion...

Closing in favor of https://github.com/huggingface/datasets/pull/4367
Add ImageNet-Sketch dataset

This PR adds the ImageNet-Sketch dataset and resolves #3953 .

_The documentation is not available anymore as the PR was closed or merged._

I think you can go ahead with uploading the data, and also ping the author in parallel. I think the images may subject to copyright anyway (scrapped from google image) so the dataset author is not allowed to set a license to the data.

I think it's fine to upload the dataset as soon as we mention explicitly that the images may be subject to copyright.
Add API code examples for loading methods

This PR adds API code examples for loading methods, let me know if I've missed any important parameters we should showcase :)

I was a bit confused about `inspect_dataset` and `inspect_metric`. The `path` parameter says it will accept a dataset identifier from the Hub. But when I try the identifier `rotten_tomatoes`, it gives me:

```py
from datasets import inspect_dataset
inspect_dataset('rotten_tomatoes', local_path='/content/rotten_tomatoes')

FileNotFoundError: Couldn't find a dataset script at /content/rotten_tomatoes/rotten_tomatoes.py or any data file in the same directory.
```

Does the user need to have an existing copy of `rotten_tomatoes.py` on their local drive (in which case, it seems like the same option as the first option in `path`)?

_The documentation is not available anymore as the PR was closed or merged._
Remove manual download from imagenet-1k

Remove the manual download code from `imagenet-1k` to make it a regular dataset.

_The documentation is not available anymore as the PR was closed or merged._

Thanks for the reviews @apsdehal and @lhoestq! As suggested by @lhoestq, I'll separate the train/val/test splits, apply the validation split fixes and shuffle the images files to simplify the script and make streaming faster.

@apsdehal I dismissed your review as it's no longer relevant after the data files changes suggested by @lhoestq.
Normalise license names

**Is your feature request related to a problem? Please describe.**
When browsing datasets, the Licenses tag cloud (bottom left of e.g. https://huggingface.co/datasets) has multiple variants of the same license. This means the options exclude datasets arbitrarily, giving users artificially low recall. The cause of the dupes is probably due to a bit of variation in metadata.

**Describe the solution you'd like**
I'd like the licenses in metadata to follow the same standard as much as possible, to remove this problem. I'd like to go ahead and normalise the dataset metadata to follow the format & values given in [src/datasets/utils/resources/licenses.json](https://github.com/huggingface/datasets/blob/master/src/datasets/utils/resources/licenses.json) .

**Describe alternatives you've considered**
None

**Additional context**
None

**Priority** 
Low


we'll add the same server-side metadata validation system as for hf.co/models soon-ish

(you can check on hf.co/models that licenses are "clean")

Fixed by #4367.
Datasets YAML tagging space is down

## Describe the bug
The neat hf spaces app for generating YAML tags for dataset `README.md`s is down

## Steps to reproduce the bug
1. Visit https://huggingface.co/spaces/huggingface/datasets-tagging

## Expected results
There'll be a HF spaces web app for generating dataset metadata YAML

## Actual results
There's an error message; here's the step where it breaks:

```
Step 18/29 : RUN pip install -r requirements.txt
 ---> Running in e88bfe7e7e0c
Defaulting to user installation because normal site-packages is not writeable
Collecting git+https://github.com/huggingface/datasets.git@update-task-list (from -r requirements.txt (line 4))
  Cloning https://github.com/huggingface/datasets.git (to revision update-task-list) to /tmp/pip-req-build-bm8t0r0k
  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/datasets.git /tmp/pip-req-build-bm8t0r0k
  WARNING: Did not find branch or tag 'update-task-list', assuming revision or ref.
  Running command git checkout -q update-task-list
  error: pathspec 'update-task-list' did not match any file(s) known to git
  error: subprocess-exited-with-error
  
  × git checkout -q update-task-list did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× git checkout -q update-task-list did not run successfully.
│ exit code: 1
╰─> See above for output.
```

## Environment info

- Platform: Linux / Brave


@lhoestq @albertvillanova `update-task-list` branch does not exist anymore, should point to `main` now i guess

Thanks for reporting, fixing it now

It's up again :)