sunny2309/githug-issues huggingface dataset:
Regarding Test Sets for the GEM datasets

@yjernite Hi, are the test sets for the GEM datasets scheduled to be [added soon](https://gem-benchmark.com/shared_task)? 

e.g.

```
from datasets import load_dataset
DATASET_NAME="common_gen"
data = load_dataset("gem", DATASET_NAME)
```

The test set doesn't have the target or references.

```
data['test'][0]
{'concept_set_id': 0, 'concepts': ['drill', 'field', 'run', 'team'], 'gem_id': 'common_gen-test-0', 'gem_parent_id': 'common_gen-test-0', 'references': [], 'target': ''}
```
How to convert datasets.arrow_dataset.Dataset to torch.utils.data.Dataset

Hi,

I'm trying to pretraine deep-speed model using HF arxiv dataset like:
```
train_ds = nlp.load_dataset('scientific_papers', 'arxiv')
train_ds.set_format(
        type="torch",
        columns=["input_ids", "attention_mask", "global_attention_mask", "labels"],
    )
engine, _, _, _ = deepspeed.initialize(
    args=args,
    model=model,
    model_parameters=[p for p in model.parameters() if p.requires_grad],
    training_data=train_ds)
```
but deepspeed.initialize accepts torch.utils.data.Dataset only. How can I convert HF-style dataset to torch-style dataset?
data_args.preprocessing_num_workers almost freezes 

Hi @lhoestq 

I am running this code from huggingface transformers https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm.py 

to speed up tokenization, since I am running on multiple datasets, I am using data_args.preprocessing_num_workers = 4 with opus100 corpus but this moves on till a point and then this freezes almost for sometime during  tokenization steps and then this is back again, overall to me taking more time than normal case, I appreciate your advice on how I can use this option properly to speed up.

thanks
adding ccnet dataset

## Adding a Dataset
- **Name:**  ccnet

- **Description:** 
Common Crawl

- **Paper:** 
https://arxiv.org/abs/1911.00359

- **Data:** 
https://github.com/facebookresearch/cc_net

- **Motivation:**
this is one of the most comprehensive clean monolingual datasets across a variety of languages. Quite important for cross-lingual reseach

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

thanks
viewer "fake_news_english" error

When I visit the [Huggingface - viewer](https://huggingface.co/datasets/viewer/) web site, under the dataset "fake_news_english" I've got this error:

> ImportError: To be able to use this dataset, you need to install the following dependencies['openpyxl'] using 'pip install # noqa: requires this pandas optional dependency for reading xlsx files' for instance'

as well as the error Traceback.
Adding the NorNE dataset for Norwegian POS and NER

NorNE is a manually annotated corpus of named entities which extends the annotation of the existing Norwegian Dependency Treebank. Comprising both of the official standards of written Norwegian (Bokmål and Nynorsk), the corpus contains around 600,000 tokens and annotates a rich set of entity types including persons, organizations, locations, geo-political entities, products, and events, in addition to a class corresponding to nominals derived from names.

See #1720.
load_dataset ignoring features

First of all, I'm sorry if it is a repeated issue or the changes are already in master, I searched and I didn't find anything. 

I'm using datasets 1.5.0

![image](https://user-images.githubusercontent.com/37592763/113114369-8f376580-920b-11eb-900d-94365b59f04b.png)

As you can see, when I load the dataset, the ClassLabels are ignored, I have to cast the dataset in order to make it work.

Code to reproduce:

```python
import datasets
data_location = "/data/prueba_multiclase"
features = datasets.Features(
        {"texto": datasets.Value("string"), "label": datasets.features.ClassLabel(names=["false", "true"])}
    )
dataset = datasets.load_dataset(
        "csv", data_files=data_location, delimiter="\t", features=features
    )
```

Dataset I used:


[prueba_multiclase.zip](https://github.com/huggingface/datasets/files/6235022/prueba_multiclase.zip) (it has to be unzipped)


Thank you! ❤️
Telugu subset missing for xtreme tatoeba dataset

from nlp import load_dataset
train_dataset = load_dataset('xtreme', 'tatoeba.tel')['validation']
ValueError: BuilderConfig tatoeba.tel not found.

but language tel is actually included in xtreme:
https://github.com/google-research/xtreme/blob/master/utils_preprocess.py
def tatoeba_preprocess(args):
  lang3_dict = {
    'afr':'af', 'ara':'ar', 'bul':'bg', 'ben':'bn',
    'deu':'de', 'ell':'el', 'spa':'es', 'est':'et',
    'eus':'eu', 'pes':'fa', 'fin':'fi', 'fra':'fr',
    'heb':'he', 'hin':'hi', 'hun':'hu', 'ind':'id',
    'ita':'it', 'jpn':'ja', 'jav':'jv', 'kat':'ka',
    'kaz':'kk', 'kor':'ko', 'mal':'ml', 'mar':'mr',
    'nld':'nl', 'por':'pt', 'rus':'ru', 'swh':'sw',
    'tam':'ta', **_'tel':'te'_**, 'tha':'th', 'tgl':'tl', <----here
    'tur':'tr', 'urd':'ur', 'vie':'vi', 'cmn':'zh',
    'eng':'en',
  }
Add configurable options to `seqeval` metric

Right now `load_metric("seqeval")` only works in the default mode of evaluation (equivalent to conll evaluation).

However, seqeval library [supports](https://github.com/chakki-works/seqeval#support-features) different evaluation schemes (IOB1, IOB2, etc.), which can be plugged in just by supporting additional kwargs in `Seqeval._compute`
https://github.com/huggingface/datasets/blob/85cf7ff920c90ca2e12bedca12b36d2a043c3da2/metrics/seqeval/seqeval.py#L109

Things that would be relevant are, for example, supporting `mode="strict", scheme=IOB2` to count only full entity match as a true positive and omit partial matches.

The only problem I see is that the spirit of `metrics` seems to not require additional imports from user. `seqeval` only supports schemes as objects, without any string aliases. 

It can be solved naively with mapping like `{"IOB2": seqeval.scheme.IOB2}`. Or just left as is and require user to explicitly import scheme from `seqeval` if he wants to configure it past the default implementation.

If that makes sense, I am happy to implement the change.
Dataset file size on disk is very large with 3D Array

Hi, 

I have created my own dataset using the provided dataset loading script. It is an image dataset where images are stored as 3D Array with dtype=uint8. 

The actual size on disk is surprisingly large. It takes 520 MB. Here is some info from `dataset_info.json`. 

`{
    "description": "",
    "citation": "",
    "homepage": "",
    "license": "",
    "features": {
        "image": {
            "shape": [224, 224, 3],
            "dtype": "uint8",
            "id": null,
            "_type": "Array3D",
        }
    },
    "post_processed": null,
    "supervised_keys": null,
    "builder_name": "shot_type_image_dataset",
    "config_name": "default",
    "version": {
        "version_str": "0.0.0",
        "description": null,
        "major": 0,
        "minor": 0,
        "patch": 0,
    },
    "splits": {
        "train": {
            "name": "train",
            "num_bytes": 520803408,
            "num_examples": 1479,
            "dataset_name": "shot_type_image_dataset",
        }
    },
    "download_checksums": {
        "": {
            "num_bytes": 16940447118,
            "checksum": "5854035705efe08b0ed8f3cf3da7b4d29cba9055c2d2d702c79785350d72ee03",
        }
    },
    "download_size": 16940447118,
    "post_processing_size": null,
    "dataset_size": 520803408,
    "size_in_bytes": 17461250526,
}`

I have created the same dataset with tensorflow_dataset and it takes only 125MB on disk.

I am wondering, is it normal behavior ? I understand `Datasets` uses Arrow for serialization wheres tf uses TF Records.

This might be a problem for large dataset. 

Thanks for your help.
TypeError when using save_to_disk in a dataset loaded with ReadInstruction split

Hi,

Loading a dataset with `load_dataset` using a split defined via `ReadInstruction` and then saving it to disk results in the following error: `TypeError: Object of type ReadInstruction is not JSON serializable`.

Here is the minimal reproducible example:

```python
from datasets import load_dataset
from datasets import ReadInstruction

data_1 = load_dataset(
    "wikiann",
    "en",
    split="validation",
)

data_1.save_to_disk("temporary_path_1")

print("Save with regular split works.")

data_2 = load_dataset(
    "wikiann",
    "en",
    split=ReadInstruction("validation", to=50, unit="%"),
)

data_2.save_to_disk("temporary_path_2")
```

and the corresponding output:

```
Reusing dataset wikiann (/xxxxx/.cache/huggingface/datasets/wikiann/en/1.1.0/0b11a6fb31eea02f38ca17610657bfba3206100685283014daceb8da291c3be9)
Save with regular split works.
Reusing dataset wikiann (/xxxxx/.cache/huggingface/datasets/wikiann/en/1.1.0/0b11a6fb31eea02f38ca17610657bfba3206100685283014daceb8da291c3be9)
Traceback (most recent call last):
  File "bug.py", line 20, in <module>
    data_2.save_to_disk("temporary_path_2")
  File "/xxxxx/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 645, in save_to_disk
    json.dump(state, state_file, indent=2, sort_keys=True)
  File "/usr/lib/python3.7/json/__init__.py", line 179, in dump
    for chunk in iterable:
  File "/usr/lib/python3.7/json/encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/usr/lib/python3.7/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.7/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/usr/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type ReadInstruction is not JSON serializable
```

Let me know if there is some misuse from my end.

Thanks in advance.
bug in mlqa dataset 

Hi 
Looking into MLQA dataset for langauge "ar":

```
 "question": [
    "\u0645\u062a\u0649 \u0628\u062f\u0627\u062a \u0627\u0644\u0645\u062c\u0644\u0629 \u0627\u0644\u0645\u062f\u0631\u0633\u064a\u0629 \u0641\u064a \u0646\u0648\u062a\u0631\u062f\u0627\u0645 \u0628\u0627\u0644\u0646\u0634\u0631?",
    "\u0643\u0645 \u0645\u0631\u0629 \u064a\u062a\u0645 \u0646\u0634\u0631\u0647\u0627 \u0641\u064a \u0646\u0648\u062a\u0631\u062f\u0627\u0645?",
    "\u0645\u0627 \u0647\u064a \u0627\u0644\u0648\u0631\u0642\u0629 \u0627\u0644\u064a\u0648\u0645\u064a\u0629 \u0644\u0644\u0637\u0644\u0627\u0628 \u0641\u064a \u0646\u0648\u062a\u0631\u062f\u0627\u0645?",
    "\u0643\u0645 \u0639\u062f\u062f \u0627\u0644\u0627\u0648\u0631\u0627\u0642 \u0627\u0644\u0627\u062e\u0628\u0627\u0631\u064a\u0629 \u0644\u0644\u0637\u0644\u0627\u0628 \u0627\u0644\u062a\u064a \u0648\u062c\u062f\u062a \u0641\u064a \u0646\u0648\u062a\u0631\u062f\u0627\u0645?",
    "\u0641\u064a \u0627\u064a \u0633\u0646\u0629 \u0628\u062f\u0627\u062a \u0648\u0631\u0642\u0629 \u0627\u0644\u0637\u0627\u0644\u0628 \u0627\u0644\u062d\u0633 \u0627\u0644\u0633\u0644\u064a\u0645 \u0628\u0627\u0644\u0646\u0634\u0631 \u0641\u064a \u0646\u0648\u062a\u0631\u062f\u0627\u0645?"
  ]
```

the questions are in the wrong format, and not readable, could you please have a look? thanks @lhoestq
TydiQA dataset is mixed and is not split per language 

Hi @lhoestq 
Currently TydiQA is mixed and user can only access the whole training set of all languages:
https://www.tensorflow.org/datasets/catalog/tydi_qa

for using this dataset, one need to train/evaluate in each separate language, and having them mixed, makes it hard to use this dataset. This is much convenient for user to have  them split and I appreciate your help on this. 

Meanwhile, till hopefully this is split per language, I greatly appreciate telling me how I can preprocess and get data per language. thanks a lot
When training with Multi-Node Multi-GPU the worker 2 has TypeError: 'NoneType' object

version: 1.5.0
met a very strange error, I am training large scale language model, and need train on 2 machines(workers).
And sometimes I will get this error `TypeError: 'NoneType' object is not iterable`
This is traceback
```

71 |   | Traceback (most recent call last):
-- | -- | --
72 |   | File "run_gpt.py", line 316, in <module>
73 |   | main()
74 |   | File "run_gpt.py", line 222, in main
75 |   | delimiter="\t", column_names=["input_ids", "attention_mask", "chinese_ref"])
76 |   | File "/data/miniconda3/lib/python3.7/site-packages/datasets/load.py", line 747, in load_dataset
77 |   | use_auth_token=use_auth_token,
78 |   | File "/data/miniconda3/lib/python3.7/site-packages/datasets/builder.py", line 513, in download_and_prepare
79 |   | self.download_post_processing_resources(dl_manager)
80 |   | File "/data/miniconda3/lib/python3.7/site-packages/datasets/builder.py", line 673, in download_post_processing_resources
81 |   | for split in self.info.splits:
82 |   | TypeError: 'NoneType' object is not iterable
83 |   | WARNING:datasets.builder:Reusing dataset csv (/usr/local/app/.cache/huggingface/datasets/csv/default-1c257ebd48e225e7/0.0.0/2960f95a26e85d40ca41a230ac88787f715ee3003edaacb8b1f0891e9f04dda2)
84 |   | Traceback (most recent call last):
85 |   | File "/data/miniconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
86 |   | "__main__", mod_spec)
87 |   | File "/data/miniconda3/lib/python3.7/runpy.py", line 85, in _run_code
88 |   | exec(code, run_globals)
89 |   | File "/data/miniconda3/lib/python3.7/site-packages/torch/distributed/launch.py", line 340, in <module>
90 |   | main()
91 |   | File "/data/miniconda3/lib/python3.7/site-packages/torch/distributed/launch.py", line 326, in main
92 |   | sigkill_handler(signal.SIGTERM, None)  # not coming back
93 |   | File "/data/miniconda3/lib/python3.7/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
94 |   | raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)

```
On worker 1 it loads the dataset well, however on worker 2 will get this error. 
And I will meet this error from time to time, sometimes it just goes well.
Dialogue action slot name and value are reversed in MultiWoZ 2.2

Hi @yjernite, thank you for adding MultiWoZ 2.2 in the huggingface datasets platform. It is beneficial!

I spot an error that the order of Dialogue action slot names and values are reversed.

https://github.com/huggingface/datasets/blob/649b2c469779bc4221e1b6969aa2496d63eb5953/datasets/multi_woz_v22/multi_woz_v22.py#L251-L262
Is dataset timit_asr broken?

Using `timit_asr` dataset, I saw all records are the same.

``` python
from datasets import load_dataset, load_metric

timit = load_dataset("timit_asr")

from datasets import ClassLabel
import random
import pandas as pd
from IPython.display import display, HTML

def show_random_elements(dataset, num_examples=10):
    assert num_examples <= len(dataset), "Can't pick more elements than there are in the dataset."
    picks = []
    for _ in range(num_examples):
        pick = random.randint(0, len(dataset)-1)
        while pick in picks:
            pick = random.randint(0, len(dataset)-1)
        picks.append(pick)

    df = pd.DataFrame(dataset[picks])
    display(HTML(df.to_html()))


show_random_elements(timit['train'].remove_columns(["file", "phonetic_detail", "word_detail", "dialect_region", "id", 
                                                    "sentence_type", "speaker_id"]), num_examples=20)

```

`output`

<img width="312" alt="Screen Shot 2021-03-28 at 17 29 04" src="https://user-images.githubusercontent.com/42398050/112746646-21acee80-8feb-11eb-84f3-dbb5d4269724.png">


I double-checked it [here](https://huggingface.co/datasets/viewer/), and met the same problem.

<img width="1374" alt="Screen Shot 2021-03-28 at 17 32 07" src="https://user-images.githubusercontent.com/42398050/112746698-9bdd7300-8feb-11eb-97ed-5babead385f4.png">
Adding ScaNN library to do MIPS?

@lhoestq Hi I am thinking of adding this new google library to do the MIPS similar to **add_faiss_idex**. As the paper suggests, it is really fast when it comes to retrieving the nearest neighbors. 

https://github.com/google-research/google-research/tree/master/scann

![image](https://user-images.githubusercontent.com/16892570/112738294-78ec9800-8fc6-11eb-9a5f-3d7ee5818e76.png)
Problem downloading GEM wiki_auto_asset_turk dataset

@yjernite 

### Summary

I am currently working on the GEM datasets and do not manage to download the wiki_auto_asset_turk data, whereas all other datasets download well with the same code.

### Steps to reproduce
Code snippet:

from datasets import load_dataset
#dataset = load_dataset('gem', 'web_nlg_en')
dataset = load_dataset('gem', 'wiki_auto_asset_turk')

```

**Expected behavior:**

I expect the dataset to start downloading (download bar appears and progresses toward 100%)

**Actual behavior:**
Instead of seeing the download bar appearing, nothing happens; the following appears in the console as expected, but nothing more:

Downloading: 36.6kB [00:00, 37.2MB/s]
Downloading: 41.7kB [00:00, ?B/s]
Downloading and preparing dataset gem/wiki_auto_asset_turk (download: 121.37 MiB, generated: 145.69 MiB, post-processed: Unknown size, total: 267.07 MiB) to C:\Users\sfmil\.cache\huggingface\datasets\gem\wiki_auto_asset_turk\1.0.0\f252756d7f1b8f019aac71a1623b2950acfe10d25d956668ac4eae4e93c58b8d...

### Is this a regression?
No, it was the first time I was trying to download this dataset (same for the other ones).

### Debug info
- Python version: Python 3.8.2
- OS version: Windows 10 Family
load_metric from local "glue.py" meet error 'NoneType' object is not callable

actual_task = "mnli" if task == "mnli-mm" else task
dataset = load_dataset(path='/home/glue.py', name=actual_task)
metric = load_metric(path='/home/glue.py', name=actual_task)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-7ab77a465d81> in <module>
      1 actual_task = "mnli" if task == "mnli-mm" else task
      2 dataset = load_dataset(path='/home/jcli/glue.py', name=actual_task)
----> 3 metric = load_metric(path='/home/jcli/glue.py', name=actual_task)

~/anaconda3/envs/pytorch/lib/python3.6/site-packages/datasets/load.py in load_metric(path, config_name, process_id, num_process, cache_dir, experiment_id, keep_in_memory, download_config, download_mode, script_version, **metric_init_kwargs)
    508         keep_in_memory=keep_in_memory,
    509         experiment_id=experiment_id,
--> 510         **metric_init_kwargs,
    511     )
    512 

TypeError: 'NoneType' object is not callable

Please help
Creating custom dataset results in error while calling the map() function

calling `map()` of `datasets` library results into an error while defining a Custom dataset.
Reproducible example:
```
import datasets
class MyDataset(datasets.Dataset):

    def __init__(self, sentences):
        "Initialization"
        self.samples = sentences

    def __len__(self):
        "Denotes the total number of samples"
        return len(self.samples)

    def __getitem__(self, index):
        "Generates one sample of data"
        # Select sample
        # Load data and get label
        samples = self.samples[index]

        return samples

def preprocess_function_train(examples):
        inputs = examples
        labels = [example+tokenizer.eos_token for example in examples ]
        inputs = tokenizer(inputs, max_length=30, padding=True, truncation=True)
        labels = tokenizer(labels, max_length=30, padding=True, truncation=True)
        model_inputs = inputs
        model_inputs["labels"] = labels["input_ids"]
        print("about to return")
        return model_inputs


##train["sentence"] is dataframe column
train_dataset = MyDataset(train['sentence'].values.tolist())
train_dataset = train_dataset.map(
            preprocess_function,
            batched = True,
            batch_size=32
        )
```

Stack trace of error:
```
Traceback (most recent call last):
  File "dir/train_generate.py", line 362, in <module>
    main()
  File "dir/train_generate.py", line 245, in main
    train_dataset = train_dataset.map(
  File "anaconda_dir/anaconda3/envs/env1/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 1244, in map
    return self._map_single(
  File "anaconda_dir/anaconda3/envs/env1/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 149, in wrapper
    unformatted_columns = set(self.column_names) - set(self._format_columns or [])
  File "anaconda_dir/anaconda3/envs/env1/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 526, in column_names
    return self._data.column_names
AttributeError: 'MyDataset' object has no attribute '_data'
```
Support for legal NLP datasets (EURLEX, ECtHR cases and EU-REG-IR)

Add support for two legal NLP datasets:

- EURLEX (https://www.aclweb.org/anthology/P19-1636/)
- ECtHR cases (https://arxiv.org/abs/2103.13084)
- EU-REG-IR (https://arxiv.org/abs/2101.10726)
Add more issue templates and customize issue template chooser

When opening an issue, it is not evident for the users how to choose a blank issue template. There is a link at the bottom of all the other issue templates (`Don’t see your issue here? Open a blank issue.`), but this is not very visible for users. This is the reason why many users finally chose the `add-dataset` template instead (this is more visible) for issues that indeed are not requesting the addition of a new dataset.

~~With this PR, the default blank issue template would be as visible as the other templates (as the `add-dataset` template), thus making easier for the users to choose it.~~

With this PR:
- more issue templates, besides `add-dataset`, are added: `bug-report` and `feature-request`
- the issue template chooser is customized, so that it now includes a link to `Discussions` for questions
Metadata validation

- `pydantic` metadata schema with dedicated validators against our taxonomy
- ci script to validate new changes against this schema and start a vertuous loop
- soft validation on tasks ids since we expect the taxonomy to undergo some changes in the near future

for reference with the current validation we have ~365~ 378 datasets with invalid metadata! full error report [_here_.](https://gist.github.com/theo-m/61b3c0c47fc6121d08d3174bd4c2a26b)