sunny2309/githug-issues huggingface dataset:
Issues in processing very large datasets

## Describe the bug
I'm trying to add a feature called "subgraph" to CNN/DM dataset (modifications on run_summarization.py of Huggingface Transformers script) --- I'm not quite sure if I'm doing it the right way, though--- but the main problem appears when the training starts where the error ` [OSError: [Errno 12] Cannot allocate memory]`  appears. I suppose this problem roots in RAM issues and how the dataset is loaded during training, but I have no clue of what I can do to fix it.  Observing the dataset's cache directory, I see that it takes ~600GB of memory and that's why I believe special care is needed when loading it into the memory. 


Here are my modifications to `run_summarization.py` code. 


```
# loading pre-computed dictionary where keys are 'id' of article and values are corresponding subgraph
graph_data_train = get_graph_data('train') 
graph_data_validation = get_graph_data('val')
...
...


with training_args.main_process_first(desc="train dataset map pre-processing"):
    train_dataset = train_dataset.map(
        preprocess_function_train,
        batched=True,
        num_proc=data_args.preprocessing_num_workers,
        remove_columns=column_names,
        load_from_cache_file=not data_args.overwrite_cache,
        desc="Running tokenizer on train dataset",
    )

```


And here is the modified preprocessed function:

```
def preprocess_function_train(examples):
        inputs, targets, sub_graphs, ids = [], [], [], []
        for i in range(len(examples[text_column])):
            if examples[text_column][i] is not None and examples[summary_column][i] is not None:
                # if examples['doc_id'][i] in graph_data.keys():
                inputs.append(examples[text_column][i])
                targets.append(examples[summary_column][i])
                sub_graphs.append(graph_data_train[examples['id'][i]])
                ids.append(examples['id'][i])

        inputs = [prefix + inp for inp in inputs]
        model_inputs = tokenizer(inputs, max_length=data_args.max_source_length, padding=padding, truncation=True,
                                 sub_graphs=sub_graphs, ids=ids)

            # Setup the tokenizer for targets
        with tokenizer.as_target_tokenizer():
            labels = tokenizer(targets, max_length=max_target_length, padding=padding, truncation=True)

        # If we are padding here, replace all tokenizer.pad_token_id in the labels by -100 when we want to ignore
        # padding in the loss.
        if padding == "max_length" and data_args.ignore_pad_token_for_loss:
            labels["input_ids"] = [
                [(l if l != tokenizer.pad_token_id else -100) for l in label] for label in labels["input_ids"]
            ]

        model_inputs["labels"] = labels["input_ids"]
        return model_inputs
```


## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:  2.1.0
- Platform: Linux Ubuntu
- Python version: 3.6
- PyArrow version: 6.0.1
Don't do unnecessary list type casting to avoid replacing None values by empty lists

In certain cases, `None` values are replaced by empty lists when casting feature types.

It happens every time you cast an array of nested lists like [None, [0, 1, 2, 3]] to a different type (to change the integer precision for example). In this case you'd get [[], [0, 1, 2, 3]] for example. This issue comes from PyArrow, see the discussion in https://github.com/huggingface/datasets/issues/3676

This issue also happens when no type casting is needed, because casting is supposed to be a no-op in this case. But as https://github.com/huggingface/datasets/issues/3676 shown, it's not the case and `None` are replaced by empty lists even if we cast to the exact same type.

In this PR I just workaround this bug in the case where no type casting is needed. In particular, I only call `pa.ListArray.from_arrays` only when necessary.

I also added a warning when some `None` are effectively replaced by empty lists. I wanted to raise an error in this case, but maybe we should wait a major update to do so

This PR fixes this particular case, that is occurring in `run_qa.py` in `transformers`:
```python
from datasets import Dataset

ds = Dataset.from_dict({"a": range(4)})
ds = ds.map(lambda x: {"b": [[None, [0]]]}, batched=True, batch_size=1, remove_columns=["a"])
print(ds.to_pandas())
# before:
#              b
# 0  [None, [0]]
# 1    [[], [0]]
# 2    [[], [0]]
# 3    [[], [0]]
#
# now:
#              b
# 0  [None, [0]]
# 1  [None, [0]]
# 2  [None, [0]]
# 3  [None, [0]]
```

cc @sgugger
Enable label alignment for token classification datasets

This PR extends the `Dataset.align_labels_with_mapping()` method to support alignment of label mappings between datasets and models for token classification (e.g. NER).

Example of usage:

```python
from datasets import load_dataset

ner_ds = load_dataset("conll2003", split="train")
# returns [3, 0, 7, 0, 0, 0, 7, 0, 0]
ner_ds[0]["ner_tags"]
# hypothetical model mapping with O <--> B-LOC
label2id = {
    "B-LOC": "0",
    "B-MISC": "7",
    "B-ORG": "3",
    "B-PER": "1",
    "I-LOC": "6",
    "I-MISC": "8",
    "I-ORG": "4",
    "I-PER": "2",
    "O": "5"
  }
ner_aligned_ds = ner_ds.align_labels_with_mapping(label2id, "ner_tags")
# returns [3, 5, 7, 5, 5, 5, 7, 5, 5]
ner_aligned_ds[0]["ner_tags"]
```

Context: we need this in AutoTrain to automatically align datasets / models during evaluation. cc @abhishekkrthakur
OpenBookQA has missing and inconsistent field names

## Describe the bug
OpenBookQA implementation is inconsistent with the original dataset.

We need to:

1. The dataset field [question][stem] is flattened into question_stem. Unflatten it to match the original format.
2. Add missing additional fields:
    - 'fact1': row['fact1'],
    - 'humanScore': row['humanScore'],
    - 'clarity': row['clarity'],
    - 'turkIdAnonymized': row['turkIdAnonymized']
3. Ensure the structure and every data item in the original OpenBookQA matches our OpenBookQA version.

## Expected results
The structure and every data item in the original OpenBookQA matches our OpenBookQA version.

## Actual results
TBD

## Environment info
- `datasets` version: 2.1.0
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.13
- PyArrow version: 7.0.0
- Pandas version: 1.4.2
CommonSenseQA has missing and inconsistent field names

## Describe the bug
In short, CommonSenseQA implementation is inconsistent with the original dataset.

More precisely, we need to:

1. Add the dataset matching "id" field. The current dataset, instead, regenerates monotonically increasing id.  
2. The [“question”][“stem”] field is flattened into "question". We should match the original dataset and unflatten it
3. Add the missing "question_concept" field in the question tree node
4. Anything else? Go over the data structure of the newly repaired CommonSenseQA and make sure it matches the original

## Expected results
Every data item of the CommonSenseQA should structurally and data-wise match the original CommonSenseQA dataset.

## Actual results
TBD

## Environment info
- `datasets` version: 2.1.0
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.13
- PyArrow version: 7.0.0
- Pandas version: 1.4.2
error downloading bigscience-catalogue-lm-data/lm_en_wiktionary_filtered

## Describe the bug
Error generated when attempting to download dataset

## Steps to reproduce the bug
```python
from datasets import load_dataset

dataset = load_dataset("bigscience-catalogue-lm-data/lm_en_wiktionary_filtered")
```

## Expected results
A clear and concise description of the expected results.

## Actual results
```
ExpectedMoreDownloadedFiles               Traceback (most recent call last)

[<ipython-input-62-4ac5cf959477>](https://localhost:8080/#) in <module>()
      1 from datasets import load_dataset
      2 
----> 3 dataset = load_dataset("bigscience-catalogue-lm-data/lm_en_wiktionary_filtered")

3 frames

[/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py](https://localhost:8080/#) in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     31         return
     32     if len(set(expected_checksums) - set(recorded_checksums)) > 0:
---> 33         raise ExpectedMoreDownloadedFiles(str(set(expected_checksums) - set(recorded_checksums)))
     34     if len(set(recorded_checksums) - set(expected_checksums)) > 0:
     35         raise UnexpectedDownloadedFile(str(set(recorded_checksums) - set(expected_checksums)))

ExpectedMoreDownloadedFiles: {'/home/leandro/catalogue_data/datasets/lm_en_wiktionary_filtered/data/file-01.jsonl.gz', '/home/leandro/catalogue_data/datasets/lm_en_wiktionary_filtered/data/file-01.jsonl.gz.lock'}
```


## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.3
- Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.13
- PyArrow version: 6.0.1
Add HF Speech Bench to Librispeech Dataset Card

Adds the HF Speech Bench to Librispeech Dataset Card in place of the Papers With Code Leaderboard. Should improve usage and visibility of this leaderboard! Wondering whether this can also be done for [Common Voice 7](https://huggingface.co/datasets/mozilla-foundation/common_voice_7_0) and [8](https://huggingface.co/datasets/mozilla-foundation/common_voice_8_0) through someone with permissions? 

cc @patrickvonplaten: more leaderboard promotion!
Rename imagenet2012 -> imagenet-1k

On the Hugging Face Hub, users refer to imagenet2012 (from #4178 ) as imagenet-1k in their model tags.

To correctly link models to imagenet, we should rename this dataset `imagenet-1k`.

Later we can add `imagenet-21k` as a new dataset if we want.

Once this one is merged we can delete the `imagenet2012` dataset repository on the Hub.

EDIT: to complete the rationale on why we should name it `imagenet-1k`:
If users specifically added the tag `imagenet-1k` , then it could be for two reasons (not sure which one is predominant), either they
- wanted to make it explicit that it’s not 21k -> the distinction is important for the community
- or they have been following this convention from other models -> the convention implicitly exists already
data leakage in `webis/conclugen` dataset

## Describe the bug
Some samples (argument-conclusion pairs) in the *training* split of the `webis/conclugen` dataset are present in both the *validation* and *test* splits, creating data leakage and distorting model results.
Furthermore, all splits contain duplicate samples.

## Steps to reproduce the bug
```python
from datasets import load_dataset

training = load_dataset("webis/conclugen", "base", split="train")
validation = load_dataset("webis/conclugen", "base", split="validation")
testing = load_dataset("webis/conclugen", "base", split="test")

# collect which sample id's are present in the training split
ids_validation = list()
ids_testing = list()

for train_sample in training:
    train_argument = train_sample["argument"]
    train_conclusion = train_sample["conclusion"]
    train_id = train_sample["id"]
    
    # test if current sample is in validation split
    if train_argument in validation["argument"]:
        for validation_sample in validation:
            validation_argument = validation_sample["argument"]
            validation_conclusion = validation_sample["conclusion"]
            validation_id = validation_sample["id"]
            if train_argument == validation_argument and train_conclusion == validation_conclusion:
                ids_validation.append(validation_id)
    
    # test if current sample is in test split
    if train_argument in testing["argument"]:
        for testing_sample in testing:
            testing_argument = testing_sample["argument"]
            testing_conclusion = testing_sample["conclusion"]
            testing_id = testing_sample["id"]
            if train_argument == testing_argument and train_conclusion == testing_conclusion:
                ids_testing.append(testing_id)
```

## Expected results
Length of both lists `ids_validation` and `ids_testing` should be zero.

## Actual results
Length of `ids_validation` = `2556`
Length of `ids_testing` = `287`

Furthermore, there seems to be duplicate samples in (at least) the *training* split, since:
`print(len(set(ids_validation)))` = `950`
`print(len(set(ids_testing)))` = `101`

All in all, around 7% of the samples of each the *validation* and *test* split seems to be present in the *training* split.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.4
- Platform: macOS-12.3.1-arm64-arm-64bit
- Python version: 3.9.10
- PyArrow version: 7.0.0
Add mr_polarity movie review sentiment classification

Add the MR (Movie Review) dataset. The original dataset contains sentences from Rotten Tomatoes labeled as either "positive" or  "negative". 

Homepage: [https://www.cs.cornell.edu/people/pabo/movie-review-data/](https://www.cs.cornell.edu/people/pabo/movie-review-data/)
paperswithcode: [https://paperswithcode.com/dataset/mr](https://paperswithcode.com/dataset/mr)

- [ ] I was not able to generate dummy data, the original dataset files have ".pos" and ".neg" as file extensions so the auto-generator does not work. Is it fine like this or should dummy data be added?
conll2003 dataset loads original data.

## Describe the bug
I load `conll2003` dataset to use refined data like [this](https://huggingface.co/datasets/conll2003/viewer/conll2003/train)  preview, but it is original data that contains `'-DOCSTART- -X- -X- O'` text.

Is this a bug or should I use another dataset_name like `lhoestq/conll2003` ?

## Steps to reproduce the bug
```python
import datasets
from datasets import load_dataset
dataset = load_dataset("conll2003")
```

## Expected results
{
    "chunk_tags": [11, 12, 12, 21, 13, 11, 11, 21, 13, 11, 12, 13, 11, 21, 22, 11, 12, 17, 11, 21, 17, 11, 12, 12, 21, 22, 22, 13, 11, 0],
    "id": "0",
    "ner_tags": [0, 3, 4, 0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
    "pos_tags": [12, 22, 22, 38, 15, 22, 28, 38, 15, 16, 21, 35, 24, 35, 37, 16, 21, 15, 24, 41, 15, 16, 21, 21, 20, 37, 40, 35, 21, 7],
    "tokens": ["The", "European", "Commission", "said", "on", "Thursday", "it", "disagreed", "with", "German", "advice", "to", "consumers", "to", "shun", "British", "lamb", "until", "scientists", "determine", "whether", "mad", "cow", "disease", "can", "be", "transmitted", "to", "sheep", "."]
}

## Actual results
```python
print(dataset)

DatasetDict({
    train: Dataset({
        features: ['text'],
        num_rows: 219554
    })
    test: Dataset({
        features: ['text'],
        num_rows: 50350
    })
    validation: Dataset({
        features: ['text'],
        num_rows: 55044
    })
})
```

```python
for i in range(20):
    print(dataset['train'][i])

{'text': '-DOCSTART- -X- -X- O'}
{'text': ''}
{'text': 'EU NNP B-NP B-ORG'}
{'text': 'rejects VBZ B-VP O'}
{'text': 'German JJ B-NP B-MISC'}
{'text': 'call NN I-NP O'}
{'text': 'to TO B-VP O'}
{'text': 'boycott VB I-VP O'}
{'text': 'British JJ B-NP B-MISC'}
{'text': 'lamb NN I-NP O'}
{'text': '. . O O'}
{'text': ''}
{'text': 'Peter NNP B-NP B-PER'}
{'text': 'Blackburn NNP I-NP I-PER'}
{'text': ''}
{'text': 'BRUSSELS NNP B-NP B-LOC'}
{'text': '1996-08-22 CD I-NP O'}
{'text': ''}
{'text': 'The DT B-NP O'}
{'text': 'European NNP I-NP B-ORG'}
```
Fix yield for crd3

Modified the `_generate_examples` function to consider all the turns for a chunk id as a single example
Modified the features accordingly

```
"turns": [
                        {
                            "names": datasets.features.Sequence(datasets.Value("string")),
                            "utterances": datasets.features.Sequence(datasets.Value("string")),
                            "number": datasets.Value("int32"),
                        }
                    ],
                }
```

I wasn't able to run `datasets-cli dummy_data datasets` command. Is there a workaround for this?
How to load VERY LARGE dataset?

### System Info

```shell
I am using transformer trainer while meeting the issue.
The trainer requests torch.utils.data.Dataset as input, which loads the whole dataset into the memory at once. Therefore, when the dataset is too large to load, there's nothing I can do except using IterDataset, which loads samples of data seperately, and results in low efficiency. 
I wonder if there are any tricks like Sharding in huggingface trainer.
Looking forward to your reply.
```


### Who can help?

Trainer: @sgugger

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

None

### Expected behavior

```shell
I wonder if there are any tricks like fairseq Sharding very large datasets https://fairseq.readthedocs.io/en/latest/getting_started.html.
Thanks a lot!
```
Why the `conll2003` dataset on huggingface only contains the `en` subset? Where is the  German data?

![image](https://user-images.githubusercontent.com/37113676/165416606-96b5db18-b16c-4b6b-928c-de8620fd943e.png)

But on huggingface datasets:
![image](https://user-images.githubusercontent.com/37113676/165416649-8fd77980-ca0d-43f0-935e-f398ba8323a4.png)

Where is the  German data?
Avoid recursion error in map if example is returned as dict value

I noticed this bug while answering [this question](https://discuss.huggingface.co/t/correct-way-to-create-a-dataset-from-a-csv-file/15686/11?u=mariosasko). 

This code replicates the bug:
```python
from datasets import Dataset
dset = Dataset.from_dict({"en": ["aa", "bb"], "fr": ["cc", "dd"]})
dset.map(lambda ex: {"translation": ex})
```
and this is the fix for it (before this PR):
```python
from datasets import Dataset
dset = Dataset.from_dict({"en": ["aa", "bb"], "fr": ["cc", "dd"]})
dset.map(lambda ex: {"translation": dict(ex)})
```

Internally, this can be fixed by merging two dicts via dict unpacking (instead of `dict.update) `in `Dataset.map`, which avoids creating recursive dictionaries.

P.S. `{**a, **b}` is slightly more performant than `a.update(b)` in my bencmarks.
DatasetDict containing Datasets with different features when pushed to hub gets remapped features

Hi there,

I am trying to load a dataset to the Hub. This dataset is a `DatasetDict` composed of various splits. Some splits have a different `Feature` mapping. Locally, the DatasetDict preserves the individual features but if I `push_to_hub` and then `load_dataset`, the features are all the same.

Dataset and code to reproduce available [here](https://huggingface.co/datasets/pietrolesci/robust_nli).

In short:

I have 3 feature mapping
```python
Tri_features = Features(
    {
        "idx": Value(dtype="int64"),
        "premise": Value(dtype="string"),
        "hypothesis": Value(dtype="string"),
        "label": ClassLabel(num_classes=3, names=["entailment", "neutral", "contradiction"]),
    }
)

Ent_features = Features(
    {
        "idx": Value(dtype="int64"),
        "premise": Value(dtype="string"),
        "hypothesis": Value(dtype="string"),
        "label": ClassLabel(num_classes=2, names=["non-entailment", "entailment"]),
    }
)

Con_features = Features(
    {
        "idx": Value(dtype="int64"),
        "premise": Value(dtype="string"),
        "hypothesis": Value(dtype="string"),
        "label": ClassLabel(num_classes=2, names=["non-contradiction", "contradiction"]),
    }
)
```

Then I create different datasets

```python
dataset_splits = {}

for split in df["split"].unique():
    print(split)
    df_split = df.loc[df["split"] == split].copy()
    
    if split in Tri_dataset:
        df_split["label"] = df_split["label"].map({"entailment": 0, "neutral": 1, "contradiction": 2})
        ds = Dataset.from_pandas(df_split, features=Tri_features)
    
    elif split in Ent_bin_dataset:
        df_split["label"] = df_split["label"].map({"non-entailment": 0, "entailment": 1})
        ds = Dataset.from_pandas(df_split, features=Ent_features)
    
    elif split in Con_bin_dataset:
        df_split["label"] = df_split["label"].map({"non-contradiction": 0, "contradiction": 1})
        ds = Dataset.from_pandas(df_split, features=Con_features)

    else:
        print("ERROR:", split)
    dataset_splits[split] = ds
datasets = DatasetDict(dataset_splits)
```

I then push to hub

```python
datasets.push_to_hub("pietrolesci/robust_nli", token="<token>")
```

Finally, I load it from the hub

```python
datasets_loaded_from_hub = load_dataset("pietrolesci/robust_nli")
```

And I get that

```python
datasets["LI_TS"].features != datasets_loaded_from_hub["LI_TS"].features
```

since 

```python
"label": ClassLabel(num_classes=2, names=["non-contradiction", "contradiction"])
```

gets remapped to 

```python
 "label": ClassLabel(num_classes=3, names=["entailment", "neutral", "contradiction"])
```
Add CMU MoCap Dataset

Resolves #3457 

Dataset Request : Add CMU Graphics Lab Motion Capture dataset [#3457](https://github.com/huggingface/datasets/issues/3457)
This PR adds the CMU MoCap Dataset.

The authors didn't respond even after multiple follow ups, so I ended up crawling the website to get categories, subcategories and description information. Some of the subjects do not have category/subcategory/description as well. I am using a subject to categories, subcategories and description map (metadata file).

Currently the loading of the dataset works for "asf/amc" and "avi" formats since they have a single download link. But "c3d" and "mpg" have multiple download links (part archives) and dl_manager.download_and_extract() extracts the files to multiple paths, is there a way to extract these multiple archives into one folder ? Any other way to go about this ?
Any suggestions/inputs on this would be helpful. Thank you.
Add Recall Metric Card

What this PR mainly does:
- add metric card for recall metric
- update docs in recall python file

Note: I've also included a .json file with all of the metric card information. I've started compiling the relevant information in this type of .json files, and then using a script I wrote to generate the formatted metric card, as well as the docs to go in the .py file. I figured I'd upload the .json because it could be useful, especially if I also make a PR with the script I'm using (let me know if that's something you think would be beneficial!)
Add Precision Metric Card

What this PR mainly does:
- add metric card for precision metric
- update docs in precision python file

Note: I've also included a .json file with all of the metric card information. I've started compiling the relevant information in this type of .json files, and then using a script I wrote to generate the formatted metric card, as well as the docs to go in the .py file. I figured I'd upload the .json because it could be useful, especially if I also make a PR with the script I'm using (let me know if that's something you think would be beneficial!)