jamesLeeeeeee/datasets-github-issues huggingface dataset:
Add KLUE dataset
Author: jungwhank | Pull request: yes | State: closed

Add `KLUE (Korean Language Understanding Evaluation)` dataset released recently from [paper](https://arxiv.org/abs/2105.09680), [github](https://github.com/KLUE-benchmark/KLUE) and [webpage](https://klue-benchmark.com/tasks).
Please let me know if there's anything missing in the code or README.
Thanks!
Cached dataset not loaded
Author: borisdayma | Pull request: no | State: closed

## Describe the bug
I have a large dataset (common_voice, english) where I use several map and filter functions.
Sometimes my cached datasets after specific functions are not loaded.
I always use the same arguments, same functions, no seed…

## Steps to reproduce the bug
```python
def filter_by_duration(batch):
    return (
        batch["duration"] <= 10
        and batch["duration"] >= 1
        and len(batch["target_text"]) > 5
    )

def prepare_dataset(batch):
    batch["input_values"] = processor(
        batch["speech"], sampling_rate=batch["sampling_rate"][0]
    ).input_values
    with processor.as_target_processor():
        batch["labels"] = processor(batch["target_text"]).input_ids
    return batch

train_dataset = train_dataset.filter(
    filter_by_duration,
    remove_columns=["duration"],
    num_proc=data_args.preprocessing_num_workers,
)

# PROBLEM HERE -> below function is reexecuted and cache is not loaded
train_dataset = train_dataset.map(
    prepare_dataset,
    remove_columns=train_dataset.column_names,
    batch_size=training_args.per_device_train_batch_size,
    batched=True,
    num_proc=data_args.preprocessing_num_workers,
)

# Later in script
set_caching_enabled(False)
# apply map on trained model to eval/test sets

```

## Expected results
The cached dataset should always be reloaded.

## Actual results
The function is reexecuted.

I have access to cached files `cache-xxxxx.arrow`.
Is there a way I can somehow load manually 2 versions and see how the hash was created for debug purposes (to know if it's an issue with dataset or function)?

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.6.2
- Platform: Linux-5.8.0-45-generic-x86_64-with-glibc2.29
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.1+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
AttributeError: 'DatasetInfo' object has no attribute 'task_templates'
Author: jungwhank | Pull request: no | State: closed

## Describe the bug
Hello, 
I'm trying to add dataset and contribute, but test keep fail with below cli.
` RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_<my_dataset>`

## Steps to reproduce the bug
It seems like a bug when I see an error with the existing dataset, not the dataset I'm trying to add.

` RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_<any_dataset>`


## Expected results
All test passed

## Actual results
```
                # check that dataset is not empty
                self.parent.assertListEqual(sorted(dataset_builder.info.splits.keys()), sorted(dataset))
                for split in dataset_builder.info.splits.keys():
                    # check that loaded datset is not empty
                    self.parent.assertTrue(len(dataset[split]) > 0)
    
                # check that we can cast features for each task template
>               task_templates = dataset_builder.info.task_templates
E               AttributeError: 'DatasetInfo' object has no attribute 'task_templates'

tests/test_dataset_common.py:175: AttributeError
```


## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.6.2
- Platform: Darwin-20.4.0-x86_64-i386-64bit
- Python version: 3.7.7
- PyTorch version (GPU?): 1.7.0 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No
Docstring mistake: dataset vs. metric
Author: PhilipMay | Pull request: no | State: closed

This:

https://github.com/huggingface/datasets/blob/d95b95f8cf3cb0cff5f77a675139b584dcfcf719/src/datasets/load.py#L582

Should better be something like:

`a metric identifier on HuggingFace AWS bucket (list all available metrics and ids with ``datasets.list_metrics()``)`

I can provide a PR l8er...
.map() function got an unexpected keyword argument 'cache_file_name'
Author: cindyxinyiwang | Pull request: no | State: closed

## Describe the bug

I'm trying to save the result of datasets.map() to a specific file, so that I can easily share it among multiple computers without reprocessing the dataset. However, when I try to pass an argument 'cache_file_name' to the .map() function, it throws an error that ".map() function got an unexpected keyword argument 'cache_file_name'". 

I believe I'm using the latest dataset 1.6.2. Also seems like the document and the actual code indicates there is an argument 'cache_file_name' for the .map() function.

Here is the code I use
## Steps to reproduce the bug
```datasets = load_from_disk(dataset_path=my_path)

[...]

def tokenize_function(examples):
    return tokenizer(examples[text_column_name])

logger.info("Mapping dataset to tokenized dataset.")
tokenized_datasets = datasets.map(
    tokenize_function,
    batched=True,
    num_proc=preprocessing_num_workers,
    remove_columns=column_names,
    load_from_cache_file=True,
   cache_file_name="my_tokenized_file"
)
```

## Actual results
    tokenized_datasets = datasets.map(
TypeError: map() got an unexpected keyword argument 'cache_file_name'

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:1.6.2
- Platform:Linux-4.18.0-193.28.1.el8_2.x86_64-x86_64-with-glibc2.10
- Python version:3.8.5
- PyArrow version:3.0.0
Paperswithcode dataset mapping
Author: julien-c | Pull request: yes | State: closed

This is a continuation of https://github.com/huggingface/huggingface_hub/pull/43, encoded directly inside dataset cards.

As discussed:
- `paperswithcode_id: null` when the dataset doesn't exist on paperswithcode's side.
- I've added this new key at the end of the yaml instead of ordering all keys alphabetically as pyyaml's default. No strong opinion on that one though
PermissionError on Windows when using temp dir for caching
Author: mariosasko | Pull request: no | State: closed

Currently, the following code raises a PermissionError on master if working on Windows:

```python
# run as a script or call exit() in REPL to initiate the temp dir cleanup
from datasets import *
d = load_dataset("sst", split="train", keep_in_memory=False)
set_caching_enabled(False)
d.map(lambda ex: ex)
```

Error stack trace:
```
Traceback (most recent call last):
  File "C:\Users\Mario\Anaconda3\envs\hf-datasets\lib\weakref.py", line 624, in _exitfunc
    f()
  File "C:\Users\Mario\Anaconda3\envs\hf-datasets\lib\weakref.py", line 548, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "C:\Users\Mario\Anaconda3\envs\hf-datasets\lib\tempfile.py", line 799, in _cleanup
    _shutil.rmtree(name)
  File "C:\Users\Mario\Anaconda3\envs\hf-datasets\lib\shutil.py", line 500, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Users\Mario\Anaconda3\envs\hf-datasets\lib\shutil.py", line 395, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Users\Mario\Anaconda3\envs\hf-datasets\lib\shutil.py", line 393, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\Mario\\AppData\\Local\\Temp\\tmp20epyhmq\\cache-87a87ffb5a956e68.arrow'
```
Concatenate several datasets with removed columns is not working.
Author: philschmid | Pull request: no | State: closed

## Describe the bug

You can't concatenate datasets when you removed columns before.

## Steps to reproduce the bug
```python
from datasets import load_dataset, concatenate_datasets

wikiann= load_dataset("wikiann","en")

wikiann["train"] = wikiann["train"].remove_columns(["langs","spans"])
wikiann["test"] = wikiann["test"].remove_columns(["langs","spans"])

assert wikiann["train"].features.type == wikiann["test"].features.type

concate = concatenate_datasets([wikiann["train"],wikiann["test"]])
```

## Expected results
Merged dataset 


## Actual results
```python
ValueError: External features info don't match the dataset:
Got
{'tokens': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None), 'ner_tags': Sequence(feature=ClassLabel(num_classes=7, names=['O', 'B-PER', 'I-PER', 'B-ORG', 'I-ORG', 'B-LOC', 'I-LOC'], names_file=None, id=None), length=-1, id=None), 'langs': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None), 'spans': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None)}
with type
struct<langs: list<item: string>, ner_tags: list<item: int64>, spans: list<item: string>, tokens: list<item: string>>

but expected something like
{'ner_tags': Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), 'tokens': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None)}
with type
struct<ner_tags: list<item: int64>, tokens: list<item: string>>
```
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: ~1.6.2~ 1.5.0
- Platform: macos
- Python version: 3.8.5
- PyArrow version: 3.0.0
News_commentary Dataset Translation Pairs are of Incorrect Language Specified Pairs
Author: anassalamah | Pull request: no | State: closed

I used load_dataset to load the news_commentary dataset for "ar-en" translation pairs but found translations from Arabic to Hindi.  

```
train_ds = load_dataset("news_commentary", "ar-en", split='train[:98%]')
val_ds = load_dataset("news_commentary", "ar-en", split='train[98%:]')

# filtering out examples that are not ar-en translations but ar-hi
val_ds = val_ds.filter(lambda example, indice: indice not in chain(range(1312,1327) ,range(1384,1399), range(1030,1042)), with_indices=True)
```

* I'm fairly new to using datasets so I might be doing something wrong
strange datasets from OSCAR corpus
Author: jerryIsHere | Pull request: no | State: open

![image](https://user-images.githubusercontent.com/50871412/119260850-4f876b80-bc07-11eb-8894-124302600643.png)
![image](https://user-images.githubusercontent.com/50871412/119260875-675eef80-bc07-11eb-9da4-ee27567054ac.png)
From the [official site ](https://oscar-corpus.com/), the Yue Chinese dataset should have 2.2KB data.
7 training instances is obviously not a right number.
As I can read Yue Chinese, I call tell the last instance is definitely not something that would appear on Common Crawl.
And even if you don't read Yue Chinese, you can tell the first six instance are problematic.
(It is embarrassing, as the 7 training instances look exactly like something from a pornographic novel or flitting messages in a chat of a dating app)
It might not be the problem of the huggingface/datasets implementation, because when I tried to download the dataset from the official site, I found out that the zip file is corrupted.
I will try to inform the host of OSCAR corpus later.
Awy a remake about this dataset in huggingface/datasets is needed, perhaps after the host of the dataset fixes the issue.

> Hi @jerryIsHere , sorry for the late response! Sadly this is normal, the problem comes form fasttext's classifier which we used to create the original corpus. In general the classifier is not really capable of properly recognizing Yue Chineese so the file ends un being just noise from Common Crawl. Some of these problems with OSCAR were already discussed [here](https://arxiv.org/pdf/2103.12028.pdf) but we are working on explicitly documenting the problems by language on our website. In fact, could please you open an issue on [our repo](https://github.com/oscar-corpus/oscar-website/issues) as well so that we can track it?

Thanks a lot, the new post is here:
https://github.com/oscar-corpus/oscar-website/issues/11
`pretty_name` for dataset in YAML tags
Author: bhavitvyamalik | Pull request: yes | State: closed

I'm updating `pretty_name` for datasets in YAML tags as discussed with @lhoestq. Here are the first 10, please let me know if they're looking good.

If dataset has 1 config, I've added `pretty_name` as `config_name: full_name_of_dataset` as config names were `plain_text`, `default`, `squad` etc (not so important in this case) whereas when dataset has >1 configs, I've added `config_name: full_name_of_dataset+config_name` so as to let user know about the `config` here.
Missing original answers in kilt-TriviaQA
Author: PaulLerner | Pull request: no | State: closed

I previously opened an issue at https://github.com/facebookresearch/KILT/issues/42 but from the answer of @fabiopetroni it seems that the problem comes from HF-datasets

## Describe the bug
The `answer` field in kilt-TriviaQA, e.g. `kilt_tasks['train_triviaqa'][0]['output']['answer']` contains a list of alternative answer which are accepted for the question.  
However it'd be nice to know the original answer to the question (the only fields in `output` are `'answer', 'meta', 'provenance'`)

## How to fix
It can be fixed by retrieving the original answer from the original TriviaQA (e.g. `trivia_qa['train'][0]['answer']['value']`), perhaps at the same place as here where one retrieves the questions https://github.com/huggingface/datasets/blob/master/datasets/kilt_tasks/README.md#loading-the-kilt-knowledge-source-and-task-data

cc @yjernite who previously answered to an issue about KILT and TriviaQA :)
datasets 1.6 ignores cache
Author: stas00 | Pull request: no | State: closed

Moving from https://github.com/huggingface/transformers/issues/11801#issuecomment-845546612 

Quoting @VictorSanh:

> 
> I downgraded datasets to `1.5.0` and printed `tokenized_datasets.cache_files` (L335):
> 
> > `{'train': [{'filename': '/home/victor/.cache/huggingface/datasets/openwebtext10k/plain_text/1.0.0/3a8df094c671b4cb63ed0b41f40fb3bd855e9ce2e3765e5df50abcdfb5ec144b/cache-c6aefe81ca4e5152.arrow'}], 'validation': [{'filename': '/home/victor/.cache/huggingface/datasets/openwebtext10k/plain_text/1.0.0/3a8df094c671b4cb63ed0b41f40fb3bd855e9ce2e3765e5df50abcdfb5ec144b/cache-97cf4c813e6469c6.arrow'}]}`
> 
> while the same command with the latest version of datasets (actually starting at `1.6.0`) gives:
> > `{'train': [], 'validation': []}`
> 

I also confirm that downgrading to `datasets==1.5.0` makes things fast again - i.e. cache is used.

to reproduce:
```
USE_TF=0 python  examples/pytorch/language-modeling/run_clm.py \
    --model_name_or_path gpt2 \
    --dataset_name "stas/openwebtext-10k" \
    --output_dir output_dir \
    --overwrite_output_dir \
    --do_train \
    --do_eval \
    --max_train_samples 1000 \
    --max_eval_samples 200 \
    --per_device_train_batch_size 4 \
    --per_device_eval_batch_size 4 \
    --num_train_epochs 1 \
    --warmup_steps 8 \
    --block_size 64 \
    --fp16 \
    --report_to none
```

the first time the startup is slow and some 5 tqdm bars. It shouldn't do it on consequent runs. but with `datasets>1.5.0` it rebuilds on every run.

@lhoestq
Accessing Arrow dataset cache_files
Author: Mehrad0711 | Pull request: no | State: closed

## Describe the bug
In datasets 1.5.0 the following code snippet would have printed the cache_files:

```
train_data = load_dataset('conll2003', split='train', cache_dir='data')
print(train_data.cache_files[0]['filename'])

```

However, in the newest release (1.6.1), it prints an empty list.

I also tried loading the dataset with `keep_in_memory=True` argument but still `cache_files` is empty.

Was wondering if this is a bug or I need to pass additional arguments so I can access the cache_files.
DuplicatedKeysError: FAILURE TO GENERATE DATASET ! load_dataset('head_qa', 'en')
Author: helloworld123-lab | Pull request: no | State: closed

Hello everyone,

I try to use head_qa dataset in [https://huggingface.co/datasets/viewer/?dataset=head_qa&config=en](url)

```
!pip install datasets
from datasets import load_dataset
dataset = load_dataset(
   'head_qa', 'en')
```
When I write above load_dataset(.), it throws the following:

```
DuplicatedKeysError                       Traceback (most recent call last)

<ipython-input-6-ea87002d32f0> in <module>()
      2 from datasets import load_dataset
      3 dataset = load_dataset(
----> 4    'head_qa', 'en')

5 frames

/usr/local/lib/python3.7/dist-packages/datasets/arrow_writer.py in check_duplicate_keys(self)
    347         for hash, key in self.hkey_record:
    348             if hash in tmp_record:
--> 349                 raise DuplicatedKeysError(key)
    350             else:
    351                 tmp_record.add(hash)

DuplicatedKeysError: FAILURE TO GENERATE DATASET !
Found duplicate Key: 1
Keys should be unique and deterministic in nature
```
How can I fix the error? Thanks
maintain YAML structure reading from README
Author: bhavitvyamalik | Pull request: yes | State: closed

How YAML used be loaded earlier in the string (structure of YAML was affected because of this and YAML for datasets with multiple configs was not being loaded correctly):
```
annotations_creators:
labeled_final:
- expert-generated
labeled_swap:
- expert-generated
unlabeled_final:
- machine-generated
language_creators:
- machine-generated
languages:
- en
licenses:
- other
multilinguality:
- monolingual
size_categories:
labeled_final:
- 10K<n<100K
labeled_swap:
- 10K<n<100K
unlabeled_final:
- 100K<n<1M
source_datasets:
- original
task_categories:
- text-classification
- text-scoring
task_ids:
- semantic-similarity-classification
- semantic-similarity-scoring
- text-scoring-other-paraphrase-identification
```
How YAML is loaded in string now:
```
annotations_creators:
  labeled_final:
  - expert-generated
  labeled_swap:
  - expert-generated
  unlabeled_final:
  - machine-generated
language_creators:
- machine-generated
languages:
- en
licenses:
- other
multilinguality:
- monolingual
size_categories:
  labeled_final:
  - 10K<n<100K
  labeled_swap:
  - 10K<n<100K
  unlabeled_final:
  - 100K<n<1M
source_datasets:
- original
task_categories:
- text-classification
- text-scoring
task_ids:
- semantic-similarity-classification
- semantic-similarity-scoring
- text-scoring-other-paraphrase-identification
```
Add missing dataset_infos.json files
Author: lewtun | Pull request: no | State: open

Some of the datasets in `datasets` are missing a `dataset_infos.json` file, e.g.

```
[PosixPath('datasets/chr_en/chr_en.py'), PosixPath('datasets/chr_en/README.md')]
[PosixPath('datasets/telugu_books/README.md'), PosixPath('datasets/telugu_books/telugu_books.py')]
[PosixPath('datasets/reclor/README.md'), PosixPath('datasets/reclor/reclor.py')]
[PosixPath('datasets/json/README.md')]
[PosixPath('datasets/csv/README.md')]
[PosixPath('datasets/wikihow/wikihow.py'), PosixPath('datasets/wikihow/README.md')]
[PosixPath('datasets/c4/c4.py'), PosixPath('datasets/c4/README.md')]
[PosixPath('datasets/text/README.md')]
[PosixPath('datasets/lm1b/README.md'), PosixPath('datasets/lm1b/lm1b.py')]
[PosixPath('datasets/pandas/README.md')]
```

For `json`, `text`, csv`, and `pandas` this is expected, but not for the others which should be fixed
ArrowDataset.save_to_disk produces files that cannot be read using pyarrow.feather
Author: Ark-kun | Pull request: no | State: open

## Describe the bug
A clear and concise description of what the bug is.

## Steps to reproduce the bug
```python
from datasets import load_dataset
from pyarrow import feather

dataset = load_dataset('imdb', split='train')
dataset.save_to_disk('dataset_dir')
table = feather.read_table('dataset_dir/dataset.arrow')
```

## Expected results
I expect that the saved dataset can be read by the official Apache Arrow methods.

## Actual results
```
  File "/usr/local/lib/python3.7/site-packages/pyarrow/feather.py", line 236, in read_table
    reader.open(source, use_memory_map=memory_map)
  File "pyarrow/feather.pxi", line 67, in pyarrow.lib.FeatherReader.open
  File "pyarrow/error.pxi", line 123, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 85, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Not a Feather V1 or Arrow IPC file
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: datasets-1.6.2
- Platform: Linux
- Python version: 3.7
- PyArrow version: 0.17.1, also 2.0.0
Improve task api code quality
Author: mariosasko | Pull request: yes | State: closed

Improves the code quality of the `TaskTemplate` dataclasses.

Changes:
* replaces `return NotImplemented` with raise `NotImplementedError` 
* replaces `sorted` with `len` in the uniqueness check 
* defines `label2id` and `id2label` in the `TextClassification` template as properties
* replaces the `object.__setattr__(self, attr, value)` syntax with (IMO nicer) `self.__dict__[attr] = value`
Loading dataset from local path
Author: kolakows | Pull request: no | State: closed

I'm trying to load a local dataset with the code below

```
ds = datasets.load_dataset('my_script.py', 
                           data_files='corpus.txt', 
                           data_dir='/data/dir', 
                           cache_dir='.')
```
But internally a BuilderConfig is created, which tries to use getmtime on the data_files string, without using data_dir. Is this a bug or am I not using the load_dataset correctly?

https://github.com/huggingface/datasets/blob/bc61954083f74e6460688202e9f77dde2475319c/src/datasets/builder.py#L153
ConvQuestions benchmark added
Author: PhilippChr | Pull request: yes | State: closed

Hello,
I would like to integrate our dataset on conversational QA. The answers are grounded in the KG.
The work was published in CIKM 2019 (https://dl.acm.org/doi/10.1145/3357384.3358016).
We hope for further research on how to deal with the challenges of factoid conversational QA.
Thanks! :)