jamesLeeeeeee/datasets-github-issues huggingface dataset:
addition of py_ast
Author: reshinthadithyan | Pull request: yes | State: closed

The dataset consists of parsed Parsed ASTs that were used to train and evaluate the DeepSyn tool. 
The Python programs are collected from GitHub repositories
by removing duplicate files, removing project forks (copy of another existing repository)
,keeping only programs that parse and have at most 30'000 nodes in the AST and 
we aim to remove obfuscated files
Add NewsPH_NLI dataset
Author: anaerobeth | Pull request: yes | State: closed

This PR adds the NewsPH-NLI Dataset, the first benchmark dataset for sentence entailment in the low-resource Filipino language. Constructed through exploting the structure of news articles. Contains 600,000 premise-hypothesis pairs, in 70-15-15 split for training, validation, and testing.

Link to the paper: https://arxiv.org/pdf/2010.11574.pdf

Link to the dataset/repo: https://github.com/jcblaisecruz02/Filipino-Text-Benchmarks
Add Fake News Detection in Filipino dataset
Author: anaerobeth | Pull request: yes | State: closed

This PR adds the Fake News Filipino Dataset, a low-resource fake news detection corpora in Filipino. Contains 3,206 expertly-labeled news samples, half of which are real and half of which are fake.

Link to the paper: http://www.lrec-conf.org/proceedings/lrec2020/index.html

Link to the dataset/repo: https://github.com/jcblaisecruz02/Tagalog-fake-news
Add Dengue dataset in Filipino
Author: anaerobeth | Pull request: yes | State: closed

This PR adds the Dengue Dataset, a benchmark dataset for low-resource multiclass classification, with 4,015 training, 500 testing, and 500 validation examples, each labeled as part of five classes. Each sample can be a part of multiple classes. Collected as tweets.

Link to the paper: https://ieeexplore.ieee.org/document/8459963

Link to the dataset/repo: https://github.com/jcblaisecruz02/Filipino-Text-Benchmarks
Add Hate Speech Dataset in Filipino
Author: anaerobeth | Pull request: yes | State: closed

This PR adds the Hate Speech Dataset, a text classification dataset in Filipino, consisting 10k tweets (training set) that are labeled as hate speech or non-hate speech. Released with 4,232 validation and 4,232 testing samples. Collected during the 2016 Philippine Presidential Elections.

Link to the paper: https://pcj.csp.org.ph/index.php/pcj/issue/download/29/PCJ%20V14%20N1%20pp1-14%202019

Link to the dataset/repo: https://github.com/jcblaisecruz02/Filipino-Text-Benchmarks
Add Adversarial SQuAD dataset
Author: cceyda | Pull request: yes | State: closed

# Adversarial SQuAD

Adding the Adversarial [SQuAD](https://github.com/robinjia/adversarial-squad) dataset as part of the sprint 🎉 
This dataset adds adversarial sentences to a subset of the SQuAD dataset's dev examples. How to get the original squad example id is explained in readme->Data Instances. The whole data is intended for use in evaluation. (Which could of course be also used for training if one wants). So there is no classical train/val/test split, but a split based on the number of adversaries added.

There are 2 splits of this dataset:

- AddSent: Has up to five candidate adversarial sentences that don't answer the question, but have a lot of words in common with the question. This adversary is does not query the model in any way.
- AddOneSent: Similar to AddSent, but just one candidate sentences was picked at random. This adversary is does not query the model in any way.

(The AddAny and AddCommon datasets mentioned in the paper are dynamically generated based on model's output distribution thus are not included here)

The failing test look like some unrelated timeout thing, will probably clear if rerun.
- [x] All tests passed
- [x] Added dummy data
- [x] Added data card (as much as I could)
ADD COVID-QA dataset
Author: olinguyen | Pull request: yes | State: closed

This PR adds the COVID-QA dataset, a question answering dataset consisting of 2,019 question/answer pairs annotated by volunteer biomedical experts on scientific articles related to COVID-19

Link to the paper: https://openreview.net/forum?id=JENSKEEzsoU
Link to the dataset/repo: https://github.com/deepset-ai/COVID-QA
Add  KorQuAD v2 Dataset
Author: cceyda | Pull request: yes | State: closed

# The Korean Question Answering Dataset v2
Adding the [KorQuAD](https://korquad.github.io/) v2 dataset as part of the sprint 🎉 
This dataset is very similar to SQuAD and is an extension of [squad_kor_v1](https://github.com/huggingface/datasets/pull/1178) which is why I added it as `squad_kor_v2`. 

- Crowd generated questions and answer (1-answer per question) for Wikipedia articles. Differently from V1 it includes the html structure and markup, which makes it a different enough dataset. (doesn't share ids between v1 and v2 either)

- [x] All tests passed
- [x] Added dummy data
- [x] Added data card (as much as I could)

Edit: 🤦 looks like squad_kor_v1 commit sneaked in here too
Add  KorQuAD v1 Dataset
Author: cceyda | Pull request: yes | State: closed

# The Korean Question Answering Dataset
Adding the [KorQuAD](https://korquad.github.io/KorQuad%201.0/) v1 dataset as part of the sprint 🎉 
This dataset is very similar to SQuAD which is why I added it as `squad_kor_v1`. There is also a v2 which I added [here](https://github.com/huggingface/datasets/pull/1180).

- Crowd generated questions and answer (1-answer per question) for Wikipedia articles.

- [x] All tests passed
- [x] Added dummy data
- [x] Added data card (as much as I could)
Add Korean NER dataset
Author: jaketae | Pull request: yes | State: closed

This PR adds the [Korean named entity recognition dataset](https://github.com/kmounlp/NER). This dataset has been used in many downstream tasks, such as training [KoBERT](https://github.com/SKTBrain/KoBERT) for NER, as seen in this [KoBERT-CRF implementation](https://github.com/eagle705/pytorch-bert-crf-ner).
Add Naver sentiment movie corpus
Author: jaketae | Pull request: yes | State: closed

This PR adds the [Naver sentiment movie corpus](https://github.com/e9t/nsmc), a dataset containing Korean movie reviews from Naver, the most commonly used search engine in Korea. This dataset is often used to benchmark models on Korean NLP tasks, as seen in [this paper](https://www.aclweb.org/anthology/2020.lrec-1.199.pdf).
❓ On-the-fly tokenization with datasets, tokenizers, and torch Datasets and Dataloaders
Author: pietrolesci | Pull request: no | State: closed

Hi there,

I have a question regarding "on-the-fly" tokenization. This question was elicited by reading the "How to train a new language model from scratch using Transformers and Tokenizers" [here](https://huggingface.co/blog/how-to-train). Towards the end there is this sentence: "If your dataset is very large, you can opt to load and tokenize examples on the fly, rather than as a preprocessing step". I've tried coming up with a solution that would combine both `datasets` and `tokenizers`, but did not manage to find a good pattern.

I guess the solution would entail wrapping a dataset into a Pytorch dataset.

As a concrete example from the [docs](https://huggingface.co/transformers/custom_datasets.html)

```python
import torch

class SquadDataset(torch.utils.data.Dataset):
    def __init__(self, encodings):
        # instead of doing this beforehand, I'd like to do tokenization on the fly
        self.encodings = encodings 

    def __getitem__(self, idx):
        return {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}

    def __len__(self):
        return len(self.encodings.input_ids)

train_dataset = SquadDataset(train_encodings)
```

How would one implement this with "on-the-fly" tokenization exploiting the vectorized capabilities of tokenizers?


----

Edit: I have come up with this solution. It does what I want, but I feel it's not very elegant

```python
class CustomPytorchDataset(Dataset):
    def __init__(self):
        self.dataset = some_hf_dataset(...)
        self.tokenizer = BertTokenizerFast.from_pretrained("bert-base-uncased")

    def __getitem__(self, batch_idx):
        instance = self.dataset[text_col][batch_idx]
        tokenized_text = self.tokenizer(instance, truncation=True, padding=True)
        return tokenized_text

    def __len__(self):
        return len(self.dataset)

    @staticmethod
    def collate_fn(batch):
        # batch is a list, however it will always contain 1 item because we should not use the
        # batch_size argument as batch_size is controlled by the sampler
        return {k: torch.tensor(v) for k, v in batch[0].items()}

torch_ds = CustomPytorchDataset()

# NOTE: batch_sampler returns list of integers and since here we have SequentialSampler
# it returns: [1, 2, 3], [4, 5, 6], etc. - check calling `list(batch_sampler)`
batch_sampler = BatchSampler(SequentialSampler(torch_ds), batch_size=3, drop_last=True)

# NOTE: no `batch_size` as now the it is controlled by the sampler!
dl = DataLoader(dataset=torch_ds, sampler=batch_sampler, collate_fn=torch_ds.collate_fn)
```
adding TabFact dataset
Author: patil-suraj | Pull request: yes | State: closed

Adding TabFact: A Large-scale Dataset for Table-based Fact Verification.

https://github.com/wenhuchen/Table-Fact-Checking

- The tables are stored as individual csv files, so need to download 16,573 🤯  csv files. As a result the `datasets_infos.json` file is huge (6.62 MB).
- Original dataset has nested structure where, where table is one example and each table has multiple statements,
flattening the structure here so that each statement is one example.
Add JFLEG
Author: j-chim | Pull request: yes | State: closed

This PR adds [JFLEG ](https://www.aclweb.org/anthology/E17-2037/), an English grammatical error correction benchmark. 

The tests were successful on real data, although it would be great if I can get some guidance on the **dummy data**. Basically, **for each source sentence there are 4 possible gold standard target sentences**. The original dataset comprise files in a flat structure, labelled by split then by source/target (e.g., dev.src, dev.ref0, ..., dev.ref3). Not sure what is the best way of adding this.

I imagine I can treat each distinct source-target pair as its own split? But having so many copies of the source sentence feels redundant, and it would make it less convenient to end-users who might want to access multiple gold standard targets simultaneously.
Add the Winograd Schema Challenge
Author: joeddav | Pull request: yes | State: closed

Adds the Winograd Schema Challenge, including configs for the more canonical wsc273 as well as wsc285 with 12 new examples.

- https://cs.nyu.edu/faculty/davise/papers/WinogradSchemas/WS.html

The data format was a bit of a nightmare but I think I got it to a workable format.
Add ReFreSD dataset
Author: mpariente | Pull request: yes | State: closed

This PR adds the **ReFreSD dataset**. 
The original data is hosted [on this github repo](https://github.com/Elbria/xling-SemDiv) and we use the `REFreSD_rationale` to expose all the data. 


Need feedback on:
- I couldn't generate the dummy data. The file we download is a tsv file, but without extension, I suppose this is the problem. I'm sure there is a simple trick to make this work. 
- The feature names. 
  - I don't know if it's better to stick to the classic `sentence1`, `sentence2` or to `sentence_en`, `sentence_fr` to be more explicit. 
  - There is a binary label (called `label`, no problem here), and a 3-class label called `#3_labels` in the original tsv. I changed it to `all_labels` but I'm sure there is better. 
- The rationales are lists of integers, extracted as a string at first. I wonder what's the best way to treat them, any idea? Also, I couldn't manage to make a `Sequence` of `int8` but I'm sure I've missed something simple. 

Thanks in advance
add wmt mlqe 2020 shared task
Author: VictorSanh | Pull request: yes | State: closed

First commit for Shared task 1 (wmt_mlqw_task1) of WMT20 MLQE (quality estimation of machine translation)
Note that I copied the tags in the README for only one (of the 7 configurations): `en-de`.
There is one configuration for each pair of languages.
Adding initial version of cord-19 dataset
Author: ggdupont | Pull request: yes | State: closed

Initial version only reading the metadata in CSV.

### Checklist:
- [x] Create the dataset script /datasets/my_dataset/my_dataset.py using the template
- [x] Fill the _DESCRIPTION and _CITATION variables
- [x] Implement _infos(), _split_generators() and _generate_examples()
- [x] Make sure that the BUILDER_CONFIGS class attribute is filled with the different configurations of the dataset and that the BUILDER_CONFIG_CLASS is specified if there is a custom config class.
- [x] Generate the metadata file dataset_infos.json for all configurations
- [x] Generate the dummy data dummy_data.zip files to have the dataset script tested and that they don't weigh too much (<50KB)
- [x] Add the dataset card README.md using the template and at least fill the tags
- [x] Both tests for the real data and the dummy data pass.

### TODO:
- [x] add more metadata
- [x] add full text
- [x] add pre-computed document embedding
Incorrect URL for MRQA SQuAD train subset
Author: yuxiang-wu | Pull request: no | State: closed

https://github.com/huggingface/datasets/blob/4ef4c8f8b7a60e35c6fa21115fca9faae91c9f74/datasets/mrqa/mrqa.py#L53

The URL for `train+SQuAD` subset of MRQA points to the dev set instead of train set. It should be `https://s3.us-east-2.amazonaws.com/mrqa/release/v2/train/SQuAD.jsonl.gz`.
Initial version of cord-19 dataset from AllenAI with only the abstract
Author: ggdupont | Pull request: yes | State: closed

Initial version only reading the metadata in CSV.

### Checklist:
- [x] Create the dataset script /datasets/my_dataset/my_dataset.py using the template
- [x] Fill the _DESCRIPTION and _CITATION variables
- [x] Implement _infos(), _split_generators() and _generate_examples()
- [x] Make sure that the BUILDER_CONFIGS class attribute is filled with the different configurations of the dataset and that the BUILDER_CONFIG_CLASS is specified if there is a custom config class.
- [x] Generate the metadata file dataset_infos.json for all configurations
- [x] Generate the dummy data dummy_data.zip files to have the dataset script tested and that they don't weigh too much (<50KB)
- [x] Add the dataset card README.md using the template and at least fill the tags
- [ ] Both tests for the real data and the dummy data pass.

### TODO:
- [ ] add more metadata
- [ ] add full text
- [ ] add pre-computed document embedding
Using a feature named "_type" fails with certain operations
Author: dcfidalgo | Pull request: no | State: closed

A column named `_type` leads to a `TypeError: unhashable type: 'dict'` for certain operations:
```python
from datasets import Dataset, concatenate_datasets

ds = Dataset.from_dict({"_type": ["whatever"]}).map()
concatenate_datasets([ds])
# or simply
Dataset(ds._data)
```
Context: We are using datasets to persist data coming from elasticsearch to feed to our pipeline, and elasticsearch has a `_type` field, hence the strange name of the column.

Not sure if you wish to support this specific column name, but if you do i would be happy to try a fix and provide a PR. I already had a look into it and i think the culprit is the `datasets.features.generate_from_dict` function. It uses the hard coded `_type` string to figure out if it reached the end of the nested feature object from a serialized dict.

Best wishes and keep up the awesome work!
Add arsentd_lev dataset
Author: moussaKam | Pull request: yes | State: closed

Add The Arabic Sentiment Twitter Dataset for Levantine dialect (ArSenTD-LEV)

Paper: [ArSentD-LEV: A Multi-Topic Corpus for Target-based Sentiment Analysis in Arabic Levantine Tweets](https://arxiv.org/abs/1906.01830)
Homepage: http://oma-project.com/
add thaisum
Author: cstorm125 | Pull request: yes | State: closed

ThaiSum, a large-scale corpus for Thai text summarization obtained from several online news websites namely Thairath, ThaiPBS, Prachathai, and The Standard. This dataset consists of over 350,000 article and summary pairs written by journalists. We evaluate the performance of various existing summarization models on ThaiSum dataset and analyse the characteristic of the dataset to present its difficulties.
adding cdt dataset
Author: abecadel | Pull request: yes | State: closed

- **Name:** *Cyberbullying Detection Task*
- **Description:** *The Cyberbullying Detection task was part of 2019 edition of PolEval competition. The goal is to predict if a given Twitter message contains a cyberbullying (harmful) content.*
- **Data:** *https://github.com/ptaszynski/cyberbullying-Polish*
- **Motivation:** *The KLEJ benchmark (Kompleksowa Lista Ewaluacji Językowych) is a set of nine evaluation tasks for the Polish language understanding.*
adding cdsc dataset
Author: abecadel | Pull request: yes | State: closed

- **Name**: *cdsc (domains: cdsc-e & cdsc-r)*
- **Description**: *Polish CDSCorpus consists of 10K Polish sentence pairs which are human-annotated for semantic relatedness and entailment. The dataset may be used for the evaluation of compositional distributional semantics models of Polish. The dataset was presented at ACL 2017. Please refer to the Wróblewska and Krasnowska-Kieraś (2017) for a detailed description of the resource.*
- **Data**: *http://2019.poleval.pl/index.php/tasks/*
- **Motivation**: *The KLEJ benchmark (Kompleksowa Lista Ewaluacji Językowych) is a set of nine evaluation tasks for the Polish language understanding.*
Add WikiANN NER dataset
Author: lewtun | Pull request: yes | State: closed

This PR adds the full set of 176 languages from the balanced train/dev/test splits of WikiANN / PAN-X from: https://github.com/afshinrahimi/mmner

Until now, only 40 of these languages were available in `datasets` as part of the XTREME benchmark

Courtesy of the dataset author, we can now download this dataset from a Dropbox URL without needing a manual download anymore 🥳, so at some point it would be worth updating the PAN-X subset of XTREME as well 😄 

Link to gist with some snippets for producing dummy data: https://gist.github.com/lewtun/5b93294ab6dbcf59d1493dbe2cfd6bb9

P.S. @yjernite I think I was confused about needing to generate a set of YAML tags per config, so ended up just adding a single one in the README.
nkjp-ner
Author: abecadel | Pull request: yes | State: closed

- **Name:** *nkjp-ner*
- **Description:** *The NKJP-NER is based on a human-annotated part of NKJP. We extracted sentences with named entities of exactly one type. The task is to predict the type of the named entity.*
- **Data:** *https://klejbenchmark.com/tasks/*
- **Motivation:** *The KLEJ benchmark (Kompleksowa Lista Ewaluacji Językowych) is a set of nine evaluation tasks for the Polish language understanding.*
Not support links with 302 redirect 
Author: chameleonTK | Pull request: no | State: closed

I have an issue adding this download link https://github.com/jitkapat/thailitcorpus/releases/download/v.2.0/tlc_v.2.0.tar.gz

it might be because it is not a direct link (it returns 302 and redirects to aws that returns 403 for head requests). 

```
r.head("https://github.com/jitkapat/thailitcorpus/releases/download/v.2.0/tlc_v.2.0.tar.gz", allow_redirects=True)                                      
# <Response [403]>
```
Add KorNLU dataset
Author: sumanthd17 | Pull request: yes | State: closed

Added Korean NLU datasets. The link to the dataset can be found [here](https://github.com/kakaobrain/KorNLUDatasets) and the paper can be found [here](https://arxiv.org/abs/2004.03289)

**Note**: The MNLI tsv file is broken, so this code currently excludes the file. Please suggest other alternative if any @lhoestq 

- [x] Followed the instructions in CONTRIBUTING.md
- [x] Ran the tests successfully
- [x] Created the dummy data
Fix squad V2 metric script
Author: sgugger | Pull request: yes | State: closed

The current squad v2 metric doesn't work with the squad (v1 or v2) datasets. The script is copied from `squad_evaluate` in transformers that requires the labels (with multiple answers) to be like this:
```
references = [{'id': 'a', 'answers': [
    {'text': 'Denver Broncos', 'answer_start': 177},
    {'text': 'Denver Broncos', 'answer_start': 177}
]}]
```
while the dataset had references like this:
```
references = [{'id': 'a', 'answers': 
    {'text': ['Denver Broncos' 'Denver Broncos'], 'answer_start': [177, 177]}
}]
```

Using one or the other format fails with the current squad v2 metric:
```
from datasets import load_metric
metric = load_metric("squad_v2")
predictions = [{'id': 'a', 'prediction_text': 'Denver Broncos', 'no_answer_probability': 0.0}]
references = [{'id': 'a', 'answers': [
    {'text': 'Denver Broncos', 'answer_start': 177},
    {'text': 'Denver Broncos', 'answer_start': 177}
]}]
metric.compute(predictions=predictions, references=references)
```
fails as well as
```
from datasets import load_metric
metric = load_metric("squad_v2")
predictions = [{'id': 'a', 'prediction_text': 'Denver Broncos', 'no_answer_probability': 0.0}]
references = [{'id': 'a', 'answers': 
    {'text': ['Denver Broncos' 'Denver Broncos'], 'answer_start': [177, 177]}
}]
metric.compute(predictions=predictions, references=references)
```

This is because arrow reformats the references behind the scene.

With this PR (tested locally), both the snippets up there work and return proper results.
Fix dataset URL and file names, and add column name in "Social Bias Frames" dataset
Author: otakumesi | Pull request: yes | State: closed

# Why I did
When I use "social_bias_frames" datasets in this library, I got 404 Errors.
So, I fixed this error and another some problems that I faced to use the dataset.

# What I did
* Modify this dataset URL
* Modify this dataset file names
* Add a "dataSource" column

Thank you!
Add GoEmotions
Author: joeddav | Pull request: yes | State: closed

Adds the GoEmotions dataset, a nice emotion classification dataset with 27 (multi-)label annotations on reddit comments. Includes both a large raw version and a narrowed version with predefined train/test/val splits, which I've included as separate configs with the latter as a default.

- Webpage/repo: https://github.com/google-research/google-research/tree/master/goemotions
- Paper: https://arxiv.org/abs/2005.00547
Adding NCHLT dataset
Author: Narsil | Pull request: yes | State: closed

https://repo.sadilar.org/handle/20.500.12185/7/discover?filtertype_0=database&filtertype_1=title&filter_relational_operator_1=contains&filter_relational_operator_0=equals&filter_1=&filter_0=Monolingual+Text+Corpora%3A+Annotated&filtertype=project&filter_relational_operator=equals&filter=NCHLT+Text+II
Add KorNLU
Author: sumanthd17 | Pull request: yes | State: closed

Added Korean NLU datasets. The link to the dataset can be found [here](https://github.com/kakaobrain/KorNLUDatasets) and the paper can be found [here](https://arxiv.org/abs/2004.03289)

**Note**: The MNLI tsv file is broken, so this code currently excludes the file. Please suggest other alternative if any @lhoestq 

- [x] Followed the instructions in CONTRIBUTING.md
- [x] Ran the tests successfully
- [x] Created the dummy data