noamaanMulla-03/datasets-issues huggingface dataset:
added TTC4900: A Benchmark Data for Turkish Text Categorization dataset

This PR adds the TTC4900 dataset which is a Turkish Text Categorization dataset by me and @basakbuluz. 

Homepage: [https://www.kaggle.com/savasy/ttc4900](https://www.kaggle.com/savasy/ttc4900)
Point of Contact: [Savaş Yıldırım](mailto:savasy@gmail.com) / @savasy



@lhoestq, can you help with creating dummy_data?


Hi @yavuzKomecoglu did you manage to build the dummy data ?

> Hi @yavuzKomecoglu did you manage to build the dummy data ?

Hi, sorry for the return. I've created dummy_data.zip manually.

> Nice thank you !
> 
> Before we merge can you fill the two sections of the dataset card I suggested ?
> And also remove one remaining print statement

I updated your suggestions. Thank you very much for your support.

I think you accidentally pushed the readme of another dataset (name_to_nation).
I removed it so you have to `git pull`

Because of that I guess your changes about the ttc4900 was not included.
Feel free to ping me once they're added




> I think you accidentally pushed the readme of another dataset (name_to_nation).
> I removed it so you have to `git pull`
> 
> Because of that I guess your changes about the ttc4900 was not included.
> Feel free to ping me once they're added

I did `git pull` and updated readme **ttc4900**.

merging since the Ci is fixed on master
Added Wiki Summary Dataset

Wiki Summary: Dataset extracted from Persian Wikipedia into the form of articles and highlights.
Link: https://github.com/m3hrdadfi/wiki-summary


@lhoestq why my tests are not running?

Maybe an issue with CircleCI, let me try to make them run

The CI error `tests/test_file_utils.py::TempSeedTest::test_tensorflow` is not related to this dataset and is fixed on master, you can ignore it

what I need to do now

Now the delimiter of the csv reader is fixed, thanks :) 

I just added a comment suggesting to try using actual URLS instead of a manual download if possible.
This would make things more convenient for the users. Can you try using the `dl_manager` to download the train/dev/test csv files instead of requiring manual download ?

Also pinging @m3hrdadfi , since I just noticed that there's already a dataset script that was created 3 weeks ago for this dataset here: https://github.com/m3hrdadfi/wiki-summary/tree/master/datasets/wiki_summary_persian

@lhoestq I am getting this error while generating the dummy data
![Screenshot (181)](https://user-images.githubusercontent.com/33005287/102628819-50a40080-4170-11eb-9e96-efce74b45ff4.png)


Can you try by adding the flag `--match_text_files "*"` ?

now it worked

@lhoestq pytest on dummy data passed, but on real data raising this issue
![Screenshot (196)](https://user-images.githubusercontent.com/33005287/102630784-fa848c80-4172-11eb-9f7e-e5a58dcf7abe.png)
how to resolve it


I see ! This is because the library did some verification to make sure it downloads the same files as in the first time you ran the `datasets-cli test` command with `--save_infos`. Since we're now downloading files, the verification fails. 

To fix that you just need to regenerate the dataset_infos.json file:
```
datasets-cli test ./datasets/wiki_summary --save_infos --all_configs --ignore_verifications
```

@lhoestq I have modified everything and It worked fine, dont know why it is not passing the tests 

Awesome thank you !

The CI error `tests/test_file_utils.py::TempSeedTest::test_tensorflow` is not related to your dataset and is fixed on master.
You can ignore it :) 

@lhoestq anything left to do ?

The dataset script is all good now ! The dummy data and the dataset_infos.json file are good too :) 

@lhoestq yay, thanks for helping me out , 

merging since the CI is fixed on master

@tanmoyio @lhoestq 

Thank you both!
Inconsistent argument names.

Just find it a wee bit odd that in the transformers library `predictions` are those made by the model:
https://github.com/huggingface/transformers/blob/master/src/transformers/trainer_utils.py#L51-L61

While in many datasets metrics they are the ground truth labels:
https://github.com/huggingface/datasets/blob/c3f53792a744ede18d748a1133b6597fdd2d8d18/metrics/accuracy/accuracy.py#L31-L40

Do you think predictions & references should be swapped? I'd be willing to do some refactoring here if you agree.


Also for the `Accuracy` metric the `accuracy_score` method should have its args in the opposite order so `accuracy_score(predictions, references,,,)`.

Thanks for pointing this out ! 🕵🏻 
Predictions and references should indeed be swapped in the docstring.
However, the call to `accuracy_score` should not be changed, it [signature](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html#sklearn.metrics.accuracy_score) being:
```
sklearn.metrics.accuracy_score(y_true, y_pred, *, normalize=True, sample_weight=None)
```

Feel free to open a PR if you want to fix this :)
Add Curiosity Dialogs Dataset

Add Facebook [Curiosity Dialogs](https://github.com/facebookresearch/curiosity) Dataset.


@lhoestq I tried manually creating the dummy files. But unfortunately it was raising an error during testing the dummy data (regarding JSON parsing).

The JSONs are pretty big so I cannot actually open it without crashing the text editor.

 Do you have any suggestions?

@lhoestq I have made all the changes you mentioned.
Added OPUS ParaCrawl

Dataset : http://opus.nlpl.eu/ParaCrawl.php


@lhoestq I saw some common changes you made on the other PR's (Similar Opus Datasets). I fixed those changes here. Can you please review it once ? 
Thanks.

@rkc007 @lhoestq just noticed a dataset named para_crawl has been added a long time ago: #91.

They're not exactly the same so it's ok to have both.

Especially the `para_crawl` that already exists only uses the text from the ParaCrawl release 4.

Could you regenerate the dataset_infos.json @rkc007 please ?
It looks like it has some issues due to the dataset class name change

@SBrandeis Thank you for suggesting changes. I made the changes you suggested. 

@lhoestq I generated `dataset_infos.json` again. I ran both tests(Dummy & Real data) and it passed. Can you please review it again?

merging since the CI is fixed on master
Muchocine - Spanish movie reviews dataset



Hi @mapmeld !
have you had a chance to take a look at my suggestions ?

Feel free to ping me if you have questions or when you're ready for a review

@lhoestq unfortunately I don't have any more information about where the dataset comes from

It's fine, you can just add the sections titles back and leave the content with `[More Information Needed]`



added missing sections, updated the Python code ✅
adding tmu-gfm-dataset

Adding TMU-GFM-Dataset for Grammatical Error Correction.

https://github.com/tmu-nlp/TMU-GFM-Dataset

A dataset for GEC metrics with manual evaluations of grammaticality, fluency, and meaning preservation for system outputs.
More detail about the creation of the dataset can be found in [Yoshimura et al. (2020)](https://www.aclweb.org/anthology/2020.coling-main.573.pdf).


@lhoestq Thank you for your code review! I think I could do the necessary corrections. Could you please check it again when you have time?

Thank you for merging!
Add dataset COrpus of Urdu News TExt Reuse (COUNTER).



Just a small revision from simon's review: 20KB for the dummy_data.zip is fine, you can keep them this way.

Also the CI is failing because of an error `tests/test_file_utils.py::TempSeedTest::test_tensorflow` that is not related to your dataset and is fixed on master. You can ignore it

merging since the Ci is fixed on master
Adding Igbo NER data 

This PR adds the Igbo NER dataset.
Data: https://github.com/IgnatiusEzeani/IGBONLP/tree/master/ig_ner 


Thanks for the PR @purvimisal. 

Few comments below. 

Hi, @lhoestq Thank you for the review. I have made all the changes. PTAL! 

the CI error is not related to your dataset, merging
Added RONEC dataset.



Thanks for the PR @iliemihai . 

Few comments - 

Can you run - 
`python datasets-cli dummy_data ./datasets/ronec --auto_generate` to generate dummy data.

Also, before committing files run : 
`make style`
`flake8 datasets`
then you can add and commit files.

> Thanks for the PR @iliemihai .
> 
> Few comments -
> 
> Can you run -
> `python datasets-cli dummy_data ./datasets/ronec --auto_generate` to generate dummy data.
> 
> Also, before committing files run :
> `make style`
> `flake8 datasets`
> then you can add and commit files.

Sorry, forgot to generate dummy data. I will do it now :D

Awesome, good job @iliemihai !
I think the PR is ready to merge.
@lhoestq would you mind double-checking this ?

Had to regenerate the dummy data since I just found out they were empty files
Add ar rest reviews

added restaurants reviews in Arabic for sentiment analysis tasks


Copy-pasted from the Slack discussion:
the annotation and language creators should be found , not unknown
the example should go under the "Data Instances" paragraph, not "Data fields"
can you remove the abstract from the citation and add it to the dataset description? More people will see that

@yjernite done! thanks for the feedback

@lhoestq not sure why it's failing tests now, I only changed cosmetics

You can ignores these errors
```

=========================== short test summary info ===========================
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_chr_en
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_chr_en
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_great_code
```

they're fixed on master

Feel free to ping me for the final review once you managed to change to ClassLabel :) 

Hey @lhoestq I was able to fix it !! I think the same errors appeared on circleCI and now it's hopefully ready to be merged?

@lhoestq done! thanks for your review 

merging since the CI is fixed on master
❓ Sharing ElasticSearch indexed dataset 

Hi there,

First of all, thank you very much for this amazing library. Datasets have become my preferred data structure for basically everything I am currently doing.

**Question:** I'm working with a dataset and I have an elasticsearch container running at localhost:9200. I added an elasticsearch index and I was wondering

- how can I know where it has been saved?

- how can I share the indexed dataset with others?

I tried to dig into the docs, but could not find anything about that.

Thank you very much for your help.

Best,
Pietro

Edit: apologies for the wrong label

Labels: name: dataset request

Hello @pietrolesci , I am not sure to understand what you are trying to do here.

If you're looking for ways to save a dataset on disk, you can you the `save_to_disk` method:
```python
>>> import datasets
>>> loaded_dataset = datasets.load("dataset_name")
>>> loaded_dataset.save_to_disk("/path/on/your/disk")
```

The saved dataset can later be retrieved using:
```python
>>> loaded_dataset = datasets.Dataset.load_from_disk("/path/on/your/disk")
```

Also, I'd recommend posting your question directly in the issue section of the [elasticsearch repo](https://github.com/elastic/elasticsearch)

Hi @SBrandeis,

Thanks a lot for picking up my request. 

Maybe I can clarify my use-case with a bit of context. Say I have the IMDb dataset. I create an ES index on it. Now I can save and reload the dataset from disk normally. Once I reload the dataset, it is easy to retrieve the ES index on my machine. I was wondering: is there a way I can share the (now) indexed version of the IMDb dataset with my colleagues without requiring them to re-index it?

Thanks a lot in advance for your consideration.

Best,

Pietro

Thanks for the clarification.

I am not familiar with ElasticSearch, but if I understand well you're trying to migrate your data along with the ES index.
My advice would be to check out ES documentation, for instance, this might help you: https://www.elastic.co/guide/en/cloud/current/ec-migrate-data.html

Let me know if it helps
Add 1.5 billion words Arabic corpus 

Needs https://github.com/huggingface/datasets/pull/1429 to work. 


Can't pass dummy data tests. For the instructions, it asks me to generate the following file `dummy_data/Youm7_XML_utf_8.rar/Youm7_utf_8.xml` which is strange, any ideas @lhoestq ?

cc: I tested the data locally and it works, maybe the dummy tests doesn't support `rar` ? 

In the dummy_data.zip files you must include the rar file as if is was already extracted.
In particular here `Youm7_XML_utf_8.rar` is a directory (not an archive).

Also I'm getting `BadRarFile: Failed the read enough data: req=16384 got=51` while trying to download and extract the `Alittihad_XML_utf_8.rar` file. Do you have this issue as well ?

I have rarfile 4.0

Sorry it was my mistake, I missed up the directories, it works now. Not sure why you got that error. I have the same version of `rarfile`. Between, there were some suggestions to change the dataset from `1bn_words_arabic` to `arabic_billion_words` like https://github.com/huggingface/datasets/tree/master/datasets/spanish_billion_words. 


I'm ok with renaming the dataset `arabic_billion_words` if you want.
Note that you will need to rename class name `ArabicBillionWords` instead of `BillionWords`
(though `BillionWords` was not matching `1bn_words_arabic` anyway)

You will need to regenerate the dataset_infos.json file after this change.
OR alternatively just replace all mentions of `billion_words` with `arabic_billion_words` in dataset_infos.json <- this trick should save you some time :)

Hmmm I'm still not able to run it on my side because of the rar error (I'm running macos)
I just tried with rarfile 3.1 and it didn't work either.
I would like to be able to run it end-to-end on my side before merging if you don't mind. Let me investigate this issue a little bit

No worries, I will investigate it as well. 

I created a minimal example in [colab ](https://colab.research.google.com/drive/11ijesuGbrQylANka0VdsZ5vXwIuxkheY?usp=sharing).

Nice thanks, maybe it's just an issue on my side then

Ok I managed to solve the BadRarFile issue on my side :) 
To fix it I had to install the `unrar` tool for macos (though it seems it's not available with `brew install` anymore, I had to install it from elsewhere).
Added Wiki Asp dataset

Hello,

I have added Wiki Asp dataset. Please review the PR.


> Awesome thank you !
> 
> I just left one comment.
> 
> Also it looks like the dummy_data.zip files are quite big (around 500KB each)
> Can you try to reduce their sizes please ? Ideally they should be <20KB each
> 
> To do so feel free to take a look inside them and in the jsonl files only keep 1 or 2 samples instead of 5 and also remove big chunks of text to only keep a few passages.

Thanks, I have updated the dummy data to keep each domain <20/30KB.

> > Awesome thank you !
> > I just left one comment.
> > Also it looks like the dummy_data.zip files are quite big (around 500KB each)
> > Can you try to reduce their sizes please ? Ideally they should be <20KB each
> > To do so feel free to take a look inside them and in the jsonl files only keep 1 or 2 samples instead of 5 and also remove big chunks of text to only keep a few passages.
> 
> Thanks, I have updated the dummy data to keep each domain <20/30KB.

Looks like this branch has other commits. I will open a new PR with suggested changes.

opened a new PR #1612
Adding the HAREM dataset

Adding the HAREM dataset, a Portuguese language dataset for NER tasks


Thanks for the changes !

Sorry if I wasn't clear about the suggestion of adding the `raw` dataset as well.
By `raw` I meant the dataset with its original features, i.e. not tokenized to follow the conll format for NER.
The `raw` dataset has data fields `doc_text`, `doc_id` and `entities`.

Alright, @lhoestq, now I understand your suggestion, but the JSON files the script downloads aren't actually the "raw" HAREM dataset, the [real raw version](https://www.linguateca.pt/primeiroHAREM/harem/ColeccaoDouradaHAREM.zip) of it is on XML format that needs a lot of preprocessing. Those JSON are just pre-processed versions of the dataset.

I can make the config of the raw version of the pre-processed dataset, but I'll leave a comment on the dataset summary about those details.

Oh I see ! Sorry I thought there really were the raw (i.e. original/unprocessed) files

The original xml format doesn't seem very practical to use :/ 
We can ignore that and only keep the default and selective configs then.
Can you revert to the old configs ?

Thanks again for adding this dataset !

Hi @lhoestq, I've reverted the changes and put more details on README about the dataset.
Adding CLIMATE-FEVER dataset

This PR request the addition of the CLIMATE-FEVER dataset:
A dataset adopting the FEVER methodology that consists of 1,535 real-world claims regarding climate-change collected on the internet. Each claim is accompanied by five manually annotated evidence sentences retrieved from the English Wikipedia that support, refute or do not give enough information to validate the claim totalling in 7,675 claim-evidence pairs. The dataset features challenging claims that relate multiple facets and disputed cases of claims where both supporting and refuting evidence are present.

More information can be found at:
- Homepage: <http://climatefever.ai>
- Paper: <https://arxiv.org/abs/2012.00614>




I `git rebase`ed my branch to `upstream/master` as suggested in point 7 of <https://huggingface.co/docs/datasets/share_dataset.html> and subsequently used `git pull` to be able to push to my remote branch. However, I think this messed up the history.

Please let me know if I should create a clean new PR with my changes.

Update: I also fixed the dataset name in the Dataset Card.

Dear @SBrandeis , @lhoestq . I am not sure how to fix the PR with respect to the additional files that are currently included in the commits. Could you provide me with an example? Otherwise I would be happy to close/re-open another PR. Please let me know if anything is missing for the review.

Hi @tdiggelm, thanks for the contribution! This dataset is really awesome.
I believe creating a new branch from master and opening a new PR with your changes is the simplest option since no review has been done yet. Feel free to ping us when it's done.

> Hi @tdiggelm, thanks for the contribution! This dataset is really awesome.
> I believe creating a new branch from master and opening a new PR with your changes is the simplest option since no review has been done yet. Feel free to ping us when it's done.

Thank you very much for your quick reply! Will do ASAP and ping you when done.

closing in favor of #1623
Fix squad V2 metric script

The current squad v2 metric doesn't work with the squad (v1 or v2) datasets. The script is copied from `squad_evaluate` in transformers that requires the labels (with multiple answers) to be like this:
```
references = [{'id': 'a', 'answers': [
    {'text': 'Denver Broncos', 'answer_start': 177},
    {'text': 'Denver Broncos', 'answer_start': 177}
]}]
```
while the dataset had references like this:
```
references = [{'id': 'a', 'answers': 
    {'text': ['Denver Broncos' 'Denver Broncos'], 'answer_start': [177, 177]}
}]
```

Using one or the other format fails with the current squad v2 metric:
```
from datasets import load_metric
metric = load_metric("squad_v2")
predictions = [{'id': 'a', 'prediction_text': 'Denver Broncos', 'no_answer_probability': 0.0}]
references = [{'id': 'a', 'answers': [
    {'text': 'Denver Broncos', 'answer_start': 177},
    {'text': 'Denver Broncos', 'answer_start': 177}
]}]
metric.compute(predictions=predictions, references=references)
```
fails as well as
```
from datasets import load_metric
metric = load_metric("squad_v2")
predictions = [{'id': 'a', 'prediction_text': 'Denver Broncos', 'no_answer_probability': 0.0}]
references = [{'id': 'a', 'answers': 
    {'text': ['Denver Broncos' 'Denver Broncos'], 'answer_start': [177, 177]}
}]
metric.compute(predictions=predictions, references=references)
```

This is because arrow reformats the references behind the scene.

With this PR (tested locally), both the snippets up there work and return proper results.


The script with changes is used and tested in [#8924](https://github.com/huggingface/transformers/pull/8924). It gives the same results as the old `evaluate_squad` function when used on the same predictions.

merging since the CI is fixed on master
Add retries to HTTP requests

## What does this PR do ?

Adding retries to HTTP GET & HEAD requests, when they fail with a `ConnectTimeout` exception.

The "canonical" way to do this is to use [urllib's Retry class](https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#urllib3.util.Retry) and wrap it in a [HttpAdapter](https://requests.readthedocs.io/en/master/api/#requests.adapters.HTTPAdapter). Seems a bit overkill to me, plus it forces us to use the `requests.Session` object. I prefer this simpler implementation. I'm open to remarks and suggestions @lhoestq @yjernite 

Fixes #1102 

Labels: name: enhancement

merging this one then :)
Not able to use 'jigsaw_toxicity_pred' dataset

 When trying to use jigsaw_toxicity_pred dataset, like this in a [colab](https://colab.research.google.com/drive/1LwO2A5M2X5dvhkAFYE4D2CUT3WUdWnkn?usp=sharing):
```
from datasets import list_datasets, list_metrics, load_dataset, load_metric

ds = load_dataset("jigsaw_toxicity_pred")
```
 
I see below error:

> FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py

During handling of the above exception, another exception occurred:

FileNotFoundError                         Traceback (most recent call last)
FileNotFoundError: Couldn't find file at https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py

During handling of the above exception, another exception occurred:

FileNotFoundError                         Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/datasets/load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, **download_kwargs)
    280                 raise FileNotFoundError(
    281                     "Couldn't find file locally at {}, or remotely at {} or {}".format(
--> 282                         combined_path, github_file_path, file_path
    283                     )
    284                 )

FileNotFoundError: Couldn't find file locally at jigsaw_toxicity_pred/jigsaw_toxicity_pred.py, or remotely at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py or https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py


Hi @jassimran,
The `jigsaw_toxicity_pred` dataset has not been released yet, it will be available with version 2 of `datasets`, coming soon.
You can still access it by installing the master (unreleased) version of datasets directly :
`pip install git+https://github.com/huggingface/datasets.git@master`
Please let me know if this helps

Thanks.That works for now.