NyssaW/github-issues huggingface dataset:
AttributeError: 'DatasetDict' object has no attribute 'train_test_split'

The following code fails with "'DatasetDict' object has no attribute 'train_test_split'" - am I doing something wrong?
```
from datasets import load_dataset
dataset = load_dataset('csv', data_files='data.txt')
dataset = dataset.train_test_split(test_size=0.1)
```

> AttributeError: 'DatasetDict' object has no attribute 'train_test_split'

Comments:
Hi @david-waterworth!

As indicated in the error message, `load_dataset("csv")` returns a `DatasetDict` object, which is mapping of `str` to `Dataset` objects. I believe in this case the behavior is to return a `train` split with all the data.
`train_test_split` is a method of the `Dataset` object, so you will need to do something like this:
```python
dataset_dict = load_dataset(`'csv', data_files='data.txt')
dataset = dataset_dict['split name, eg train']
dataset.train_test_split(test_size=0.1)
```

Please let me know if this helps. 🙂 

Thanks, that's working - the same issue also tripped me up with training. 

I also agree https://github.com/huggingface/datasets/issues/767 would be a useful addition. 

Closing this now

> ```python
> dataset_dict = load_dataset(`'csv', data_files='data.txt')
> dataset = dataset_dict['split name, eg train']
> dataset.train_test_split(test_size=0.1)
> ```

I am getting error like
KeyError: 'split name, eg train'
Could you please tell me how to solve this?

dataset = load_dataset('csv', data_files=['files/datasets/dataset.csv'])
dataset = dataset['train']
dataset = dataset.train_test_split(test_size=0.1)

!curl -L "https://app.roboflow.com/ds/YQYgzFyKns?key=f0IwaEetrr" > roboflow.zip; unzip roboflow.zip; rm roboflow.zip

from datasets import load_dataset

dataset = load_dataset("imagefolder", data_dir="/content/")
dataset["train"][0]

dataset["train"][-1]

train_ds = load_dataset("imagefolder", data_dir="/content/train/")
test_ds = load_dataset("imagefolder", data_dir="/content/test/")
val_ds = load_dataset("imagefolder", data_dir="/content/valid/")

train_ds.features

and i got error 
AttributeError                            Traceback (most recent call last)
[<ipython-input-6-289222110c33>](https://localhost:8080/#) in <cell line: 1>()
----> 1 train_ds.features

AttributeError: 'DatasetDict' object has no attribute 'features'

This has been closed, you should open a new issue describing what your problem is.
connection error 

Hi
I am hitting to this error, thanks 

```
> Traceback (most recent call last):
  File "finetune_t5_trainer.py", line 379, in <module>
    main()
  File "finetune_t5_trainer.py", line 208, in main
    if training_args.do_eval or training_args.evaluation_strategy != EvaluationStrategy.NO
  File "finetune_t5_trainer.py", line 207, in <dictcomp>
    for task in data_args.eval_tasks}
  File "/workdir/seq2seq/data/tasks.py", line 70, in get_dataset
    dataset = self.load_dataset(split=split)
  File "/workdir/seq2seq/data/tasks.py", line 66, in load_dataset
    return datasets.load_dataset(self.task.name, split=split, script_version="master")
  File "/usr/local/lib/python3.6/dist-packages/datasets/load.py", line 589, in load_dataset
    path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
  File "/usr/local/lib/python3.6/dist-packages/datasets/load.py", line 267, in prepare_module
    local_path = cached_path(file_path, download_config=download_config)
  File "/usr/local/lib/python3.6/dist-packages/datasets/utils/file_utils.py", line 308, in cached_path
    use_etag=download_config.use_etag,
  File "/usr/local/lib/python3.6/dist-packages/datasets/utils/file_utils.py", line 487, in get_from_cache
    raise ConnectionError("Couldn't reach {}".format(url))
ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/master/datasets/boolq/boolq.py
el/0 I1217 01:11:33.898849 354161 main shadow.py:210 Current job status: FINISHED
```

Comments:
This happen quite often when they are too many concurrent requests to github.

i can understand it’s a bit cumbersome to handle on the user side. Maybe we should try a few times in the lib (eg with timeout) before failing, what do you think @lhoestq ?

Yes currently there's no retry afaik. We should add retries

Retries were added in #1603 :) 
It will be available in the next release

Hi @lhoestq thank you for the modification, I will use`script_version="master"` for now  :), to my experience, also setting timeout to a larger number like 3*60 which I normally use helps a lot on this.
Access to key in DatasetDict map

It is possible that we want to do different things in the `map` function (and possibly other functions too) of a `DatasetDict`, depending on the key. I understand that `DatasetDict.map` is a really thin wrapper of `Dataset.map`, so it is easy to directly implement this functionality in the client code. Still, it'd be nice if there can be a flag, similar to `with_indices`, that allows the callable to know the key inside `DatasetDict`.

Comments:
Indeed that would be cool

Also FYI right now the easiest way to do this is
```python
dataset_dict["train"] = dataset_dict["train"].map(my_transform_for_the_train_set)
dataset_dict["test"] = dataset_dict["test"].map(my_transform_for_the_test_set)
```

I don't feel like adding an extra param for this simple usage makes sense, considering how many args `map` already has. 

(Feel free to re-open this issue if you don't agree with me)

I still think this is useful, since it's common that the data processing is different for training/dev/testing. And I don't know if the fact that `map` currently takes many arguments is a good reason not to support a useful feature.
IWSLT-17 Link Broken

```
FileNotFoundError: Couldn't find file at https://wit3.fbk.eu/archive/2017-01-trnmted//texts/DeEnItNlRo/DeEnItNlRo/DeEnItNlRo-DeEnItNlRo.tgz
```

Comments:
Sorry, this is a duplicate of #1287. Not sure why it didn't come up when I searched `iwslt` in the issues list.

Closing this since its a duplicate
Add helper to resolve namespace collision

Many projects use a module called `datasets`, however this is incompatible with huggingface datasets. It would be great if there if there was some helper or similar function to resolve such a common conflict. 

Comments:
Do you have an example?

I was thinking about using something like [importlib](https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly) to over-ride the collision. 

**Reason requested**: I use the [following template](https://github.com/jramapuram/ml_base/) repo where I house all my datasets as a submodule.

Alternatively huggingface could consider some submodule type structure like:

`import huggingface.datasets`
`import huggingface.transformers`

`datasets` is a very common module in ML and should be an end-user decision and not scope all of python ¯\_(ツ)_/¯ 


That's a interesting option indeed. We'll think about it.

It also wasn't initially obvious to me that the samples which contain `import datasets` were in fact importing a huggingface library (in fact all the huggingface imports are very generic - transformers, tokenizers, datasets...)
added irc disentangle dataset

added irc disentanglement dataset

Comments:
@lhoestq sorry, this was the only way I was able to fix the pull request 

@lhoestq Thank you for the feedback. I wondering whether I should be passing an 'id' field in the dictionary since the 'connections' reference the 'id' of the linked messages. This 'id' would just be the same as the id_ that is in the yielded tuple.

Yes indeed it would be cool to have the ids in the dictionary. This way the dataset can be shuffled and all without losing information about the connections. Can you add it if you don't mind ?

Thanks :) could you also add the ids in the dictionary since they're useful for the connection links ?

Thanks !
Also it looks like the dummy_data.zip were regenerated and are now back to being too big (300KB each).
Can you reduce their sizes ? You can actually just revert to the ones you had before the last commit
FileNotFoundError for `amazon_polarity`

Version: `datasets==v1.1.3`

### Reproduction
```python
from datasets import load_dataset
data = load_dataset("amazon_polarity")
```
crashes with
```bash
FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/amazon_polarity/amazon_polarity.py
```
and 
```bash
FileNotFoundError: Couldn't find file at https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/amazon_polarity/amazon_polarity.py
```
and
```bash
FileNotFoundError: Couldn't find file locally at amazon_polarity/amazon_polarity.py, or remotely at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/amazon_polarity/amazon_polarity.py or https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/amazon_polarity/amazon_polarity.py
```

Comments:
Hi @phtephanx , the `amazon_polarity` dataset has not been released yet. It will be available in the coming soon v2of `datasets` :) 

You can still access it now if you want, but you will need to install datasets via the master branch:
`pip install git+https://github.com/huggingface/datasets.git@master`
Adding CLIMATE-FEVER dataset

This PR request the addition of the CLIMATE-FEVER dataset:
A dataset adopting the FEVER methodology that consists of 1,535 real-world claims regarding climate-change collected on the internet. Each claim is accompanied by five manually annotated evidence sentences retrieved from the English Wikipedia that support, refute or do not give enough information to validate the claim totalling in 7,675 claim-evidence pairs. The dataset features challenging claims that relate multiple facets and disputed cases of claims where both supporting and refuting evidence are present.

More information can be found at:
- Homepage: <http://climatefever.ai>
- Paper: <https://arxiv.org/abs/2012.00614>



Comments:
I `git rebase`ed my branch to `upstream/master` as suggested in point 7 of <https://huggingface.co/docs/datasets/share_dataset.html> and subsequently used `git pull` to be able to push to my remote branch. However, I think this messed up the history.

Please let me know if I should create a clean new PR with my changes.

Update: I also fixed the dataset name in the Dataset Card.

Dear @SBrandeis , @lhoestq . I am not sure how to fix the PR with respect to the additional files that are currently included in the commits. Could you provide me with an example? Otherwise I would be happy to close/re-open another PR. Please let me know if anything is missing for the review.

Hi @tdiggelm, thanks for the contribution! This dataset is really awesome.
I believe creating a new branch from master and opening a new PR with your changes is the simplest option since no review has been done yet. Feel free to ping us when it's done.

> Hi @tdiggelm, thanks for the contribution! This dataset is really awesome.
> I believe creating a new branch from master and opening a new PR with your changes is the simplest option since no review has been done yet. Feel free to ping us when it's done.

Thank you very much for your quick reply! Will do ASAP and ping you when done.

closing in favor of #1623
Add comet metric

Hey! I decided to add our new Crosslingual Optimized Metric for Evaluation of Translation (COMET) to the list of the available metrics.

COMET was [presented at EMNLP20](https://www.aclweb.org/anthology/2020.emnlp-main.213/) and it is the highest performing metric, so far, on the WMT19 benchmark.

We also participated in the [WMT20 Metrics shared task ](http://www.statmt.org/wmt20/pdf/2020.wmt-1.101.pdf) where once again COMET was validated as a top-performing metric. 


I hope that this metric will help researcher's and industry workers to better validate their MT systems in the future 🤗 !

Cheers,
Ricardo





Comments:
I also thought a bit about the fact that "sources" can't be added to the batch.. but changing that would require a lot more changes. And I agree that the idea of adding them as part of the references is not ideal. Conceptually they are not references.

I would keep it like this for now.. And in the future, work on a more consistent batch interface.
Added the dataset clickbait_news_bg

There was a problem with my [previous PR 1445](https://github.com/huggingface/datasets/pull/1445) after rebasing, so I'm copying the dataset code into a new branch and submitting a new PR.

Comments:
Hi @tsvm  Great work! 
Since you have raised a clean PR could you close the earlier one -   #1445 ? 


> Hi @tsvm Great work!
> Since you have raised a clean PR could you close the earlier one - #1445 ?

Done.
Add Microsoft Research Sequential Question Answering (SQA) Dataset

For more information: https://msropendata.com/datasets/b25190ed-0f59-47b1-9211-5962858142c2

Comments:
I proposed something a few weeks ago in #898 (un-merged) but I think that the way that @mattbui added the dataset in the present PR is smarter and simpler should replace my PR #898.

(Narrator voice: *And it was around that time that Thomas realized that the community was now a lot smarter than him and he should hand-over the library he had started with @lhoestq to the community and stop pretending he knew everything about it.*)
Create README.md


Comments:
@ManuelFay thanks you so much for adding a dataset card, this is such a cool contribution!

This looks like it uses an old template for the card we've moved things around a bit and we have an app you should be using to get the tags and the structure of the Data Fields paragraph :) Would you mind moving your text to the newer format (we're also asking contributors to keep the full template structure, even if some sections still have [More Information Needed] for the time being)

Here's the link to the instructions:
https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card

Out of curiosity, what was your landing point for filling out the card? Did you follow the "Update on Github" when navigating the datasets? Trying to make the instructions as clear as possible :) 

@yjernite 

Perfect, I'll follow the instructions when I have a bit more time tomorrow ! I was actually browsing the new contributions after the dataset sprint and realized most of the "old" datasets were not tagged, so I just copied and pasted the readme from another dataset and was not aware there was precise instructions... Will fix !

BTW, amazing job with the retriBert work, I used the contrastive + in-batch negative quite a bit for various projects. Probably neither the time nor place to talk about that but I was curious as to why, in your original work, you prefered using a simple projection in the last layer to differentiate the question vs answer embedding, rather than allowing for bias in the dense layer or even just to fine-tune 2 different embedders for question + answer ? 

Cool! Looking forward to the next version!

Quick answer for retriBERT is that I expected a simple projection to generalize better and more importantly only having to store the gradients for the proj means training with larger batches :) If you want to keep chatting about it, feel free to send me an email!

Hi @ManuelFay ! 
If you're still interested in completing the FQuAD dataset card, note that we've generated one that is pre-filled.
Therefore feel free to complete it with the content you already have in your README.md.
This would be awesome ! And thanks again for your contribution :)

Yo @lhoestq , just not sure about the tag table at the top, I used @yjernite eli5 template so hope it's okay ! Also want to signal the streamlit app for dataset tagging has a weird behavior with the size categories when filling in the form. 

Thanks to you guys for doing that and sorry about the time it took, i completely forgot about it !
adding tmu-gfm-dataset

Adding TMU-GFM-Dataset for Grammatical Error Correction.

https://github.com/tmu-nlp/TMU-GFM-Dataset

A dataset for GEC metrics with manual evaluations of grammaticality, fluency, and meaning preservation for system outputs.
More detail about the creation of the dataset can be found in [Yoshimura et al. (2020)](https://www.aclweb.org/anthology/2020.coling-main.573.pdf).

Comments:
@lhoestq Thank you for your code review! I think I could do the necessary corrections. Could you please check it again when you have time?

Thank you for merging!
Add dataset COrpus of Urdu News TExt Reuse (COUNTER).


Comments:
Just a small revision from simon's review: 20KB for the dummy_data.zip is fine, you can keep them this way.

Also the CI is failing because of an error `tests/test_file_utils.py::TempSeedTest::test_tensorflow` that is not related to your dataset and is fixed on master. You can ignore it

merging since the Ci is fixed on master
Lama

This the LAMA dataset for probing facts and common sense from language models. 

See https://github.com/facebookresearch/LAMA for more details.

Comments:
Let me know why the pyarrow test is failing. For one of the config "trex", I had to load an initial datafile for a dictionary which is used to augment the rest of the datasets. In the dummy data, the dictionary file was truncated so I had to fudge that. I'm not sure if that is the issue.


@ontocord it just needs a rerun and it will be good to go.

THanks @tanmoyio. How do I do a rerun?

@ontocord contributor can’t rerun it, the maintainers will rerun it, it may take lil bit of time as there are so many PRs left to be reviewed and merged 

@lhoestq not sure why it is failing. i've made all modifications. 

merging since the CI is fixed on master
Adding Igbo NER data 

This PR adds the Igbo NER dataset.
Data: https://github.com/IgnatiusEzeani/IGBONLP/tree/master/ig_ner 

Comments:
Thanks for the PR @purvimisal. 

Few comments below. 

Hi, @lhoestq Thank you for the review. I have made all the changes. PTAL! 

the CI error is not related to your dataset, merging
Opus CAPES added

Dataset : http://opus.nlpl.eu/CAPES.php

Comments:
@lhoestq I saw some common changes you made on the other PR's (Similar Opus Datasets). I fixed those changes here. Can you please review it once ? 
Thanks.

Hi @rkc007 , thanks for the contribution.
Unfortunately, the CAPES dataset has already been added here: #1307
I'm closing the PR 

@lhoestq FYI
added air_dialogue

UPDATE2 (3797ce5): Updated for multi-configs 

UPDATE (7018082): manually created the dummy_datasets. All tests were cleared locally. Pushed it to origin/master

DRAFT VERSION (57fdb20):  (_no longer draft_)
Uploaded the air_dialogue database. 
dummy_data creation was failing in local, since the original downloaded file has some nested folders. Pushing it since the tests with real data was cleared. Will re-check & update via manually creating some dummy_data

Comments:
Added OPUS ParaCrawl

Dataset : http://opus.nlpl.eu/ParaCrawl.php

Comments:
@lhoestq I saw some common changes you made on the other PR's (Similar Opus Datasets). I fixed those changes here. Can you please review it once ? 
Thanks.

@rkc007 @lhoestq just noticed a dataset named para_crawl has been added a long time ago: #91.

They're not exactly the same so it's ok to have both.

Especially the `para_crawl` that already exists only uses the text from the ParaCrawl release 4.

Could you regenerate the dataset_infos.json @rkc007 please ?
It looks like it has some issues due to the dataset class name change

@SBrandeis Thank you for suggesting changes. I made the changes you suggested. 

@lhoestq I generated `dataset_infos.json` again. I ran both tests(Dummy & Real data) and it passed. Can you please review it again?

merging since the CI is fixed on master
Monero

Biomedical Romanian dataset :)

Comments:
Hi @iliemihai  - you need to add the Readme file! Otherwise seems good. 
Also don't forget to run `make style` & `flake8 datasets` locally, from the datasets folder

@skyprince999 I will add the README.d for it. Thank you :D 

Thanks for your contribution, @iliemihai. Are you still interested in adding this dataset?

We are removing the dataset scripts from this GitHub repo and moving them to the Hugging Face Hub: https://huggingface.co/datasets

We would suggest you create this dataset there. Please, feel free to tell us if you need some help.
Adding PolEval2019 Machine Translation Task dataset

Facing an error with pytest in training. Dummy data is passing.
README has to be updated.

Comments:
**NOTE:**

- Train and Dev: Manually downloaded (auto download is repeatedly giving `ConnectionError` for one of the files), Test: Auto Download
- Dummy test is passing
- The json file has been created with hard-coded paths for the manual downloads _(hardcoding has been removed from the final uploaded script)_
- datasets-cli is still **failing** . It is not picking the right directory for the config. For instance, my folder structure is as below:
 ```
            ~/Downloads/Data/
             |--- English-to-Polish
                   |--- (corresponding files) 
             |--- Russian-Polish
                   |--- (corresponding files) 
```

When ru-pl is selected, ideally it has to search in Russian-Polish folder, but it is searching in '/Downloads/Data/' folder and hence getting a FileNotFound error.

The command run is 
`python datasets-cli test datasets/poleval2019_mt/ --save_infos --all_configs --data_dir ~/Downloads/Data/
`


Hi !
Thanks for the changes :)

The only error left is the dummy data. Since we changed for standard downloads instead of manual downloads its structure changed. Fortunately you can auto-generate the dummy data with this command:

```
datasets-cli dummy_data ./datasets/poleval2019_mt --auto_generate --match_text_files "*"
```

Can you regenerate the dummy data using this command please ?

Thank you for the help @lhoestq !! I was generating the dummy dataset in a wrong way! That _--match_text_files "*"_ did the trick! Now all the tests have passed! :-)

Hi @vrindaprabhu ! Do you still have the Poleval2019 data files somewhere by any chance ? It appears the google drive URLs are not working anymore

Hi @lhoestq. Just checked. I do not have the backup of the data anywhere. It also appears that PolEval does not repeat its tasks, the data seem to have gone forever. Do you think I should try contacting the organizers for more info?

We tried already and they don't have the data anymore :(
Added Wiki Summary Dataset

Wiki Summary: Dataset extracted from Persian Wikipedia into the form of articles and highlights.
Link: https://github.com/m3hrdadfi/wiki-summary

Comments:
@lhoestq why my tests are not running?

Maybe an issue with CircleCI, let me try to make them run

The CI error `tests/test_file_utils.py::TempSeedTest::test_tensorflow` is not related to this dataset and is fixed on master, you can ignore it

what I need to do now

Now the delimiter of the csv reader is fixed, thanks :) 

I just added a comment suggesting to try using actual URLS instead of a manual download if possible.
This would make things more convenient for the users. Can you try using the `dl_manager` to download the train/dev/test csv files instead of requiring manual download ?

Also pinging @m3hrdadfi , since I just noticed that there's already a dataset script that was created 3 weeks ago for this dataset here: https://github.com/m3hrdadfi/wiki-summary/tree/master/datasets/wiki_summary_persian

@lhoestq I am getting this error while generating the dummy data
![Screenshot (181)](https://user-images.githubusercontent.com/33005287/102628819-50a40080-4170-11eb-9e96-efce74b45ff4.png)


Can you try by adding the flag `--match_text_files "*"` ?

now it worked

@lhoestq pytest on dummy data passed, but on real data raising this issue
![Screenshot (196)](https://user-images.githubusercontent.com/33005287/102630784-fa848c80-4172-11eb-9f7e-e5a58dcf7abe.png)
how to resolve it


I see ! This is because the library did some verification to make sure it downloads the same files as in the first time you ran the `datasets-cli test` command with `--save_infos`. Since we're now downloading files, the verification fails. 

To fix that you just need to regenerate the dataset_infos.json file:
```
datasets-cli test ./datasets/wiki_summary --save_infos --all_configs --ignore_verifications
```

@lhoestq I have modified everything and It worked fine, dont know why it is not passing the tests 

Awesome thank you !

The CI error `tests/test_file_utils.py::TempSeedTest::test_tensorflow` is not related to your dataset and is fixed on master.
You can ignore it :) 

@lhoestq anything left to do ?

The dataset script is all good now ! The dummy data and the dataset_infos.json file are good too :) 

@lhoestq yay, thanks for helping me out , 

merging since the CI is fixed on master

@tanmoyio @lhoestq 

Thank you both!
adding HindEncorp

adding Hindi Wikipedia corpus

Comments:
@lhoestq I have created a new PR by reforking and creating a new branch 

@rahul-art unfortunately this didn't quite work, here's how you can try again:
- `git checkout master` to go back to the main branch
- `git pull upstream master` to make it up to date
- `git checkout -b add_hind_encorp` to create a new branch

Then add the dataset script, `README.md`, `dummy_data.zip`, and `dataset_infos.json` to the tracked files for the branch with `git add` (please add all of these files individually, NOT the whole directory as we don't want the other data files)
Then after you have passed the style checks and the local tests, do:
- `git commit . -m initial_commit`
- `git push --set-upstream origin add_hind_encorp`

Then you can go to this branch on the WebApp and open a new PR

@yjernite #1557 created new PR
added TTC4900: A Benchmark Data for Turkish Text Categorization dataset

This PR adds the TTC4900 dataset which is a Turkish Text Categorization dataset by me and @basakbuluz. 

Homepage: [https://www.kaggle.com/savasy/ttc4900](https://www.kaggle.com/savasy/ttc4900)
Point of Contact: [Savaş Yıldırım](mailto:savasy@gmail.com) / @savasy


Comments:
@lhoestq, can you help with creating dummy_data?


Hi @yavuzKomecoglu did you manage to build the dummy data ?

> Hi @yavuzKomecoglu did you manage to build the dummy data ?

Hi, sorry for the return. I've created dummy_data.zip manually.

> Nice thank you !
> 
> Before we merge can you fill the two sections of the dataset card I suggested ?
> And also remove one remaining print statement

I updated your suggestions. Thank you very much for your support.

I think you accidentally pushed the readme of another dataset (name_to_nation).
I removed it so you have to `git pull`

Because of that I guess your changes about the ttc4900 was not included.
Feel free to ping me once they're added




> I think you accidentally pushed the readme of another dataset (name_to_nation).
> I removed it so you have to `git pull`
> 
> Because of that I guess your changes about the ttc4900 was not included.
> Feel free to ping me once they're added

I did `git pull` and updated readme **ttc4900**.

merging since the Ci is fixed on master
Added Wiki Asp dataset

Hello,

I have added Wiki Asp dataset. Please review the PR.

Comments:
> Awesome thank you !
> 
> I just left one comment.
> 
> Also it looks like the dummy_data.zip files are quite big (around 500KB each)
> Can you try to reduce their sizes please ? Ideally they should be <20KB each
> 
> To do so feel free to take a look inside them and in the jsonl files only keep 1 or 2 samples instead of 5 and also remove big chunks of text to only keep a few passages.

Thanks, I have updated the dummy data to keep each domain <20/30KB.

> > Awesome thank you !
> > I just left one comment.
> > Also it looks like the dummy_data.zip files are quite big (around 500KB each)
> > Can you try to reduce their sizes please ? Ideally they should be <20KB each
> > To do so feel free to take a look inside them and in the jsonl files only keep 1 or 2 samples instead of 5 and also remove big chunks of text to only keep a few passages.
> 
> Thanks, I have updated the dummy data to keep each domain <20/30KB.

Looks like this branch has other commits. I will open a new PR with suggested changes.

opened a new PR #1612
Add Hippocorpus Dataset


Comments:
> Before we merge can you try to reduce the size of the dummy_data.zip file ?
> 
> To do so feel free to only keep a few lines of the csv files ans also remove unnecessary chunks of texts (for example keep only the first sentences of a story).

Hi @lhoestq, I have reduced the size of the dummy_data.zip file by making the necessary changes you had suggested. 

merging since the CI is fixed on master
Adding Igbo monolingual dataset

This PR adds the Igbo Monolingual dataset.
Data: https://github.com/IgnatiusEzeani/IGBONLP/tree/master/ig_monoling
Paper: https://arxiv.org/abs/2004.00648  

Comments:
@lhoestq Thank you for the review. I have made all the changes you mentioned. PTAL!
add id_panl_bppt, a parallel corpus for en-id

Parallel Text Corpora for English - Indonesian

Comments:
Hi @lhoestq, thanks for the review. I will have a look and update it accordingly.

Strange error message :-)

```
>           tf_context = tf.python.context.context()  # eager mode context
E           AttributeError: module 'tensorflow' has no attribute 'python'
```
Ro sent

Movies reviews dataset for Romanian language.

Comments:
Hi @iliemihai, it looks like this PR holds changes from your previous PR #1493 .
Would you mind removing them from the branch please ?

@SBrandeis I am sorry. Yes I will remove them. Thank you :D 

Hi @lhoestq @SBrandeis @iliemihai

Is this still in progress or can I take over this one?

Thanks,
Gunjan

Hi,
While trying to add this dataset, I found some potential issues. 
The homepage mentioned is : https://github.com/katakonst/sentiment-analysis-tensorflow/tree/master/datasets/ro/, where the dataset is different from the URLs: https://raw.githubusercontent.com/dumitrescustefan/Romanian-Transformers/examples/examples/sentiment_analysis/ro/train.csv. It is unclear which dataset is "correct". I checked the total examples (train+test) in both places and they do not match.

We should use the data from dumitrescustefan and set the homepage to his repo IMO, since he's first author of the paper of the dataset.

Hi @lhoestq,

Cool, I'll get working on it.

Thanks

Hi @lhoestq, 

This PR can be closed.

Closing in favor of #2011 
Thanks again for adding it !
Add : Conv AI 2 (Messed up original PR)

@lhoestq Sorry I messed up the previous 2 PR's -> https://github.com/huggingface/datasets/pull/1462 -> https://github.com/huggingface/datasets/pull/1383. So created a new one. Also, everything is fixed in this PR. Can you please review it ?
Thanks in advance. 

Comments:
Add eHealth Knowledge Discovery dataset

This Spanish dataset can be used to mine knowledge from unstructured health texts. 

In particular, for:
- Entity recognition
- Relation extraction


Comments:
Thank you very much for your review @lewtun ! 

I've updated the script metadata, created the README and fixed the two details you commented.

Ready for another review! 🤗 

I've updated the task tag as we discussed and also added a couple of lines of code to make sure I include all the available examples.

Thank you again!
Add semeval 2020 task 11

Adding in propaganda detection task (task 11) from Sem Eval 2020

Comments:
@SBrandeis : Thanks for the feedback! Just updated to use context manager for the `open`s and removed the placeholder text from the `README`!

Great, thanks @ZacharySBrown !
Failing tests seem to be unrelated to your changes, merging the current master branch into yours should fix them.
ru_reviews dataset adding

RuReviews: An Automatically Annotated Sentiment Analysis Dataset for Product Reviews in Russian

Comments:
Hi @lhoestq  

I have added the readme as well 

Please do have a look at it when suitable 

Chatted with @darshan-gandhi on Slack about parsing examples into a separate text and sentiment field

Thanks for your contribution, @darshan-gandhi. Are you still interested in adding this dataset?

We are removing the dataset scripts from this GitHub repo and moving them to the Hugging Face Hub: https://huggingface.co/datasets

We would suggest you create this dataset there. Please, feel free to tell us if you need some help.
how to get all the options of a property in datasets 

Hi
could you tell me how I can get all unique options of a property of dataset?
for instance in case of boolq, if the user wants to know which unique labels it has, is there a way to access unique labels without getting all training data lables and then forming a set i mean? thanks

Comments:
In a dataset, labels correspond to the `ClassLabel` feature that has the `names` property that returns string represenation of the integer classes (or `num_classes` to get the number of different classes).

I think the `features` attribute of the dataset object is what you are looking for:
```
>>> dataset.features
{'sentence1': Value(dtype='string', id=None),
 'sentence2': Value(dtype='string', id=None),
 'label': ClassLabel(num_classes=2, names=['not_equivalent', 'equivalent'], names_file=None, id=None),
 'idx': Value(dtype='int32', id=None)
}
>>> dataset.features["label"].names
['not_equivalent', 'equivalent']
```

For reference: https://huggingface.co/docs/datasets/exploring.html
Added dataset Makhzan

Need help with the dummy data.

Comments:
The only CI error comes from 
```
FAILED tests/test_file_utils.py::TempSeedTest::test_tensorflow
```

which is not related to your PR and is fixed on master.

You can ignore it

@lhoestq I've made the changes. Please review and merge. 

I have a similar PR https://github.com/huggingface/datasets/pull/1562 for another dataset. I'll incorporate your comment about sorting and reducing dummy dataset size there.

The CI raises an error `FAILED tests/test_file_utils.py::TempSeedTest::test_tensorflow` but it's not related to this dataset.
This issue is fixed on master

You did all the work ;) thanks

merging since the CI is fixed on master
Add SelQA Dataset

Add the SelQA Dataset, a new benchmark for selection-based question answering tasks
Repo: https://github.com/emorynlp/selqa/
Paper: https://arxiv.org/pdf/1606.08513.pdf

Comments:
Hii please follow me

The CI error `FAILED tests/test_file_utils.py::TempSeedTest::test_tensorflow` is not related with this dataset and is fixed on master. You can ignore it

merging since the Ci is fixed on master
Add nq_open question answering dataset

Added nq_open Open-domain question answering dataset.

The NQ-Open task is currently being used to evaluate submissions to the EfficientQA competition, which is part of the NeurIPS 2020 competition track.

Comments:
@SBrandeis  thanks for the review, I applied your suggested changes, but CI is failing now not sure about the error.

Many thanks @Nilanshrajput  !
The failing tests on CI are not related to your changes, merging master on your branch should fix them :)
If you're interested in what causes the CI to fail, checkout [this commit](https://github.com/huggingface/datasets/commit/9a0f1e20ca1e783cb14c1ab1cc2f54b0b5b201e8)

@SBrandeis done!


Hello @Nilanshrajput, your PR includes changes from other branches too now (485 files changed)
Would you mind creating another branch from master with your changes and opening a new PR?

@SBrandeis sorry i messed up the git history, #1587 I opened this new pr! 

closing in favor of #1587
Adding COVID QA dataset in Chinese and English from UC SanDiego


Comments:
Changed the pre-processing based on the comments raised in [PR-1482](https://github.com/huggingface/datasets/pull/1482).The below command is passing in my local environment:

`python datasets-cli test datasets/covid_qa_ucsd/ --save_infos --all_configs --data_dir ~/Downloads/Medical-Dialogue-Dataset/CovidDailogue/`
Add Senti_Lex Dataset

TODO:
Fix feature format issue
Create dataset_info.json file
Run pytests
Make Style

Comments:
Better will be if you close this PR and make a fresh PR

Feel free to ping me if you also have questions about the dummy data

also it looks like this PR includes changes about dummy_data.zip files in the ./datasets//un_pc folder. Can you remove them ?

Thanks for all the advice @lhoestq. I've implemented the changes you kindly highlighted and have made sure the scripts pass all the test. I've also marked this as ready for review as I believe it's in a good place to be merged now.

Great suggestion I fixed the dummy data and the file paths.
Added RONEC dataset.


Comments:
Thanks for the PR @iliemihai . 

Few comments - 

Can you run - 
`python datasets-cli dummy_data ./datasets/ronec --auto_generate` to generate dummy data.

Also, before committing files run : 
`make style`
`flake8 datasets`
then you can add and commit files.

> Thanks for the PR @iliemihai .
> 
> Few comments -
> 
> Can you run -
> `python datasets-cli dummy_data ./datasets/ronec --auto_generate` to generate dummy data.
> 
> Also, before committing files run :
> `make style`
> `flake8 datasets`
> then you can add and commit files.

Sorry, forgot to generate dummy data. I will do it now :D

Awesome, good job @iliemihai !
I think the PR is ready to merge.
@lhoestq would you mind double-checking this ?

Had to regenerate the dummy data since I just found out they were empty files
Fake news english 4


Comments:
Thanks for the PR @MisbahKhan789 !

Few comments to help you along (I'm NOT a maintainer, just offering help to unblock the process) :-
 - Could you re-run `make style` and fix the errors related to code quality specific to your dataset in the `datasets/fake_news_english` folder?
(These seem to show errors that need manual fixes on running `flake8 datasets/fake_news_english`)
- Please run the local tests and check if they pass
`RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_real_dataset_<your-dataset-name>`
 `RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_<your-dataset-name>`
(If not, you may have to regenerate the dummy data and the `dataset_infos.json` files)



Hii please follow me

> Thanks for the PR @MisbahKhan789 !
> 
> Few comments to help you along (I'm NOT a maintainer, just offering help to unblock the process) :-
> 
> * Could you re-run `make style` and fix the errors related to code quality specific to your dataset in the `datasets/fake_news_english` folder?
>   (These seem to show errors that need manual fixes on running `flake8 datasets/fake_news_english`)
> * Please run the local tests and check if they pass
>   `RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_real_dataset_<your-dataset-name>`
>   `RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_<your-dataset-name>`
>   (If not, you may have to regenerate the dummy data and the `dataset_infos.json` files)

Hey Bharat, thanks for the reply. I actually submitted a new PR with the changes that are required :)
Adding NELL

NELL is a knowledge base and knowledge graph along with sentences used to create the KB. See http://rtw.ml.cmu.edu/rtw/ for more details.

Comments:
hi @lhoestq, I wanted to push another change to this branch b/c I found a bug in the parsing. I need to swap arg1 and arg2. I tried to git push -u origin nell but it didn't work. So I tried to do git push --force -u origin nell which seems to work, but nothing is happening to this branch. I think this is because it's closed. Do I need to open another PR?

The change should be below in _generate_examples:
                    best_arg1 = row[9]
                    best_arg2 = row[8]


Hi @ontocord !

Yup the easiest thing to do is to open a new PR with a title like "Fix NELL dataset argument order". If it's a simple fix we can look at it pretty fast :)
added conv_ai_3 dataset

Dataset : https://github.com/aliannejadi/ClariQ/


Comments:
@lhoestq Thank you for suggesting changes. I fixed all the changes you suggested. Can you please review it again? 

@lhoestq Thank you for reviewing and suggesting changes. I made the requested changes. Can you please review it again?

Thanks @lhoestq for reviewing it again. I made the required changes. Can you please have a look ?

merging since the CI is fixed on master
Added Times of India News Headlines Dataset

Dataset name: Times of India News Headlines
link: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/DPQMQH

Comments:
@lhoestq @abhishekkrthakur what happened here ?


@lhoestq everything alright here ?

@tanmoyio please have patience. @lhoestq has to look at 150+ PRs and it may take time. The PR looks good to me but we wait for his confirmation :) 🤗
Adding medical database chinese and english

Error in creating dummy dataset

Comments:
Let me know it that helps !
Also feel free to ping me if you have other questions or if I can help you.

Now I am getting an Assertion Error!
![image](https://user-images.githubusercontent.com/16264631/101943915-f5bf5600-3c11-11eb-84e5-045bbc472162.png)


All tests have passed. However, PyTest is still failing with the `AssertionError` as before. Also _datasets_info.json_ actually does not seem to provide much info. Please review and let me know what has to be improved.

Thanks!

[PR-1503](https://github.com/huggingface/datasets/pull/1503) on the COVID dialog dataset from the same University has similar features. I kept it separate because it is only on COVID qa. Also consists of only single files per language. Kindly let me know if it has to be added as two more configurations to this existing dataset itself. If it has to be added, then can I still freeze the file names like in PR-1503?

It's ok to have them separate
Add narrativeQA

Redo of #1368 #309 #499

In redoing the dummy data a few times, I ended up adding a load of files to git. Hopefully this should work.

Comments:
@lhoestq this is now only failing some random windows test (it appears to be somewhere in wnut_17)

This is a connection error, you can ignore it :) 
The level of activity on the lib is quite overwhelming, it stresses a bit the CI ^^
Inconsistent argument names.

Just find it a wee bit odd that in the transformers library `predictions` are those made by the model:
https://github.com/huggingface/transformers/blob/master/src/transformers/trainer_utils.py#L51-L61

While in many datasets metrics they are the ground truth labels:
https://github.com/huggingface/datasets/blob/c3f53792a744ede18d748a1133b6597fdd2d8d18/metrics/accuracy/accuracy.py#L31-L40

Do you think predictions & references should be swapped? I'd be willing to do some refactoring here if you agree.

Comments:
Also for the `Accuracy` metric the `accuracy_score` method should have its args in the opposite order so `accuracy_score(predictions, references,,,)`.

Thanks for pointing this out ! 🕵🏻 
Predictions and references should indeed be swapped in the docstring.
However, the call to `accuracy_score` should not be changed, it [signature](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html#sklearn.metrics.accuracy_score) being:
```
sklearn.metrics.accuracy_score(y_true, y_pred, *, normalize=True, sample_weight=None)
```

Feel free to open a PR if you want to fix this :)
Adding the HAREM dataset

Adding the HAREM dataset, a Portuguese language dataset for NER tasks

Comments:
Thanks for the changes !

Sorry if I wasn't clear about the suggestion of adding the `raw` dataset as well.
By `raw` I meant the dataset with its original features, i.e. not tokenized to follow the conll format for NER.
The `raw` dataset has data fields `doc_text`, `doc_id` and `entities`.

Alright, @lhoestq, now I understand your suggestion, but the JSON files the script downloads aren't actually the "raw" HAREM dataset, the [real raw version](https://www.linguateca.pt/primeiroHAREM/harem/ColeccaoDouradaHAREM.zip) of it is on XML format that needs a lot of preprocessing. Those JSON are just pre-processed versions of the dataset.

I can make the config of the raw version of the pre-processed dataset, but I'll leave a comment on the dataset summary about those details.

Oh I see ! Sorry I thought there really were the raw (i.e. original/unprocessed) files

The original xml format doesn't seem very practical to use :/ 
We can ignore that and only keep the default and selective configs then.
Can you revert to the old configs ?

Thanks again for adding this dataset !

Hi @lhoestq, I've reverted the changes and put more details on README about the dataset.
Add wiki lingua dataset

Hello @lhoestq ,

I am opening a fresh pull request as advised in my original PR https://github.com/huggingface/datasets/pull/1308

Thanks

Comments:
it’s failing because of `RemoteDatasetTest.test_load_dataset_orange_sum`
which i think is not the dataset you are doing a PR for. Try rebasing with:
```
git fetch upstream
git rebase upstream/master
git push -u -f origin your_branch
```

> it’s failing because of `RemoteDatasetTest.test_load_dataset_orange_sum`
> which i think is not the dataset you are doing a PR for. Try rebasing with:
> 
> ```
> git fetch upstream
> git rebase upstream/master
> git push -u -f origin your_branch
> ```

Thanks, my branch seems to be up to date. 
```Current branch add-wiki-lingua-dataset is up to date.```

Also where do the google drive urls come from ?

looks like this PR includes changes about many other files than the ones for wiki_lingua.

Can you create another branch and another PR ?
(or you can try to fix this branch with rebase and push force if you're familiar with it)

Thanks for fixing the dummy data and removing the glob call :) 

> looks like this PR includes changes about many other files than the ones for wiki_lingua.
> 
> Can you create another branch and another PR ?
> (or you can try to fix this branch with rebase and push force if you're familiar with it)

Easier to create a new branch and submit, I have submitted a new PR #1582 

Closing this one in favor of #1582
add Indonesian newspapers (id_newspapers_2018)

The dataset contains around 500K articles (136M of words) from 7 Indonesian newspapers. The size of uncompressed 500K json files (newspapers-json.tgz) is around 2.2GB.

Comments:
Looks like there's a `Path` issue on windows. Could you try switching to
`glob.glob(os.path.join(article_dir, "*.json"))`

> Looks like there's a `Path` issue on windows. Could you try switching to
> `glob.glob(os.path.join(article_dir, "*.json"))`

Thanks, I replaced it with glob. Let's see if it solves the issue. Anyway, the main directory has a space, could it make the issue on windows? the test on linux don't have this problem.

It seems glob doesn't help also. Btw, one of the failing test tried to connect aws which failed:
```
C:\tools\miniconda3\lib\site-packages\urllib3\connection.py:160: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('s3.amazonaws.com', 443), timeout = 10, source_address = None
socket_options = [(6, 1, 1)]

```
Why did it try to connect to aws? I don't use it.

It seems that the circleci make a test for whole datasets repository, that means if only one of the dataset in the official repository has a download issue, this will also affect the test of a new dataset like mine, isn't it?
I changed the url to my newspaper dataset which contains only few simple json files and simple directory structure. But it still failed. And it failed not only on windows test. This is one of the error message:
```
-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_chr_en
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_chr_en
===== 4 failed, 2667 passed, 2052 skipped, 4 warnings in 432.05s (0:07:12) =====

Exited with code exit status 1
CircleCI received exit code 1
```
The test failed on twitter dataset even my dataset has nothing to do with twitter? 

merging since the CI is fixed on master

Hi, thanks for merging the dataset. I create a new PR (#1499) since I need to update the link to the dataset.