rIsHu009/github-issues-updated huggingface dataset:
Title: Adding UKP Argument Aspect Similarity Corpus
Author: rabeehk (CONTRIBUTOR)
State: closed
Created: 2020-12-24T11:01:31Z
Labels: dataset request

Hi, this would be great to have this dataset included.

## Adding a Dataset
- **Name:** UKP Argument Aspect Similarity Corpus
- **Description:** The UKP Argument Aspect Similarity Corpus (UKP ASPECT) includes 3,595 sentence pairs over 28 controversial topics. Each sentence pair was annotated via crowdsourcing as either “high similarity”, “some similarity”, “no similarity” or “not related” with respect to the topic.
- **Paper:** https://www.aclweb.org/anthology/P19-1054/
- **Data:** https://tudatalib.ulb.tu-darmstadt.de/handle/tudatalib/1998
- **Motivation:** this is one of the datasets currently used frequently in recent adapter papers like https://arxiv.org/pdf/2005.00247.pdf 

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Thank you

Comments:
Adding a link to the guide on adding a dataset if someone want to give it a try: https://github.com/huggingface/datasets#add-a-new-dataset-to-the-hub

we should add this guide to the issue template @lhoestq 

thanks @thomwolf , this is added now. The template is correct, sorry my mistake not to include it. 

Available here: https://huggingface.co/datasets/UKPLab/UKP_ASPECT
Title: `Dataset.map` disable progress bar
Author: Nickil21 (NONE)
State: closed
Created: 2020-12-23T17:53:42Z

I can't find anything to turn off the `tqdm` progress bars while running a preprocessing function using `Dataset.map`. I want to do akin to `disable_tqdm=True` in the case of `transformers`. Is there something like that?

Comments:
Progress bar can be disabled like this:
```python
from datasets.utils.logging import set_verbosity_error
set_verbosity_error()
```

There is this line in `Dataset.map`:
```python
not_verbose = bool(logger.getEffectiveLevel() > WARNING)
```

So any logging level higher than `WARNING` turns off the progress bar.

From the linked issues above, an up-to-date solution is:

```python
from datasets.utils.logging import disable_progress_bar
disable_progress_bar()
```

https://github.com/huggingface/datasets/blob/c6e08fcfc3a04e53430c26fa7c07da4cb18d977d/src/datasets/utils/logging.py#L233-L236

Why not have a parameter in the function such as `progress: bool = True`?

+1. We shouldn't need to play with logging levels for a simple thing like this. For instance, trainers have an option `show_progress` that does exactly this.

Bump on this, such a simple QOL issue why can't we fix this?
Title: Add CLIMATE-FEVER dataset
Author: tdiggelm (CONTRIBUTOR)
State: closed
Created: 2020-12-22T13:34:05Z

As suggested by @SBrandeis , fresh PR that adds CLIMATE-FEVER. Replaces PR #1579.

---

A dataset adopting the FEVER methodology that consists of 1,535 real-world claims regarding climate-change collected on the internet. Each claim is accompanied by five manually annotated evidence sentences retrieved from the English Wikipedia that support, refute or do not give enough information to validate the claim totalling in 7,675 claim-evidence pairs. The dataset features challenging claims that relate multiple facets and disputed cases of claims where both supporting and refuting evidence are present.

More information can be found at:

* Homepage: http://climatefever.ai
* Paper: https://arxiv.org/abs/2012.00614

Comments:
Thank you @lhoestq for your comments! 😄 I added your suggested changes, ran the tests and regenerated `dataset_infos.json` and `dummy_data`.
Title: Can't call shape on the output of select()
Author: noaonoszko (CONTRIBUTOR)
State: closed
Created: 2020-12-22T13:18:40Z

I get the error `TypeError: tuple expected at most 1 argument, got 2` when calling `shape` on the output of `select()`.
It's line 531 in shape in arrow_dataset.py that causes the problem:
``return tuple(self._indices.num_rows, self._data.num_columns)``
This makes sense, since `tuple(num1, num2)` is not a valid call.
 
Full code to reproduce:

```python
dataset = load_dataset("cnn_dailymail", "3.0.0")
train_set = dataset["train"]
t = train_set.select(range(10))
print(t.shape)

Comments:
Indeed that's a typo, do you want to open a PR to fix it?

Yes, created a PR
Title: Adding myPOS2017 dataset
Author: hungluumfc (NONE)
State: closed
Created: 2020-12-22T04:04:55Z
Labels: dataset contribution

myPOS Corpus (Myanmar Part-of-Speech Corpus) for Myanmar language NLP Research and Developments

Comments:
I've updated the code and Readme to reflect your comments.
Thank you very much,

looks like this PR includes changes about many other files than the ones for myPOS2017

Could you open another branch and another PR please ?
(or fix this branch)

Hi @hungluumfc ! Have you had a chance to fix this PR so that it only includes the changes for `mypos` ? 

Feel free to ping me if you have questions or if I can help :) 

Thanks for your contribution, @hungluumfc. Are you still interested in adding this dataset?

We are removing the dataset scripts from this GitHub repo and moving them to the Hugging Face Hub: https://huggingface.co/datasets

We would suggest you create this dataset there. Please, feel free to tell us if you need some help.
Title: data loader for reading comprehension task
Author: songfeng (CONTRIBUTOR)
State: closed
Created: 2020-12-21T22:40:34Z

added doc2dial data loader and dummy data for reading comprehension task.

Comments:
Thank you for all the feedback! I have updated the dummy data with a zip under 30KB, which needs to include at least one data instance from both document domain and dialog domain. Please let me know if it is still too big. Thanks!

Thank you again for the feedback! I am not too sure what the preferable style for data instance in readme, but still added my edits. Thanks!
Title: Can't filter language:EN on https://huggingface.co/datasets
Author: davidefiocco (NONE)
State: closed
Created: 2020-12-21T15:23:23Z

When visiting https://huggingface.co/datasets, I don't see an obvious way to filter only English datasets. This is unexpected for me, am I missing something? I'd expect English to be selectable in the language widget. This problem reproduced on Mozilla Firefox and MS Edge:

![screenshot](https://user-images.githubusercontent.com/4547987/102792244-892e1f00-43a8-11eb-9e89-4826ca201a87.png)

Comments:
cc'ing @mapmeld 

Full language list is now deployed to https://huggingface.co/datasets ! Recommend close

Cool @mapmeld ! My 2 cents (for a next iteration), it would be cool to have a small search widget in the filter dropdown as you have a ton of languages now here! Closing this in the meantime.
Title: added TurkishMovieSentiment dataset
Author: yavuzKomecoglu (CONTRIBUTOR)
State: closed
Created: 2020-12-21T11:03:16Z

This PR adds the **TurkishMovieSentiment: This dataset contains turkish movie reviews.**

- **Homepage:** [https://www.kaggle.com/mustfkeskin/turkish-movie-sentiment-analysis-dataset/tasks](https://www.kaggle.com/mustfkeskin/turkish-movie-sentiment-analysis-dataset/tasks)
- **Point of Contact:** [Mustafa Keskin](https://www.linkedin.com/in/mustfkeskin/)

Comments:
> I just generated the dataset_infos.json file
> 
> Thanks for adding this one !

Thank you very much for your support.
Title: shuffle with torch generator 
Author: rabeehkarimimahabadi (NONE)
State: closed
Created: 2020-12-20T00:57:14Z
Labels: enhancement

Hi
I need to shuffle mutliple large datasets with `generator = torch.Generator()` for a distributed sampler which needs to make sure datasets are consistent across different cores, for this, this is really necessary for me to use  torch generator, based on documentation this generator is not supported with datasets, I really need to make shuffle work with this generator and I was wondering what I can do about this issue, thanks for your help 

@lhoestq

Comments:
Is there a way one can convert the two generator? not sure overall what alternatives I could have to shuffle the datasets with a torch generator, thanks 

@lhoestq  let me please expalin in more details, maybe you could help me suggesting an alternative to solve the issue for now, I have multiple large datasets using huggingface library, then I need to define a distributed sampler on top of it, for this I need to shard the datasets and give each shard to each core, but before sharding I need to shuffle the dataset, if you are familiar with distributed sampler in pytorch, this needs to be done based on seed+epoch generator to make it consistent across the cores they do it through defining a torch generator, I was wondering if you could tell me how I can shuffle the data for now, I am unfortunately blocked by this and have a limited time left, and I greatly appreciate your help on this. thanks 

@lhoestq  Is there a way I could shuffle the datasets from this library with a custom defined shuffle function? thanks for your help on this. 

Right now the shuffle method only accepts the `seed` (optional int) or `generator` (optional `np.random.Generator`) parameters.

Here is a suggestion to shuffle the data using your own shuffle method using `select`.
`select` can be used to re-order the dataset samples or simply pick a few ones if you want.
It's what is used under the hood when you call `dataset.shuffle`.

To use `select` you must have the list of re-ordered indices of your samples.

Let's say you have a `shuffle` methods that you want to use. Then you can first build your shuffled list of indices:
```python
shuffled_indices = shuffle(range(len(dataset)))
```

Then you can shuffle your dataset using the shuffled indices with 
```python
shuffled_dataset = dataset.select(shuffled_indices)
```

Hope that helps

thank you @lhoestq  thank you very much for responding to my question, this greatly helped me and remove the blocking for continuing my work, thanks. 

@lhoestq  could you confirm the method proposed does not bring the whole data into memory? thanks 

Yes the dataset is not loaded into memory

great. thanks a lot.
Title: shuffle does not accept seed 
Author: rabeehk (CONTRIBUTOR)
State: closed
Created: 2020-12-19T20:59:39Z
Labels: bug

Hi
I need to shuffle the dataset, but this needs to be based on epoch+seed to be consistent across the cores, when I pass seed to shuffle, this does not accept seed, could you assist me with this? thanks  @lhoestq

Comments:
Hi, did you check the doc on `shuffle`?
https://huggingface.co/docs/datasets/package_reference/main_classes.html?datasets.Dataset.shuffle#datasets.Dataset.shuffle

Hi Thomas
thanks for reponse, yes, I did checked it, but this does not work for me please see 

```
(internship) rkarimi@italix17:/idiap/user/rkarimi/dev$ python 
Python 3.7.9 (default, Aug 31 2020, 12:42:55) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import datasets 
2020-12-20 01:48:50.766004: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2020-12-20 01:48:50.766029: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
>>> data = datasets.load_dataset("scitail", "snli_format")
cahce dir  /idiap/temp/rkarimi/cache_home_1/datasets
cahce dir  /idiap/temp/rkarimi/cache_home_1/datasets
Reusing dataset scitail (/idiap/temp/rkarimi/cache_home_1/datasets/scitail/snli_format/1.1.0/fd8ccdfc3134ce86eb4ef10ba7f21ee2a125c946e26bb1dd3625fe74f48d3b90)
>>> data.shuffle(seed=2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: shuffle() got an unexpected keyword argument 'seed'

```

datasets version
`datasets                  1.1.2                     <pip>
`


Thanks for reporting ! 

Indeed it looks like an issue with `suffle` on `DatasetDict`. We're going to fix that.
In the meantime you can shuffle each split (train, validation, test) separately:
```python
shuffled_train_dataset = data["train"].shuffle(seed=42)
```
Title: Not able to use 'jigsaw_toxicity_pred' dataset
Author: jassimran (NONE)
State: closed
Created: 2020-12-19T17:35:48Z

When trying to use jigsaw_toxicity_pred dataset, like this in a [colab](https://colab.research.google.com/drive/1LwO2A5M2X5dvhkAFYE4D2CUT3WUdWnkn?usp=sharing):
```
from datasets import list_datasets, list_metrics, load_dataset, load_metric

ds = load_dataset("jigsaw_toxicity_pred")
```
 
I see below error:

> FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py

During handling of the above exception, another exception occurred:

FileNotFoundError                         Traceback (most recent call last)
FileNotFoundError: Couldn't find file at https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py

During handling of the above exception, another exception occurred:

FileNotFoundError                         Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/datasets/load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, **download_kwargs)
    280                 raise FileNotFoundError(
    281                     "Couldn't find file locally at {}, or remotely at {} or {}".format(
--> 282                         combined_path, github_file_path, file_path
    283                     )
    284                 )

FileNotFoundError: Couldn't find file locally at jigsaw_toxicity_pred/jigsaw_toxicity_pred.py, or remotely at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py or https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py

Comments:
Hi @jassimran,
The `jigsaw_toxicity_pred` dataset has not been released yet, it will be available with version 2 of `datasets`, coming soon.
You can still access it by installing the master (unreleased) version of datasets directly :
`pip install git+https://github.com/huggingface/datasets.git@master`
Please let me know if this helps

Thanks.That works for now.
Title: adding  ted_talks_iwslt
Author: skyprince999 (CONTRIBUTOR)
State: closed
Created: 2020-12-19T07:36:41Z

UPDATE2: (2nd Jan) Wrote a long writeup on the slack channel. I don't think this approach is correct. Basically this created language pairs (109*108) 
Running the `pytest `went for more than 40+ hours and it was still running!  
So working on a different approach, such that the number of configs = number of languages. Will make a new pull request with that. 

UPDATE: This requires manual download dataset

This is a draft version

Comments:
Closing this with reference to the new approach #1676
Title: added Semantic Scholar Open Research Corpus
Author: bhavitvyamalik (CONTRIBUTOR)
State: closed
Created: 2020-12-18T19:21:24Z

I picked up this dataset [Semantic Scholar Open Research Corpus](https://allenai.org/data/s2orc) but it contains 6000 files to be downloaded. I tried the current code with 100 files and it worked fine (took ~15GB space). For 6000 files it would occupy ~900GB space which I don’t have. Can someone from the HF team with that much of disk space help me with generate dataset_infos and dummy_data?

Comments:
I think we’ll need complete dataset_infos.json to create YAML tags. I ran the script again with 100 files after going through your comments and it was occupying ~16 GB space. So in total it should take ~960GB and I don’t have this much memory available with me. Also, I'll have to download the whole dataset for generating dummy data, right?
Title: Navigation version breaking
Author: mttk (NONE)
State: closed
Created: 2020-12-18T15:36:24Z

Hi, 

when navigating docs (Chrome, Ubuntu) (e.g. on this page: https://huggingface.co/docs/datasets/loading_metrics.html#using-a-custom-metric-script) the version control dropdown has the wrong string displayed as the current version: 

![image](https://user-images.githubusercontent.com/3007947/102632187-02cad080-414f-11eb-813b-28f3c8d80def.png)

**Edit:** this actually happens _only_ if you open a link to a concrete subsection.

IMO, the best way to fix this without getting too deep into the intricacies of retrieving version numbers from the URL would be to change [this](https://github.com/huggingface/datasets/blob/master/docs/source/_static/js/custom.js#L112) line to:
```
let label = (version in versionMapping) ? version : stableVersion
```
which delegates the check to the (already maintained) keys of the version mapping dictionary & should be more robust. There's a similar ternary expression [here](https://github.com/huggingface/datasets/blob/master/docs/source/_static/js/custom.js#L97) which should also fail in this case.

I'd also suggest swapping this [block](https://github.com/huggingface/datasets/blob/master/docs/source/_static/js/custom.js#L80-L90) to `string.contains(version) for version in versionMapping` which might be more robust. I'd add a PR myself but I'm by no means competent in JS :)

I also have a side question wrt. docs versioning: I'm trying to make docs for a project which are versioned alike to your dropdown versioning. I was wondering how do you handle storage of multiple doc versions on your server? Do you update what `https://huggingface.co/docs/datasets` points to for every stable release & manually create new folders for each released version?
So far I'm building & publishing (scping) the docs to the server with a github action which works well for a single version, but would ideally need to reorder the public files triggered on a new release.

Comments:
Not relevant for our current docs :).
Title: Add tests for the download functions ?
Author: SBrandeis (CONTRIBUTOR)
State: closed
Created: 2020-12-18T12:49:25Z
Labels: enhancement

AFAIK the download functions in `DownloadManager` are not tested yet. It could be good to add some to ensure behavior is as expected.

Comments:
We have some tests now for it under `tests/test_download_manager.py`.
Title: Add retries to HTTP requests
Author: SBrandeis (CONTRIBUTOR)
State: closed
Created: 2020-12-18T12:41:31Z
Labels: enhancement

## What does this PR do ?

Adding retries to HTTP GET & HEAD requests, when they fail with a `ConnectTimeout` exception.

The "canonical" way to do this is to use [urllib's Retry class](https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#urllib3.util.Retry) and wrap it in a [HttpAdapter](https://requests.readthedocs.io/en/master/api/#requests.adapters.HTTPAdapter). Seems a bit overkill to me, plus it forces us to use the `requests.Session` object. I prefer this simpler implementation. I'm open to remarks and suggestions @lhoestq @yjernite 

Fixes #1102

Comments:
merging this one then :)
Title: second update of the id_newspapers_2018
Author: cahya-wirawan (CONTRIBUTOR)
State: closed
Created: 2020-12-18T10:10:20Z

The feature "url" is currently set wrongly to data["date"], this PR fix it to data["url"].
I added also an additional POC.

Comments:
I close this PR, since it based on 1 week old repo. And I will create a new one
Title: AttributeError: 'DatasetDict' object has no attribute 'train_test_split'
Author: david-waterworth (NONE)
State: closed
Created: 2020-12-18T05:37:10Z
Labels: question

The following code fails with "'DatasetDict' object has no attribute 'train_test_split'" - am I doing something wrong?
```
from datasets import load_dataset
dataset = load_dataset('csv', data_files='data.txt')
dataset = dataset.train_test_split(test_size=0.1)
```

> AttributeError: 'DatasetDict' object has no attribute 'train_test_split'

Comments:
Hi @david-waterworth!

As indicated in the error message, `load_dataset("csv")` returns a `DatasetDict` object, which is mapping of `str` to `Dataset` objects. I believe in this case the behavior is to return a `train` split with all the data.
`train_test_split` is a method of the `Dataset` object, so you will need to do something like this:
```python
dataset_dict = load_dataset(`'csv', data_files='data.txt')
dataset = dataset_dict['split name, eg train']
dataset.train_test_split(test_size=0.1)
```

Please let me know if this helps. 🙂 

Thanks, that's working - the same issue also tripped me up with training. 

I also agree https://github.com/huggingface/datasets/issues/767 would be a useful addition. 

Closing this now

> ```python
> dataset_dict = load_dataset(`'csv', data_files='data.txt')
> dataset = dataset_dict['split name, eg train']
> dataset.train_test_split(test_size=0.1)
> ```

I am getting error like
KeyError: 'split name, eg train'
Could you please tell me how to solve this?

dataset = load_dataset('csv', data_files=['files/datasets/dataset.csv'])
dataset = dataset['train']
dataset = dataset.train_test_split(test_size=0.1)

!curl -L "https://app.roboflow.com/ds/YQYgzFyKns?key=f0IwaEetrr" > roboflow.zip; unzip roboflow.zip; rm roboflow.zip

from datasets import load_dataset

dataset = load_dataset("imagefolder", data_dir="/content/")
dataset["train"][0]

dataset["train"][-1]

train_ds = load_dataset("imagefolder", data_dir="/content/train/")
test_ds = load_dataset("imagefolder", data_dir="/content/test/")
val_ds = load_dataset("imagefolder", data_dir="/content/valid/")

train_ds.features

and i got error 
AttributeError                            Traceback (most recent call last)
[<ipython-input-6-289222110c33>](https://localhost:8080/#) in <cell line: 1>()
----> 1 train_ds.features

AttributeError: 'DatasetDict' object has no attribute 'features'

This has been closed, you should open a new issue describing what your problem is.
Title: connection error 
Author: rabeehkarimimahabadi (NONE)
State: closed
Created: 2020-12-17T09:18:34Z

Hi
I am hitting to this error, thanks 

```
> Traceback (most recent call last):
  File "finetune_t5_trainer.py", line 379, in <module>
    main()
  File "finetune_t5_trainer.py", line 208, in main
    if training_args.do_eval or training_args.evaluation_strategy != EvaluationStrategy.NO
  File "finetune_t5_trainer.py", line 207, in <dictcomp>
    for task in data_args.eval_tasks}
  File "/workdir/seq2seq/data/tasks.py", line 70, in get_dataset
    dataset = self.load_dataset(split=split)
  File "/workdir/seq2seq/data/tasks.py", line 66, in load_dataset
    return datasets.load_dataset(self.task.name, split=split, script_version="master")
  File "/usr/local/lib/python3.6/dist-packages/datasets/load.py", line 589, in load_dataset
    path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
  File "/usr/local/lib/python3.6/dist-packages/datasets/load.py", line 267, in prepare_module
    local_path = cached_path(file_path, download_config=download_config)
  File "/usr/local/lib/python3.6/dist-packages/datasets/utils/file_utils.py", line 308, in cached_path
    use_etag=download_config.use_etag,
  File "/usr/local/lib/python3.6/dist-packages/datasets/utils/file_utils.py", line 487, in get_from_cache
    raise ConnectionError("Couldn't reach {}".format(url))
ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/master/datasets/boolq/boolq.py
el/0 I1217 01:11:33.898849 354161 main shadow.py:210 Current job status: FINISHED
```

Comments:
This happen quite often when they are too many concurrent requests to github.

i can understand it’s a bit cumbersome to handle on the user side. Maybe we should try a few times in the lib (eg with timeout) before failing, what do you think @lhoestq ?

Yes currently there's no retry afaik. We should add retries

Retries were added in #1603 :) 
It will be available in the next release

Hi @lhoestq thank you for the modification, I will use`script_version="master"` for now  :), to my experience, also setting timeout to a larger number like 3*60 which I normally use helps a lot on this.
Title: Access to key in DatasetDict map
Author: ZhaofengWu (NONE)
State: closed
Created: 2020-12-17T07:02:20Z
Labels: enhancement

It is possible that we want to do different things in the `map` function (and possibly other functions too) of a `DatasetDict`, depending on the key. I understand that `DatasetDict.map` is a really thin wrapper of `Dataset.map`, so it is easy to directly implement this functionality in the client code. Still, it'd be nice if there can be a flag, similar to `with_indices`, that allows the callable to know the key inside `DatasetDict`.

Comments:
Indeed that would be cool

Also FYI right now the easiest way to do this is
```python
dataset_dict["train"] = dataset_dict["train"].map(my_transform_for_the_train_set)
dataset_dict["test"] = dataset_dict["test"].map(my_transform_for_the_test_set)
```

I don't feel like adding an extra param for this simple usage makes sense, considering how many args `map` already has. 

(Feel free to re-open this issue if you don't agree with me)

I still think this is useful, since it's common that the data processing is different for training/dev/testing. And I don't know if the fact that `map` currently takes many arguments is a good reason not to support a useful feature.
Title: IWSLT-17 Link Broken
Author: ZhaofengWu (NONE)
State: closed
Created: 2020-12-17T00:46:42Z
Labels: duplicate

dataset bug

```
FileNotFoundError: Couldn't find file at https://wit3.fbk.eu/archive/2017-01-trnmted//texts/DeEnItNlRo/DeEnItNlRo/DeEnItNlRo-DeEnItNlRo.tgz
```

Comments:
Sorry, this is a duplicate of #1287. Not sure why it didn't come up when I searched `iwslt` in the issues list.

Closing this since its a duplicate
Title: Add helper to resolve namespace collision
Author: jramapuram (NONE)
State: closed
Created: 2020-12-16T20:17:24Z

Many projects use a module called `datasets`, however this is incompatible with huggingface datasets. It would be great if there if there was some helper or similar function to resolve such a common conflict.

Comments:
Do you have an example?

I was thinking about using something like [importlib](https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly) to over-ride the collision. 

**Reason requested**: I use the [following template](https://github.com/jramapuram/ml_base/) repo where I house all my datasets as a submodule.

Alternatively huggingface could consider some submodule type structure like:

`import huggingface.datasets`
`import huggingface.transformers`

`datasets` is a very common module in ML and should be an end-user decision and not scope all of python ¯\_(ツ)_/¯ 


That's a interesting option indeed. We'll think about it.

It also wasn't initially obvious to me that the samples which contain `import datasets` were in fact importing a huggingface library (in fact all the huggingface imports are very generic - transformers, tokenizers, datasets...)
Title: Update doc2dial.py
Author: songfeng (CONTRIBUTOR)
State: closed
Created: 2020-12-16T18:50:56Z

Added data loader for machine reading comprehension tasks proposed in the Doc2Dial EMNLP 2020 paper.

Comments:
Thanks for adding the `doc2dial_rc` config :) 

It looks like you're missing the dummy data for this config though. Could you add them please ?
Also to fix the CI you'll need to format the code with `make style`
Title: added irc disentangle dataset
Author: dhruvjoshi1998 (CONTRIBUTOR)
State: closed
Created: 2020-12-16T13:25:58Z

added irc disentanglement dataset

Comments:
@lhoestq sorry, this was the only way I was able to fix the pull request 

@lhoestq Thank you for the feedback. I wondering whether I should be passing an 'id' field in the dictionary since the 'connections' reference the 'id' of the linked messages. This 'id' would just be the same as the id_ that is in the yielded tuple.

Yes indeed it would be cool to have the ids in the dictionary. This way the dataset can be shuffled and all without losing information about the connections. Can you add it if you don't mind ?

Thanks :) could you also add the ids in the dictionary since they're useful for the connection links ?

Thanks !
Also it looks like the dummy_data.zip were regenerated and are now back to being too big (300KB each).
Can you reduce their sizes ? You can actually just revert to the ones you had before the last commit
Title: FileNotFoundError for `amazon_polarity`
Author: phtephanx (NONE)
State: closed
Created: 2020-12-16T12:51:05Z

Version: `datasets==v1.1.3`

### Reproduction
```python
from datasets import load_dataset
data = load_dataset("amazon_polarity")
```
crashes with
```bash
FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/amazon_polarity/amazon_polarity.py
```
and 
```bash
FileNotFoundError: Couldn't find file at https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/amazon_polarity/amazon_polarity.py
```
and
```bash
FileNotFoundError: Couldn't find file locally at amazon_polarity/amazon_polarity.py, or remotely at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/amazon_polarity/amazon_polarity.py or https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/amazon_polarity/amazon_polarity.py
```

Comments:
Hi @phtephanx , the `amazon_polarity` dataset has not been released yet. It will be available in the coming soon v2of `datasets` :) 

You can still access it now if you want, but you will need to install datasets via the master branch:
`pip install git+https://github.com/huggingface/datasets.git@master`
Title: Adding CLIMATE-FEVER dataset
Author: tdiggelm (CONTRIBUTOR)
State: closed
Created: 2020-12-15T16:49:22Z

This PR request the addition of the CLIMATE-FEVER dataset:
A dataset adopting the FEVER methodology that consists of 1,535 real-world claims regarding climate-change collected on the internet. Each claim is accompanied by five manually annotated evidence sentences retrieved from the English Wikipedia that support, refute or do not give enough information to validate the claim totalling in 7,675 claim-evidence pairs. The dataset features challenging claims that relate multiple facets and disputed cases of claims where both supporting and refuting evidence are present.

More information can be found at:
- Homepage: <http://climatefever.ai>
- Paper: <https://arxiv.org/abs/2012.00614>

Comments:
I `git rebase`ed my branch to `upstream/master` as suggested in point 7 of <https://huggingface.co/docs/datasets/share_dataset.html> and subsequently used `git pull` to be able to push to my remote branch. However, I think this messed up the history.

Please let me know if I should create a clean new PR with my changes.

Update: I also fixed the dataset name in the Dataset Card.

Dear @SBrandeis , @lhoestq . I am not sure how to fix the PR with respect to the additional files that are currently included in the commits. Could you provide me with an example? Otherwise I would be happy to close/re-open another PR. Please let me know if anything is missing for the review.

Hi @tdiggelm, thanks for the contribution! This dataset is really awesome.
I believe creating a new branch from master and opening a new PR with your changes is the simplest option since no review has been done yet. Feel free to ping us when it's done.

> Hi @tdiggelm, thanks for the contribution! This dataset is really awesome.
> I believe creating a new branch from master and opening a new PR with your changes is the simplest option since no review has been done yet. Feel free to ping us when it's done.

Thank you very much for your quick reply! Will do ASAP and ping you when done.

closing in favor of #1623
Title: Add comet metric
Author: ricardorei (CONTRIBUTOR)
State: closed
Created: 2020-12-15T08:56:00Z

Hey! I decided to add our new Crosslingual Optimized Metric for Evaluation of Translation (COMET) to the list of the available metrics.

COMET was [presented at EMNLP20](https://www.aclweb.org/anthology/2020.emnlp-main.213/) and it is the highest performing metric, so far, on the WMT19 benchmark.

We also participated in the [WMT20 Metrics shared task ](http://www.statmt.org/wmt20/pdf/2020.wmt-1.101.pdf) where once again COMET was validated as a top-performing metric. 


I hope that this metric will help researcher's and industry workers to better validate their MT systems in the future 🤗 !

Cheers,
Ricardo

Comments:
I also thought a bit about the fact that "sources" can't be added to the batch.. but changing that would require a lot more changes. And I agree that the idea of adding them as part of the references is not ideal. Conceptually they are not references.

I would keep it like this for now.. And in the future, work on a more consistent batch interface.
Title: Documentation for loading CSV datasets misleads the user
Author: onurgu (CONTRIBUTOR)
State: closed
Created: 2020-12-14T19:04:37Z

Documentation for loading CSV datasets misleads the user into thinking setting `quote_char' to False will disable quoting.

There are two problems here:
  i) `quote_char' is misspelled, must be `quotechar'
  ii) the documentation should mention `quoting'

Comments:
Title: Added the dataset clickbait_news_bg
Author: tsvm (CONTRIBUTOR)
State: closed
Created: 2020-12-14T17:03:00Z

There was a problem with my [previous PR 1445](https://github.com/huggingface/datasets/pull/1445) after rebasing, so I'm copying the dataset code into a new branch and submitting a new PR.

Comments:
Hi @tsvm  Great work! 
Since you have raised a clean PR could you close the earlier one -   #1445 ? 


> Hi @tsvm Great work!
> Since you have raised a clean PR could you close the earlier one - #1445 ?

Done.
Title: Add Microsoft Research Sequential Question Answering (SQA) Dataset
Author: mattbui (CONTRIBUTOR)
State: closed
Created: 2020-12-14T12:02:30Z

For more information: https://msropendata.com/datasets/b25190ed-0f59-47b1-9211-5962858142c2

Comments:
I proposed something a few weeks ago in #898 (un-merged) but I think that the way that @mattbui added the dataset in the present PR is smarter and simpler should replace my PR #898.

(Narrator voice: *And it was around that time that Thomas realized that the community was now a lot smarter than him and he should hand-over the library he had started with @lhoestq to the community and stop pretending he knew everything about it.*)
Title: Create README.md
Author: ManuelFay (CONTRIBUTOR)
State: closed
Created: 2020-12-14T11:40:23Z


Comments:
@ManuelFay thanks you so much for adding a dataset card, this is such a cool contribution!

This looks like it uses an old template for the card we've moved things around a bit and we have an app you should be using to get the tags and the structure of the Data Fields paragraph :) Would you mind moving your text to the newer format (we're also asking contributors to keep the full template structure, even if some sections still have [More Information Needed] for the time being)

Here's the link to the instructions:
https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card

Out of curiosity, what was your landing point for filling out the card? Did you follow the "Update on Github" when navigating the datasets? Trying to make the instructions as clear as possible :) 

@yjernite 

Perfect, I'll follow the instructions when I have a bit more time tomorrow ! I was actually browsing the new contributions after the dataset sprint and realized most of the "old" datasets were not tagged, so I just copied and pasted the readme from another dataset and was not aware there was precise instructions... Will fix !

BTW, amazing job with the retriBert work, I used the contrastive + in-batch negative quite a bit for various projects. Probably neither the time nor place to talk about that but I was curious as to why, in your original work, you prefered using a simple projection in the last layer to differentiate the question vs answer embedding, rather than allowing for bias in the dense layer or even just to fine-tune 2 different embedders for question + answer ? 

Cool! Looking forward to the next version!

Quick answer for retriBERT is that I expected a simple projection to generalize better and more importantly only having to store the gradients for the proj means training with larger batches :) If you want to keep chatting about it, feel free to send me an email!

Hi @ManuelFay ! 
If you're still interested in completing the FQuAD dataset card, note that we've generated one that is pre-filled.
Therefore feel free to complete it with the content you already have in your README.md.
This would be awesome ! And thanks again for your contribution :)

Yo @lhoestq , just not sure about the tag table at the top, I used @yjernite eli5 template so hope it's okay ! Also want to signal the streamlit app for dataset tagging has a weird behavior with the size categories when filling in the form. 

Thanks to you guys for doing that and sorry about the time it took, i completely forgot about it !
Title: adding tmu-gfm-dataset
Author: forest1988 (CONTRIBUTOR)
State: closed
Created: 2020-12-14T09:45:30Z

Adding TMU-GFM-Dataset for Grammatical Error Correction.

https://github.com/tmu-nlp/TMU-GFM-Dataset

A dataset for GEC metrics with manual evaluations of grammaticality, fluency, and meaning preservation for system outputs.
More detail about the creation of the dataset can be found in [Yoshimura et al. (2020)](https://www.aclweb.org/anthology/2020.coling-main.573.pdf).

Comments:
@lhoestq Thank you for your code review! I think I could do the necessary corrections. Could you please check it again when you have time?

Thank you for merging!
Title: Add dataset COrpus of Urdu News TExt Reuse (COUNTER).
Author: arkhalid (CONTRIBUTOR)
State: closed
Created: 2020-12-14T06:32:48Z


Comments:
Just a small revision from simon's review: 20KB for the dummy_data.zip is fine, you can keep them this way.

Also the CI is failing because of an error `tests/test_file_utils.py::TempSeedTest::test_tensorflow` that is not related to your dataset and is fixed on master. You can ignore it

merging since the Ci is fixed on master
Title: Lama
Author: huu4ontocord (CONTRIBUTOR)
State: closed
Created: 2020-12-14T03:27:10Z

This the LAMA dataset for probing facts and common sense from language models. 

See https://github.com/facebookresearch/LAMA for more details.

Comments:
Let me know why the pyarrow test is failing. For one of the config "trex", I had to load an initial datafile for a dictionary which is used to augment the rest of the datasets. In the dummy data, the dictionary file was truncated so I had to fudge that. I'm not sure if that is the issue.


@ontocord it just needs a rerun and it will be good to go.

THanks @tanmoyio. How do I do a rerun?

@ontocord contributor can’t rerun it, the maintainers will rerun it, it may take lil bit of time as there are so many PRs left to be reviewed and merged 

@lhoestq not sure why it is failing. i've made all modifications. 

merging since the CI is fixed on master
Title: Adding Igbo NER data 
Author: purvimisal (CONTRIBUTOR)
State: closed
Created: 2020-12-13T23:52:11Z

This PR adds the Igbo NER dataset.
Data: https://github.com/IgnatiusEzeani/IGBONLP/tree/master/ig_ner

Comments:
Thanks for the PR @purvimisal. 

Few comments below. 

Hi, @lhoestq Thank you for the review. I have made all the changes. PTAL! 

the CI error is not related to your dataset, merging
Title: Added Opus TedTalks
Author: rkc007 (CONTRIBUTOR)
State: closed
Created: 2020-12-13T22:29:33Z

Dataset : http://opus.nlpl.eu/TedTalks.php

Comments:
@lhoestq I saw some common changes you made on the other PR's (Similar Opus Datasets). I fixed those changes here. Can you please review it once ? 
Thanks.

merging since the CI is fixed on master
Title: Opus CAPES added
Author: rkc007 (CONTRIBUTOR)
State: closed
Created: 2020-12-13T22:11:34Z

Dataset : http://opus.nlpl.eu/CAPES.php

Comments:
@lhoestq I saw some common changes you made on the other PR's (Similar Opus Datasets). I fixed those changes here. Can you please review it once ? 
Thanks.

Hi @rkc007 , thanks for the contribution.
Unfortunately, the CAPES dataset has already been added here: #1307
I'm closing the PR 

@lhoestq FYI
Title: added air_dialogue
Author: skyprince999 (CONTRIBUTOR)
State: closed
Created: 2020-12-13T21:59:02Z

UPDATE2 (3797ce5): Updated for multi-configs 

UPDATE (7018082): manually created the dummy_datasets. All tests were cleared locally. Pushed it to origin/master

DRAFT VERSION (57fdb20):  (_no longer draft_)
Uploaded the air_dialogue database. 
dummy_data creation was failing in local, since the original downloaded file has some nested folders. Pushing it since the tests with real data was cleared. Will re-check & update via manually creating some dummy_data

Comments:
Title: Added OPUS ParaCrawl
Author: rkc007 (CONTRIBUTOR)
State: closed
Created: 2020-12-13T21:44:29Z

Dataset : http://opus.nlpl.eu/ParaCrawl.php

Comments:
@lhoestq I saw some common changes you made on the other PR's (Similar Opus Datasets). I fixed those changes here. Can you please review it once ? 
Thanks.

@rkc007 @lhoestq just noticed a dataset named para_crawl has been added a long time ago: #91.

They're not exactly the same so it's ok to have both.

Especially the `para_crawl` that already exists only uses the text from the ParaCrawl release 4.

Could you regenerate the dataset_infos.json @rkc007 please ?
It looks like it has some issues due to the dataset class name change

@SBrandeis Thank you for suggesting changes. I made the changes you suggested. 

@lhoestq I generated `dataset_infos.json` again. I ran both tests(Dummy & Real data) and it passed. Can you please review it again?

merging since the CI is fixed on master
Title: Monero
Author: iliemihai (CONTRIBUTOR)
State: closed
Created: 2020-12-13T19:56:48Z
Labels: dataset contribution

Biomedical Romanian dataset :)

Comments:
Hi @iliemihai  - you need to add the Readme file! Otherwise seems good. 
Also don't forget to run `make style` & `flake8 datasets` locally, from the datasets folder

@skyprince999 I will add the README.d for it. Thank you :D 

Thanks for your contribution, @iliemihai. Are you still interested in adding this dataset?

We are removing the dataset scripts from this GitHub repo and moving them to the Hugging Face Hub: https://huggingface.co/datasets

We would suggest you create this dataset there. Please, feel free to tell us if you need some help.
Title: Generics kb new branch
Author: bpatidar (CONTRIBUTOR)
State: closed
Created: 2020-12-13T19:33:10Z

Datasets need manual downloads. Have thus created dummy data as well. But pytest on real and dummy data are failing.
I have completed the readme , tags and other required things. I need to create the metadata json once tests get successful.
Opening a PR while working with Yacine Jernite to resolve my pytest issues.

Comments:
Title: Adding PolEval2019 Machine Translation Task dataset
Author: vrindaprabhu (CONTRIBUTOR)
State: closed
Created: 2020-12-13T17:50:03Z

Facing an error with pytest in training. Dummy data is passing.
README has to be updated.

Comments:
**NOTE:**

- Train and Dev: Manually downloaded (auto download is repeatedly giving `ConnectionError` for one of the files), Test: Auto Download
- Dummy test is passing
- The json file has been created with hard-coded paths for the manual downloads _(hardcoding has been removed from the final uploaded script)_
- datasets-cli is still **failing** . It is not picking the right directory for the config. For instance, my folder structure is as below:
 ```
            ~/Downloads/Data/
             |--- English-to-Polish
                   |--- (corresponding files) 
             |--- Russian-Polish
                   |--- (corresponding files) 
```

When ru-pl is selected, ideally it has to search in Russian-Polish folder, but it is searching in '/Downloads/Data/' folder and hence getting a FileNotFound error.

The command run is 
`python datasets-cli test datasets/poleval2019_mt/ --save_infos --all_configs --data_dir ~/Downloads/Data/
`


Hi !
Thanks for the changes :)

The only error left is the dummy data. Since we changed for standard downloads instead of manual downloads its structure changed. Fortunately you can auto-generate the dummy data with this command:

```
datasets-cli dummy_data ./datasets/poleval2019_mt --auto_generate --match_text_files "*"
```

Can you regenerate the dummy data using this command please ?

Thank you for the help @lhoestq !! I was generating the dummy dataset in a wrong way! That _--match_text_files "*"_ did the trick! Now all the tests have passed! :-)

Hi @vrindaprabhu ! Do you still have the Poleval2019 data files somewhere by any chance ? It appears the google drive URLs are not working anymore

Hi @lhoestq. Just checked. I do not have the backup of the data anywhere. It also appears that PolEval does not repeat its tasks, the data seem to have gone forever. Do you think I should try contacting the organizers for more info?

We tried already and they don't have the data anymore :(
Title: Added Wiki Summary Dataset
Author: tanmoyio (CONTRIBUTOR)
State: closed
Created: 2020-12-13T16:33:46Z

Wiki Summary: Dataset extracted from Persian Wikipedia into the form of articles and highlights.
Link: https://github.com/m3hrdadfi/wiki-summary

Comments:
@lhoestq why my tests are not running?

Maybe an issue with CircleCI, let me try to make them run

The CI error `tests/test_file_utils.py::TempSeedTest::test_tensorflow` is not related to this dataset and is fixed on master, you can ignore it

what I need to do now

Now the delimiter of the csv reader is fixed, thanks :) 

I just added a comment suggesting to try using actual URLS instead of a manual download if possible.
This would make things more convenient for the users. Can you try using the `dl_manager` to download the train/dev/test csv files instead of requiring manual download ?

Also pinging @m3hrdadfi , since I just noticed that there's already a dataset script that was created 3 weeks ago for this dataset here: https://github.com/m3hrdadfi/wiki-summary/tree/master/datasets/wiki_summary_persian

@lhoestq I am getting this error while generating the dummy data
![Screenshot (181)](https://user-images.githubusercontent.com/33005287/102628819-50a40080-4170-11eb-9e96-efce74b45ff4.png)


Can you try by adding the flag `--match_text_files "*"` ?

now it worked

@lhoestq pytest on dummy data passed, but on real data raising this issue
![Screenshot (196)](https://user-images.githubusercontent.com/33005287/102630784-fa848c80-4172-11eb-9f7e-e5a58dcf7abe.png)
how to resolve it


I see ! This is because the library did some verification to make sure it downloads the same files as in the first time you ran the `datasets-cli test` command with `--save_infos`. Since we're now downloading files, the verification fails. 

To fix that you just need to regenerate the dataset_infos.json file:
```
datasets-cli test ./datasets/wiki_summary --save_infos --all_configs --ignore_verifications
```

@lhoestq I have modified everything and It worked fine, dont know why it is not passing the tests 

Awesome thank you !

The CI error `tests/test_file_utils.py::TempSeedTest::test_tensorflow` is not related to your dataset and is fixed on master.
You can ignore it :) 

@lhoestq anything left to do ?

The dataset script is all good now ! The dummy data and the dataset_infos.json file are good too :) 

@lhoestq yay, thanks for helping me out , 

merging since the CI is fixed on master

@tanmoyio @lhoestq 

Thank you both!