noamaanMulla-03/datasets-issues huggingface dataset:
Allow several tags sets

Hi !

Currently we have three dataset cards : snli, cnn_dailymail and allocine.
For each one of those datasets a set of tag is defined. The set of tags contains fields like `multilinguality`, `task_ids`, `licenses` etc.

For certain datasets like `glue` for example, there exist several configurations: `sst2`, `mnli` etc. Therefore we should define one set of tags per configuration. However the current format used for tags only supports one set of tags per dataset.

In this PR I propose a simple change in the yaml format used for tags to allow for several sets of tags.

Let me know what you think, especially @julien-c let me know if it's good for you since it's going to be parsed by moon-landing


Closing since we don't need to update the tags of those three datasets (for each one of them there is only one tag set)
create our own DownloadManager

I tried to create our own - and way simpler - download manager, by replacing all the complicated stuff with our own `cached_path` solution.
With this implementation, I tried `dataset = nlp.load('squad')` and it seems to work fine.

For the implementation, what I did exactly:
- I copied the old download manager
- I removed all the dependences to the old `download` files
- I replaced all the download + extract calls by calls to `cached_path`
- I removed unused parameters (extract_dir, compute_stats) (maybe compute_stats could be re-added later if we want to compute stats...)
- I left some functions unimplemented for now. We will probably have to implement them because they are used by some datasets scripts (download_kaggle_data, iter_archive) or because we may need them at some point (download_checksums, _record_sizes_checksums)

Let me know if you think that this is going the right direction or if you have remarks.
Note: I didn't write any test yet as I wanted to read your remarks first


Looks great to me! 

The new download manager is ready. I removed the old folder and I fixed a few remaining dependencies.
I tested it on squad and a few others from the dataset folder and it works fine.

The only impact of these changes is that it breaks the `download_and_prepare` script that was used to register the checksums when we create a dataset, as the checksum logic is not implemented.

Let me know if you have remarks

Ok merged it (a bit fast for you to update the copyright, now I see that. but it's ok, we'll do a pass on these doc/copyright before releasing anyway)

Actually two additional things here @lhoestq (I merged too fast sorry, let's make a new PR for additional developments):
- I think we can remove some dependencies now (e.g. `promises`) in setup.py, can you have a look?
- also, I think we can remove the boto3 dependency like here: https://github.com/huggingface/transformers/pull/3968
[api request] API to obtain "dataset_module" dynamic path?

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is.

This is an awesome library. 

It seems like the dynamic module path in this library has broken some of hyperparameter tuning functionality: https://discuss.huggingface.co/t/using-hyperparameter-search-in-trainer/785/34

This is because Ray will spawn new processes, and each process will load modules by path. However, we need to explicitly inform Ray to load the right modules, or else it will error upon import. 

I'd like an API to obtain the dynamic paths. This will allow us to support this functionality in this awesome library while being future proof.

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

`datasets.get_dynamic_paths -> List[str]` will be sufficient for my use case.

By offering this API, we will be able to address the following issues (by patching the ray integration sufficiently):

https://github.com/huggingface/blog/issues/106
https://github.com/huggingface/transformers/issues/11565
https://discuss.huggingface.co/t/using-hyperparameter-search-in-trainer/785/34
https://discuss.huggingface.co/t/using-hyperparameter-search-in-trainer/785/35



Labels: name: enhancement

Hi @richardliaw, 

First, thanks for the compliments.

In relation with your request, currently, the dynamic modules path is obtained this way:
```python
from datasets.load import init_dynamic_modules, MODULE_NAME_FOR_DYNAMIC_MODULES

dynamic_modules_path = init_dynamic_modules(MODULE_NAME_FOR_DYNAMIC_MODULES)
```

Let me know if it is OK for you this way. 

I could set `MODULE_NAME_FOR_DYNAMIC_MODULES` as default value, so that you could instead obtain the path with:
```
dynamic_modules_path = datasets.load.init_dynamic_modules()
```

Hi @albertvillanova, the default value proposal seems great :) Looking forward to this!

I like the idea as well ! thanks @albertvillanova 

Hi @richardliaw, the feature is on the master branch and will be included in the next release in a couple of weeks.

awesome work @albertvillanova !
Bbaw egyptian

This is the "hieroglyph corpus" that I could unfortunately not contribute during the marathon. I re-extracted it again now, so that it is in the state as used in my paper (seee documentation). I hope it satiesfies your requirements and wish every scientist out their loads of fun deciphering a 5.000 years old language :-)


Hi @phiwi,

Thanks for contributing this nice dataset. If you have any blocking problem or question, do not hesitate to ask here. We are pleased to help you.

Could you please first synchronize with our master branch? From your branch `bbaw_egyptian`, type:
```
git fetch upstream master
git merge upstream/master
```

Thanks ! Can you check that you have `black==21.4b0` and run `make style` again ? This should fix the "check_code_quality" CI issue

Reformatted with black.

Hi @phiwi, there are still some minor problems in relation with the tags you used in the dataset card (README.md).

Here you can find the output of the metadata validator:
```
WARNING:root:❌ Failed to validate 'datasets/bbaw_egyptian/README.md':
Could not validate the metada, found the following errors:
* field 'size_categories':
	['100K<n<1000K'] are not registered tags for 'size_categories', reference at https://github.com/huggingface/datasets/tree/master/src/datasets/utils/resources/size_categories.json
* field 'task_ids':
	['machine translation'] are not registered tags for 'task_ids', reference at https://github.com/huggingface/datasets/tree/master/src/datasets/utils/resources/tasks.json
* field 'languages':
	['eg'] are not registered tags for 'languages', reference at https://github.com/huggingface/datasets/tree/master/src/datasets/utils/resources/languages.json

``` 

@albertvillanova corrected :-)

Thanks, @phiwi. Now all tests should pass green.

However, I think there is still an issue with the language code:
- the code for the Ancient Egyptian is not `ar-EG`
- there is no ISO 639-1 code for the Ancient Egyptian
- there is an ISO 639-2 code: `egy`; but this code will not pass the validation test because it is not in the list of valid codes

I am not sure what to do in this case... Maybe @lhoestq has an idea? Maybe adding the code to the list? https://github.com/huggingface/datasets/blob/master/src/datasets/utils/resources/languages.json

I have just checked that in the [list of valid codes](https://github.com/huggingface/datasets/blob/master/src/datasets/utils/resources/languages.json) there are already ISO 639-2 codes. Therefore, I would suggest you to add it to the list:
```
"egy": "Egyptian (Ancient)",
```
and change it in the dataset card.

Done.

Hope, everything is okay right now.
Enable auto-download for PAN-X / Wikiann domain in XTREME

This PR replaces the manual download of the `PAN-X.lang` domains with an auto-download from a Dropbox link provided by the Wikiann author. We also add the relevant dummy data for these domains.

While re-generating `dataset_infos.json` I ran into a `KeyError` in the `udpos.Arabic` domain so have included a fix for this as well.
Add SLR52, SLR53 and SLR54 to OpenSLR

Add large speech datasets for Sinhala, Bengali and Nepali.


Hi @lhoestq , I am not sure about the error message:
```
#!/bin/bash -eo pipefail
./scripts/datasets_metadata_validator.py
WARNING:root:❌ Failed to validate 'datasets/openslr/README.md':
__init__() got an unexpected keyword argument 'SLR32'
INFO:root:❌ Failed on 1 files.

Exited with code exit status 1
CircleCI received exit code 1 
```
Could you have a look please? Thanks.

Hi ! The error is unrelated to your PR and has been fixed on master
Next time feel free to merge master into your branch to fix the CI error ;)
load_dataset("timit_asr") gives back duplicates of just one sample text

## Describe the bug
When you look up on key ["train"] and then ['text'], you get back a list  with just one sentence duplicated 4620 times. Namely, the sentence "Would such an act of refusal be useful?". Similarly when you look up ['test'] and then ['text'], the list is one sentence repeated "The bungalow was pleasantly situated near the shore." 1680 times. 

I tried to work around the issue by downgrading to datasets version 1.3.0, inspired by [this post](https://www.gitmemory.com/issue/huggingface/datasets/2052/798904836) and removing the entire huggingface directory from ~/.cache, but I still get the same issue.  

## Steps to reproduce the bug
```python
from datasets import load_dataset
timit = load_dataset("timit_asr")
print(timit['train']['text'])
print(timit['test']['text'])
```

## Expected Result
Rows of diverse text, like how it is shown in the [wav2vec2.0 tutorial](https://colab.research.google.com/github/patrickvonplaten/notebooks/blob/master/Fine_tuning_Wav2Vec2_for_English_ASR.ipynb)
<img width="485" alt="Screen Shot 2021-05-05 at 9 09 57 AM" src="https://user-images.githubusercontent.com/33647474/117146094-d9b77f00-ad81-11eb-8306-f281850c127a.png">


## Actual results
Rows of repeated text.
<img width="319" alt="Screen Shot 2021-05-05 at 9 11 53 AM" src="https://user-images.githubusercontent.com/33647474/117146231-f8b61100-ad81-11eb-834a-fc10410b0c9c.png">


## Versions
- Datasets: 1.3.0
- Python: 3.9.1
- Platform: macOS-11.2.1-x86_64-i386-64bit}


Labels: name: bug

Upgrading datasets to version 1.6 fixes the issue

This bug was fixed in #1995. Upgrading the `datasets` should work! 

Thanks @ekeleshian for having reported.

I am closing this issue once that you updated `datasets`. Feel free to reopen it if the problem persists.
Add Topical-Chat

## Adding a Dataset
- **Name:** Topical-Chat
- **Description:** a knowledge-grounded human-human conversation dataset where the underlying knowledge spans 8 broad topics and conversation partners don’t have explicitly defined roles
- **Paper:** https://www.isca-speech.org/archive/Interspeech_2019/pdfs/3079.pdf
- **Data:** https://github.com/alexa/Topical-Chat
- **Motivation:** Good quality, knowledge-grounded dataset that spans a broad range of topics

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Labels: name: dataset request
Add SubjQA dataset

Hello datasetters 🙂!

Here's an interesting dataset about extractive question-answering on _subjective_ product / restaurant reviews. It's quite challenging for models fine-tuned on SQuAD and provides a nice example of domain adaptation (i.e. fine-tuning a SQuAD model on this domain gives better performance).

I found a bug in the start/end indices that I've proposed a fix for here: https://github.com/megagonlabs/SubjQA/pull/2

Unfortunately, the dataset creators are unresponsive, so for now I am using my fork as the source. Will update the URL if/when the creators respond.


I'm not sure why the windows test fails, but looking at the logs it looks like some caching issue on one of the metrics ... maybe re-run and 🤞 ?

Hi @lewtun, thanks for adding this dataset!

If the dataset is going to be referenced heavily, I think it's worth spending some time to make the dataset card really great :) To start, the information that is currently in the `Data collection` paragraph should probably be organized in the `Dataset Creation` section.

Here's a link to the [relevant section of the guide](https://github.com/huggingface/datasets/blob/master/templates/README_guide.md#dataset-creation), let me know if you have any questions!

> If the dataset is going to be referenced heavily, I think it's worth spending some time to make the dataset card really great :) To start, the information that is currently in the `Data collection` paragraph should probably be organized in the `Dataset Creation` section.

great idea @yjernite! i've added some extra information / moved things as you suggest and will wrap up the rest tomorrow :)

hi @yjernite and @lhoestq, i've fleshed out the dataset card and think this is now ready for another round of review!
Index error in Dataset.map

The following code, if executed on master, raises an IndexError (due to overflow):
```python
>>> from datasets import *
>>> d = load_dataset("bookcorpus", split="train")
Reusing dataset bookcorpus (C:\Users\Mario\.cache\huggingface\datasets\bookcorpus\plain_text\1.0.0\44662c4a114441c35200992bea923b170e6f13f2f0beb7c14e43759cec498700)
2021-05-08 21:23:46.859818: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll
>>> d.map(lambda ex: ex)
  0%|▎                                                                    | 289430/74004228 [00:13<58:41, 20935.33ex/s]c:\users\mario\desktop\projects\datasets-1\src\datasets\table.py:84: RuntimeWarning: overflow encountered in int_scalars
  k = i + ((j - i) * (x - arr[i]) // (arr[j] - arr[i]))
  0%|▎                                                                    | 290162/74004228 [00:13<59:11, 20757.23ex/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\users\mario\desktop\projects\datasets-1\src\datasets\arrow_dataset.py", line 1498, in map
    new_fingerprint=new_fingerprint,
  File "c:\users\mario\desktop\projects\datasets-1\src\datasets\arrow_dataset.py", line 174, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "c:\users\mario\desktop\projects\datasets-1\src\datasets\fingerprint.py", line 340, in wrapper
    out = func(self, *args, **kwargs)
  File "c:\users\mario\desktop\projects\datasets-1\src\datasets\arrow_dataset.py", line 1799, in _map_single
    for i, example in enumerate(pbar):
  File "C:\Users\Mario\Anaconda3\envs\hf-datasets\lib\site-packages\tqdm\std.py", line 1133, in __iter__
    for obj in iterable:
  File "c:\users\mario\desktop\projects\datasets-1\src\datasets\arrow_dataset.py", line 1145, in __iter__
    format_kwargs=format_kwargs,
  File "c:\users\mario\desktop\projects\datasets-1\src\datasets\arrow_dataset.py", line 1337, in _getitem
    pa_subtable = query_table(self._data, key, indices=self._indices if self._indices is not None else None)
  File "c:\users\mario\desktop\projects\datasets-1\src\datasets\formatting\formatting.py", line 368, in query_table
    pa_subtable = _query_table(table, key)
  File "c:\users\mario\desktop\projects\datasets-1\src\datasets\formatting\formatting.py", line 79, in _query_table
    return table.fast_slice(key % table.num_rows, 1)
  File "c:\users\mario\desktop\projects\datasets-1\src\datasets\table.py", line 128, in fast_slice
    i = _interpolation_search(self._offsets, offset)
  File "c:\users\mario\desktop\projects\datasets-1\src\datasets\table.py", line 91, in _interpolation_search
    raise IndexError(f"Invalid query '{x}' for size {arr[-1] if len(arr) else 'none'}.")
IndexError: Invalid query '290162' for size 74004228.
```
Tested on Windows, can run on Linux if needed.

EDIT:
It seems like for this to happen, the default NumPy dtype has to be np.int32.

Labels: name: bug
Fix overflow issue in interpolation search

Fixes #2335 

More info about this error can be found [here](https://stackoverflow.com/questions/53239890/why-do-i-keep-getting-this-error-runtimewarning-overflow-encountered-in-int-sc/53240100). 


~~Seems like the CI failure is unrelated to this PR~~ (fixed with the merge). 

@lhoestq Can you please verify that everything is OK in terms of speed? Another solution is to change the offsets array dtype to np.int64 (but this doesn't scale in theory compared to Python integer which is unbound). I'm not sure why on my 64-bit machine the default numpy dtype is np.int32 tho.

Hi ! Thanks for the fix.
Unfortunately in terms of speed this is not acceptable :/
The `get_batch_of_1024_random_rows` metric or the `benchmark_getitem_100B ` benchmark is almost at 1sec instead of a few milliseconds.

Would it be possible to avoid the overflow by simply passing `dtype=np.int64` to `np.cumsum` ?
On windows machines the default is int32 unfortunately so we have to force the dtype to be int64



Yes, casting the array to np.int64 should work as well. Another option would be to cast the array elements (`arr[i], arr[j]`) in interpolation search to Python integers (bound only with memory) before multiplication (the error stems from this part: `(j - i) * (x - arr[i])`) when working with big values. But for now, the first option is OK for the sake of simplicity.
NonMatchingChecksumError for web_of_science dataset

NonMatchingChecksumError when trying to download the web_of_science dataset. 

>NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://data.mendeley.com/datasets/9rw3vkcfy4/6/files/c9ea673d-5542-44c0-ab7b-f1311f7d61df/WebOfScience.zip?dl=1']

Setting `ignore_verfications=True` results in OSError.

>OSError: Cannot find data file. 
Original error:
[Errno 20] Not a directory: '/root/.cache/huggingface/datasets/downloads/37ab2c42f50d553c1d0ea432baca3e9e11fedea4aeec63a81e6b7e25dd10d4e7/WOS5736/X.txt'

```python
dataset = load_dataset('web_of_science', 'WOS5736')
```
There are 3 data instances and they all don't work. 'WOS5736', 'WOS11967', 'WOS46985'

datasets 1.6.2
python 3.7.10
Ubuntu 18.04.5 LTS

Labels: name: bug

I've raised a PR for this. Should work with `dataset = load_dataset("web_of_science", "WOS11967", ignore_verifications=True)`once it gets merged into the main branch. Thanks for reporting this!
Datasets cli improvements

This PR:
* replaces the code from the `bug_report.md` that was used to get relevant system info with a dedicated command (a more elegant approach than copy-pasting the code IMO)
* removes the `download` command (copied from the transformers repo?)
* adds missing help messages to the cli commands






Additionally, I've deleted the points that are not very relevant for this repo (I guess the deleted points originate from the transformers repo). With this change, running `datasets-cli` is identical to copy-pasting the code from `bug_report.md`, but is more elegant because it doesn't require launching the REPL and copy-pasting the code.
Added the HLGD dataset

Added the Headline Grouping Dataset (HLGD), from the NAACL2021 paper: News Headline Grouping as a Challenging NLU Task
Dataset Link: https://github.com/tingofurro/headline_grouping
Paper link: https://people.eecs.berkeley.edu/~phillab/pdfs/NAACL2021_HLG.pdf


Is there anything else needed from my end?

Thanks Bhavitvya and Quentin, this was very streamlined!
Problem downloading GEM wiki_auto_asset_turk dataset

@yjernite 

### Summary

I am currently working on the GEM datasets and do not manage to download the wiki_auto_asset_turk data, whereas all other datasets download well with the same code.

### Steps to reproduce
Code snippet:

from datasets import load_dataset
#dataset = load_dataset('gem', 'web_nlg_en')
dataset = load_dataset('gem', 'wiki_auto_asset_turk')

```

**Expected behavior:**

I expect the dataset to start downloading (download bar appears and progresses toward 100%)

**Actual behavior:**
Instead of seeing the download bar appearing, nothing happens; the following appears in the console as expected, but nothing more:

Downloading: 36.6kB [00:00, 37.2MB/s]
Downloading: 41.7kB [00:00, ?B/s]
Downloading and preparing dataset gem/wiki_auto_asset_turk (download: 121.37 MiB, generated: 145.69 MiB, post-processed: Unknown size, total: 267.07 MiB) to C:\Users\sfmil\.cache\huggingface\datasets\gem\wiki_auto_asset_turk\1.0.0\f252756d7f1b8f019aac71a1623b2950acfe10d25d956668ac4eae4e93c58b8d...

### Is this a regression?
No, it was the first time I was trying to download this dataset (same for the other ones).

### Debug info
- Python version: Python 3.8.2
- OS version: Windows 10 Family


Hi,

sadly I can't replicate the problem on my Windows machine. Try to update the library to the newest version with:
```bash
pip install git+https://github.com/huggingface/datasets
``` 

Thanks for the answer! I updated the library but unfortunately it didn't solve the problem.

Is there an error message ?
What stacktrace do you get if you interrupt the execution of the program while downloading ?

Sorry for the long time since my last comment, I tried again and don't seem to have the problem anymore, thanks for your support!

Great ! I'm closing the issue then. Feel free to re-open if you experience this issue again
normalized TOCs and titles in data cards

I started fixing some of the READMEs that were failing the tests introduced by @gchhablani but then realized that there were some consistent differences between earlier and newer versions of some of the titles (e.g. Data Splits vs Data Splits Sample Size, Supported Tasks vs Supported Tasks and Leaderboards). We also had different versions of the Table of Content

This PR normalizes all of them to the newer version


Oh right! I'd be in favor of still having the same TOC across the board, we can either leave it as is or add a `[More Info Needed]` `Contributions` Section wherever it's currently missing, wdyt?

(I thought those were programmatically updated based on git history :D )

Merging for now to avoid conflict since there are so many changes but let's figure out the contributions section next ;)
SNLI dataset has labels of -1 

There are a number of rows with a label of -1 in the SNLI dataset. The dataset descriptions [here](https://nlp.stanford.edu/projects/snli/) and [here](https://github.com/huggingface/datasets/tree/master/datasets/snli) don't list  -1 as a label possibility, and neither does the dataset viewer. As examples, see index 107 or 124 of the test set.

It isn't clear what these labels mean. I found a [line of code](https://github.com/huggingface/datasets/blob/80e59ef178d3bb2090d091bc32315c655eb0633d/datasets/snli/snli.py#L94) that seems to put them in but it seems still unclear why they are there. The current workaround is to just drop the rows from any model being trained. 

Perhaps the documentation should be updated.


Hi @puzzler10, 
Those examples where `gold_label` field was empty, -1 label was alloted to it. In order to remove it you can filter the samples from train/val/test splits. Here's how you can drop those rows from the dataset:
`dataset = load_dataset("snli")`
`dataset_test_filter = dataset['test'].filter(lambda example: example['label'] != -1)`

I agree it should have been mentioned in the documentation. I'll raise a PR regarding the same. Thanks for pointing out!
simpllify faiss index save

Fixes #2350

In some cases, Faiss GPU index objects do not have neither "device" nor "getDevice". Possibly this happens when some part of the index is computed on CPU.

In particular, this would happen with the index `OPQ16_128,IVF512,PQ32` (issue #2350). I did check it, but it is likely that `OPQ` or `PQ` transforms cause it.

I propose, instead of using the index object to get the device, to infer it form the `FaissIndex.device` field as it is done in `.add_vectors`. Here we assume that `.device` always corresponds to the index placement and it seems reasonable.