konrud/llm_course_chapter5_own_dataset_github_issues huggingface dataset:
Adding Roman Urdu Hate Speech dataset

This Pull request will add the Roman Urdu Hate speech Dataset. 

@lhoestq can you review when you have some time? Also were the previous CI fails due to the Google Drive tests which were excluded by #3982 ?

> were the previous CI fails due to the Google Drive tests which were excluded by https://github.com/huggingface/datasets/pull/3982 ?

Yes exactly, merging `master` into your branch fixed the CI ;)

_The documentation is not available anymore as the PR was closed or merged._
Cannot preview cnn_dailymail dataset

## Dataset viewer issue for '*cnn_dailymail*'

**Link:** https://huggingface.co/datasets/cnn_dailymail

*short description of the issue*

Am I the one who added this dataset ? Yes-No


I guess the cache got corrupted due to a previous issue with Google Drive service.

The cache should be regenerated, e.g. by passing `download_mode="force_redownload"`.

CC: @severo 

Note that the dataset preview uses its own cache, not `datasets`' cache. So `download_mode="force_redownload"` doesn't help. But yes indeed the cache must be refreshed.

The CNN Dailymail dataste is currently hosted on Google Drive, which is an unreliable host and we've had many issues with it. Unless we found another most reliable host for the data, we will keep running into issues from time to time.

At Hugging Face we're not allowed to host the CNN Dailymail data by ourselves AFAIK

Yes @lhoestq, I didn't explain myself well: my previous message was addressed to @severo. 

I remove the tag dataset-viewer, since it's more an issue with the hosting on Google Drive

Sounds good. I was looking for another host of this dataset but couldn't find any (yet)

It seems like the issue is with the streaming mode, not with the hosting:

```python
>>> import datasets
>>> dataset = datasets.load_dataset('cnn_dailymail', name="3.0.0", split="train", streaming=True, download_mode="force_redownload")
Downloading builder script: 9.35kB [00:00, 10.2MB/s]
Downloading metadata: 9.50kB [00:00, 12.2MB/s]
>>> len(list(dataset))
0
>>> dataset = datasets.load_dataset('cnn_dailymail', name="3.0.0", split="train", streaming=False)
Reusing dataset cnn_dailymail (/home/slesage/.cache/huggingface/datasets/cnn_dailymail/3.0.0/3.0.0/3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234)
>>> len(dataset)
287113
```

Note, in particular, that the streaming mode is failing silently, returning 0 row while I would have expected an exception instead. The result is that the dataset viewer shows `No data` instead of a detailed error.

<img width="1511" alt="Capture d’écran 2022-04-12 à 11 50 46" src="https://user-images.githubusercontent.com/1676121/162935341-d50f1e73-d053-41d4-917f-e79708a0ca23.png">


Well this is because the host (Google Drive) returns a document that is not the actual data, but an error page

Do you think that `datasets` should detect this anyway and throw an exception?

Yes it definitely should ! I don't have the bandwidth to work on this right now though

Indeed, streaming was not supported: tgz archives were not properly iterated.

I've opened a PR to support streaming.

However, keep in mind that Google Drive will keep generating issues from time to time, like 403,...
Cannot preview 'indonesian-nlp/eli5_id' dataset

## Dataset viewer issue for '*indonesian-nlp/eli5_id*'

**Link:** https://huggingface.co/datasets/indonesian-nlp/eli5_id

I can not see the dataset preview.
```
Server Error
Status code:   400
Exception:     Status400Error
Message:       Not found. Maybe the cache is missing, or maybe the dataset does not exist.
```
Am I the one who added this dataset ? Yes


Hi @cahya-wirawan, thanks for reporting.

Your dataset is working OK in streaming mode:
```python
In [1]: from datasets import load_dataset
   ...: ds = load_dataset("indonesian-nlp/eli5_id", split="train", streaming=True)
   ...: item = next(iter(ds))
   ...: item
Using custom data configuration indonesian-nlp--eli5_id-9fe728a7e760fb7b

Out[1]: 
{'q_id': '1oy5tc',
 'title': 'dalam sepak bola apa gunanya menyia-nyiakan dua permainan pertama dengan terburu-buru - di tengah - bukan permainan terburu-buru biasa saya mendapatkannya',
 'selftext': '',
 'document': '',
 'subreddit': 'explainlikeimfive',
 'answers': {'a_id': ['ccwtgnz', 'ccwtmho', 'ccwt946', 'ccwvj0u'],
  'text': ['Jaga pertahanan tetap jujur, rasakan operan terburu-buru, buka permainan yang lewat. Pelanggaran yang terlalu satu dimensi akan gagal. Dan mereka yang bergegas ke tengah kadang-kadang dapat dibuka lebar-lebar untuk ukuran yard yang besar.',
   'Jika Anda melempar bola sepanjang waktu, maka pertahanan akan beradaptasi untuk selalu menutupi umpan. Dengan melakukan permainan lari sederhana sesekali, Anda memaksa pertahanan untuk tetap dekat dan menjaga dari lari. Terkadang, pelanggaran dapat membuat pertahanan lengah dengan berpura-pura berlari dan membebaskan penerima mereka. Selain itu, Anda tidak perlu mendapatkan yard besar di setiap permainan. Terkadang, paling baik mendapatkan beberapa yard sekaligus. Selama Anda mendapatkan yang pertama, Anda dalam kondisi yang baik.',
   'Dalam kebanyakan kasus, O-Line seharusnya membuat lubang untuk dilalui kembali. Jika Anda menjalankan terlalu banyak permainan ke luar / melempar, pertahanan akan mengejar. Juga, 2 permainan 5 yard memberi Anda satu set down baru.',
   'Saya Anda tidak suka jenis drama itu, tonton CFL. Kami hanya mendapatkan 3 down sehingga Anda tidak bisa menyia-nyiakannya. Lebih banyak lagi yang lewat.'],
  'score': [3, 2, 2, 2]},
 'title_urls': {'url': []},
 'selftext_urls': {'url': []},
 'answers_urls': {'url': []}}
```
Therefore, it should be properly rendered in the previewer. Let me ping @severo to have a look at it.

Thanks @albertvillanova for checking it. Btw, I have another dataset indonesian-nlp/lfqa_id which has the same issue. However, this dataset is still private, is it the reason why the preview doesn't work?

Yes, preview is not supported on private datasets yet. We are working on that though...

Thanks for the confirmation 

Fixed. Thanks for your feedback.
[feat] Add TextVQA dataset

This would be the first classification-based vision-and-language dataset in the datasets library.

Currently, the dataset downloads everything you need beforehand. See the [paper](https://arxiv.org/abs/1904.08920) for more details.

Test Plan:
- Ran the full and the dummy data test locally

_The documentation is not available anymore as the PR was closed or merged._

Hey :) Have you had a chance to continue this PR ? Let me know if you have questions or if I can help

Hey @lhoestq, let me wrap this up soon. I will resolve your comments in next push.
Scores from Index at extra positions are not filtered out

If a FAISS index has fewer records than the requested number of top results (k), then it returns -1 in indices for the additional positions. The get_nearest_examples method only filters out the extra results from the dataset samples. It would be better to filter out extra scores too.

Reference: https://github.com/huggingface/datasets/blob/2.0.0/src/datasets/search.py#L693


Hi! Yes, that makes sense! Would you like to submit a PR to fix this?

Created PR https://github.com/huggingface/datasets/pull/3971
The dataset preview is not available for tdklab/Hebrew_Squad_v1.1 dataset

## Dataset viewer issue for 'tdklab/Hebrew_Squad_v1.1'

**Link:**  https://huggingface.co/api/datasets/tdklab/Hebrew_Squad_v1.1?full=true

The dataset preview is not available for this dataset.

Am I the one who added this dataset ? Yes

Hi @MatanBenChorin, thanks for reporting.

Please, take into account that the preview may take some time until it properly renders (we are working to reduce this time).

Maybe @severo can give more details on this.

Hi, 
Thank you

Thanks for reporting. We are looking at it and will give updates here.

I imagine the dataset has been moved to https://huggingface.co/datasets/tdklab/Hebrew_Squad_v1, which still has an issue:

```
Server Error

Status code:   400
Exception:     NameError
Message:       name 'HebrewSquad' is not defined
```

The issue is not related to the dataset viewer but to the loading script (cc @albertvillanova @lhoestq @mariosasko)

```python
>>> import datasets as ds
>>> hf_token = "hf_..." # <- required because the dataset is gated
>>> d = ds.load_dataset('tdklab/Hebrew_Squad_v1', use_auth_token=hf_token)
...
NameError: name 'HebrewSquad' is not defined
```

Yes indeed there is an error in [Hebrew_Squad_v1.py:L40](https://huggingface.co/datasets/tdklab/Hebrew_Squad_v1/blob/main/Hebrew_Squad_v1.py#L40)

Here is the fix @MatanBenChorin :

```diff
-         HebrewSquad(
+         HebrewSquadConfig(
```
Add ImageNet Sketch

## Adding a Dataset
- **Name:** ImageNet Sketch
- **Description:** ImageNet-Sketch is a dataset consisting of sketch-like images, that matches the ImageNet classification validation set in categories and scale.
- **Paper:** [Learning Robust Global Representations by Penalizing Local Predictive Power](https://arxiv.org/abs/1905.13549)
- **Data:** https://github.com/HaohanWang/ImageNet-Sketch
- **Motivation:** Allows for evaluating the robustness of vision models.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Can you assign this task to me? @nreimers @mariosasko 

Hi! Sure! Let us know if you need any pointers.
Google BLEU Metric Card

Add metric card for Google BLEU (GLEU) metric

One thing I noticed while writing this up is that, while this metric was made specifically to be better than BLEU at the sentence level instead of the corpus level, the current implementation only allows the calculation of the corpus-level statistic. I think changing this would be a good thing to put on the to do list for the future.

A few things that aren't clear for me:
- "Because it performs better on individual sentence pairs as compared to BLEU, Google BLEU has also been used in RL experiments." -- why is this the case? why would that make it more usable for RL? (also, you should put "Reinforcement Learning" explicitly, not just the acronym)
- (Minor issue) -- I put inputs before the first example code, I think that's clearer somehow

Otherwise, it looks great, good job @emibaylor !
BLEU metric card

Add BLEU metric card

_The documentation is not available anymore as the PR was closed or merged._

Some thoughts:
- For values, e.g. "Defaults to False", I would put False in code: `False`. Same for : "Defaults to `4`."
- I would put the following remark in "Limitations": 
> "BLEU's output is always a number between 0 and 1. This value indicates how similar the candidate text is to the reference texts, with values closer to 1 representing more similar texts. Few human translations will attain a score of 1, since this would indicate that the candidate is identical to one of the reference translations. For this reason, it is not necessary to attain a score of 1. Because there are more opportunities to match, adding additional reference translations will increase the BLEU score."

- Add some values from the original BLEU paper (https://aclanthology.org/P02-1040.pdf)
Fix comet metric

The COMET metric has been broken for a while since big breaking changes happened. We did not catch them in the CI because the slow test mocks the download_model function that was changed.

This PR fixes the metric, updates the download_model mock and updates the doctest.

_The documentation is not available anymore as the PR was closed or merged._

Finally I'm done updating the dependencies ^^'

cc @sashavor can you review my changes in the metric card please ?

Looks good to me! Just fixed a tiny typo :wink: 

Thanks !
reddit_tifu dataset: Checksums didn't match for dataset source files

## Describe the bug
When loading the reddit_tifu dataset, it throws the exception "Checksums didn't match for dataset source files"


## Steps to reproduce the bug
```python
import datasets
from datasets import load_dataset

print(datasets.__version__)

# load_dataset('billsum')

load_dataset('reddit_tifu', 'short')
```


## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.17.0
- Platform: mac os
- Python version: Python 3.7.6
- PyArrow version: 3.0.0


Hi @XingxingZhang, 

We have already fixed this. You should update `datasets` version to at least 1.18.4:
```shell
pip install -U datasets
```
And then force the redownload:
```python
load_dataset("...", download_mode="force_redownload")
```

Duplicate of:
- #3773

thanks @albertvillanova . by upgrading to 1.18.4 and using `load_dataset("...", download_mode="force_redownload")` fixed 
 the bug.

using the following as you suggested in another thread can also fixed the bug
```
pip install git+https://github.com/huggingface/datasets#egg=datasets
```


The latter solution (installing from GitHub) was proposed because the fix was not released yet. But last week we made the 1.18.4 patch release (with the fix), so no longer necessary to install from GitHub.

You can now install from PyPI, as usual:
```shell
pip install -U datasets
```
billsum dataset: Checksums didn't match for dataset source files:

## Describe the bug

When loading the `billsum` dataset, it throws the exception "Checksums didn't match for dataset source files"

```
  File "virtualenv_projects/codex/lib/python3.7/site-packages/datasets/utils/info_utils.py", line 40, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/uc?export=download&id=1g89WgFHMRbr4QrvA0ngh26PY081Nv3lx']
```

## Steps to reproduce the bug
```python
import datasets
from datasets import load_dataset

print(datasets.__version__)

load_dataset('billsum')
```


## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.17.0

- Platform: mac os
- Python version: Python 3.7.6
- PyArrow version: 3.0.0






Hi @XingxingZhang, thanks for reporting.

This was due to a change in Google Drive service:
- #3786 

We have already fixed it:
- #3787

You should update `datasets` version to at least 1.18.4:
```shell
pip install -U datasets
```
And then force the redownload:
```python
load_dataset("...", download_mode="force_redownload")
```

thanks @albertvillanova 

@albertvillanova 
YOU Said: pip install git+ https://github.com/huggingface/datasets.git Then set: dataset=load_dataset ("multinews", download_mode="force-redownload"). I changed the ’datautils‘ file according to this setting: traindata=load_dataset (path='wikitext ', name='wikitext-2-raw v1', split='train ', download_mode="force-redownload")
Testdata=load_dataset (path='wikitext ', name='wikitext-2-raw v1', split='test ', download_mode="force-redownload")
then the bug is
![image](https://github.com/huggingface/datasets/assets/149936473/ee956e8f-e6f1-46bf-b514-0a7a0a0e0e37)
![image](https://github.com/huggingface/datasets/assets/149936473/f1318686-942c-4341-a61d-9be7a4b5747a)
I have tried both versions
![image](https://github.com/huggingface/datasets/assets/149936473/d3f0d786-304b-4596-83c6-49c3cac58aad)
Missing languages in lvwerra/github-code dataset

Hi,

I'm working with the github-code dataset. First of all, thank you for creating this amazing dataset!
I've noticed that two languages are missing from the dataset: TypeScript and Scala. 
Looks like they're also omitted from the query you used to get the original code.
Are there any plans to add them in the future?

Thanks!

Thanks for contacting @Eytan-S.

I think @lvwerra could better answer this. 

That seems to be an oversight - I originally planned to include them in the dataset and for some reason they were in the list of languages but not in the query. Since there is an issue with the deduplication step I'll rerun the pipeline anyway and will double check the query.

Thanks for reporting this @Eytan-S!

Can confirm that the two languages are indeed missing from the dataset. Here are the file counts per language:
```Python
{'Assembly': 82847,
 'Batchfile': 236755,
 'C': 14127969,
 'C#': 6793439,
 'C++': 7368473,
 'CMake': 175076,
 'CSS': 1733625,
 'Dockerfile': 331966,
 'FORTRAN': 141963,
 'GO': 2259363,
 'Haskell': 340521,
 'HTML': 11165464,
 'Java': 19515696,
 'JavaScript': 11829024,
 'Julia': 58177,
 'Lua': 576279,
 'Makefile': 679338,
 'Markdown': 8454049,
 'PHP': 11181930,
 'Perl': 497490,
 'PowerShell': 136827,
 'Python': 7203553,
 'Ruby': 4479767,
 'Rust': 321765,
 'SQL': 655657,
 'Scala': 0,
 'Shell': 1382786,
 'TypeScript': 0,
 'TeX': 250764,
 'Visual Basic': 155371}
 ```

@Eytan-S check out v1.1 of the `github-code` dataset where issue should be fixed:

|    | Language     |File Count| Size (GB)|
|---:|:-------------|---------:|-------:|
|  0 | Java         | 19548190 | 107.7  |
|  1 | C            | 14143113 | 183.83 |
|  2 | JavaScript   | 11839883 |  87.82 |
|  3 | HTML         | 11178557 | 118.12 |
|  4 | PHP          | 11177610 |  61.41 |
|  5 | Markdown     |  8464626 |  23.09 |
|  6 | C++          |  7380520 |  87.73 |
|  7 | Python       |  7226626 |  52.03 |
|  8 | C#           |  6811652 |  36.83 |
|  9 | Ruby         |  4473331 |  10.95 |
| 10 | GO           |  2265436 |  19.28 |
| 11 | TypeScript   |  1940406 |  24.59 |
| 12 | CSS          |  1734406 |  22.67 |
| 13 | Shell        |  1385648 |   3.01 |
| 14 | Scala        |   835755 |   3.87 |
| 15 | Makefile     |   679430 |   2.92 |
| 16 | SQL          |   656671 |   5.67 |
| 17 | Lua          |   578554 |   2.81 |
| 18 | Perl         |   497949 |   4.7  |
| 19 | Dockerfile   |   366505 |   0.71 |
| 20 | Haskell      |   340623 |   1.85 |
| 21 | Rust         |   322431 |   2.68 |
| 22 | TeX          |   251015 |   2.15 |
| 23 | Batchfile    |   236945 |   0.7  |
| 24 | CMake        |   175282 |   0.54 |
| 25 | Visual Basic |   155652 |   1.91 |
| 26 | FORTRAN      |   142038 |   1.62 |
| 27 | PowerShell   |   136846 |   0.69 |
| 28 | Assembly     |    82905 |   0.78 |
| 29 | Julia        |    58317 |   0.29 |

Thanks @lvwerra.
Fix Wikipedia version and re-add tests

To keep backward compatibility when loading using "wikipedia" dataset ID (https://huggingface.co/datasets/wikipedia), we have created the pre-processed data for the same languages we were offering before, but with updated date "20220301":
- de
- en
- fr
- frr
- it
- simple

These pre-processed data can be accessed, e.g.:
```python
ds = load_dataset("wikipedia", "20220301.frr", split="train")
```

The next step will be to offer the pre-processed data for many other languages, but when loading using "wikimedia/wikipedia": https://huggingface.co/datasets/wikimedia/wikipedia

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3936). All of your documentation changes will be reflected on that endpoint.
Add align_labels_with_mapping docs

This PR documents the `align_labels_with_mapping` function to ensure predicted labels are aligned with the dataset, or to assign a different mapping of labels to ids (requested by @mariosasko 🎉 ).

For this specific code sample, the current dataset has a `mixed` label that the original [dataset](https://huggingface.co/datasets/poem_sentiment#data-fields) didn't. Is there a way to remove this label so it is completely aligned with the original dataset mappings? Otherwise, I'll just leave it as it is.

_The documentation is not available anymore as the PR was closed or merged._
Load a local dataset twice

## Describe the bug
Load a local "dataset" composed of two csv files twice.

## Steps to reproduce the bug
Put the two joined files in a repository named "Data".
Then in python:

import datasets as ds
ds.load_dataset('Data', data_files = {'file1.csv', 'file2.csv'})

## Expected results
Should give something like (because files have only one data row):
Title, clicks
Truc et astuce, 123
Machin, 12

## Actual results
Gives 
Title, clicks
Truc et astuce, 123
Machin, 12
Truc et astuce, 123
Machin, 12

## Environment info
[file1.csv](https://github.com/huggingface/datasets/files/8256322/file1.csv)
[file2.csv](https://github.com/huggingface/datasets/files/8256323/file2.csv)
- `datasets` version: 2.0.0
- Platform: Linux-5.4.0-65-generic-x86_64-with-glibc2.10
- Python version: 3.8.12
- PyArrow version: 7.0.0
- Pandas version: 1.4.1

Hi @caush, thanks for reporting:

In order to load local CSV files, you can use our "csv" loading script: https://huggingface.co/docs/datasets/loading#csv
```python
dataset = load_dataset("csv", data_files=["data/file1.csv", "data/file2.csv"])
```
OR:
```python
dataset = load_dataset("csv", data_dir="data")
```

Alternatively, you may also use:
```python
dataset = load_dataset("data")
Frugal score deprecations

## Describe the bug
The frugal score returns a really verbose output with warnings that can be easily changed. 

## Steps to reproduce the bug
```python
# Sample code to reproduce the bug
from datasets.load import load_metric

frugal = load_metric("frugalscore")
frugal.compute(predictions=["Do you like spinachis"], references=["Do you like spinach"])
```

## Expected results
A clear and concise description of the expected results.
```
{'scores': [0.9946]}
```

## Actual results
Specify the actual results or traceback.
```
PyTorch: setting up devices
The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 864.09ba/s]
Using amp half precision backend
The following columns in the test set  don't have a corresponding argument in `BertForSequenceClassification.forward` and have been ignored: sentence2, sentence1. If sentence2, sentence1 are not expected by `BertForSequenceClassification.forward`,  you can safely ignore this message.
***** Running Prediction *****
  Num examples = 1
  Batch size = 64
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 4644.85it/s]
 {'scores': [0.9946]}
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.17.0
- Platform: Linux-5.13.0-30-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 7.0.0


Hi @Ierezell, thanks for reporting.

I'm making a PR to suppress those logs from the terminal.
Fix main_classes docs index

Currently the `main_classes` documentation has a wrong index. I believe this comes from issues in the examples of the Translation feature types

![image](https://user-images.githubusercontent.com/42851186/158426345-2ee1ceef-ddf3-4a6f-a93e-d1a8f38a44f5.png)


_The documentation is not available anymore as the PR was closed or merged._

Hmm it's still not good 
![image](https://user-images.githubusercontent.com/42851186/158429361-e19ce25b-c259-4ded-8473-075deafdbb96.png)

any idea what could cause this ?

Ok fixed :)
Document cases for github datasets

In general we recommend adding the new dataset under a username or organization in the Hugging Face Hub at [hf.co/datasets](hf.co/datasets), but users can still add a dataset on github in some cases.

I added a paragraph in the documentation to explain in which cases it can make more sense to open a PR on github:

- when you need the dataset to be reviewed
- when you need long-term maintenance from the HF team
- when there’s no clear org name / namespace that you can put the dataset under

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3924). All of your documentation changes will be reflected on that endpoint.

Yay!
Add methods to IterableDatasetDict

Following the new methods added in #3826 and https://github.com/huggingface/datasets/pull/3862 I added several methods to IterableDatasetDict:
    - map
    - filter
    - shuffle
    - with_format
    - cast
    - cast_column
    - remove_columns
    - rename_column
    - rename_columns


The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3923). All of your documentation changes will be reflected on that endpoint.

Is this feature stale or needs any help to it ? If so I can quickly send a PR. Thanks

CC : @lhoestq, @albertvillanova 

These features have been merged and are already available, thanks :)

Hello @lhoestq, I see that `IterableDataset` doesn't allow features like `take`, `len`, `slice`  which can enable a lot of stuffs. Is it worth an addition ? Or is it intended that they didn't have those features ?

IterableDataset objects don't have `len` or `slice` because they can be possibly unbounded (you don't know in advance how many items they contain). Though IterableDataset.take and IterableDataset.skip do exist.
'datasets.features' is not a package

@albertvillanova 
python 3.9
os: ubuntu 20.04

In conda environment

torch installed by


```/env/bin/pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html```
datasets package is installed by 
```
/env/bin/pip  install datasets==1.8.0

```
During runing the code I have this error 


```
[6]<stderr>:  File "/home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/lib/python3.9/site-packages/torch/serialization.py", line 875, in find_class
[6]<stderr>:    return super().find_class(mod_name, name)
[6]<stderr>:ModuleNotFoundError: No module named 'datasets.features.features'; 'datasets.features' is not a package

```
precisely this error appears when 

torch.load('data_file.pt')
```

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/lib/python3.9/site-packages/torch/serialization.py", line 607, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/lib/python3.9/site-packages/torch/serialization.py", line 882, in _load
    result = unpickler.load()
  File "/home/arij/Memory-transformer-with-hierarchical-attention_MLM/env/lib/python3.9/site-packages/torch/serialization.py", line 875, in find_class
    return super().find_class(mod_name, name)
ModuleNotFoundError: No module named 'datasets.features.features'; 'datasets.features' is not a package
```

Why I am getting this error?


Hi @Arij-Aladel,

You are using a very old version of our library `datasets`: 1.8.0
Current version is 2.0.0 (and the previous one was 1.18.4)

Please, try to update `datasets` library and check if the problem persists:
```shell
/env/bin/pip install -U datasets

The problem I can no I have build my project on this version and old version on transformers. I have preprocessed the data again to use it. Thank for your reply
AttributeError: 'DatasetDict' object has no attribute 'features'

## Describe the bug
Receiving the error when trying to check for Dataset features

## Steps to reproduce the bug
from datasets import Dataset
dataset = Dataset.from_pandas(df[['id', 'words', 'bboxes', 'ner_tags', 'image_path']])

dataset.features

## Expected results
A clear and concise description of the expected results.

## Actual results
Getting the following errror

AttributeError: 'DatasetDict' object has no attribute 'features'

## Environment info
Copy-and-paste the text below in your GitHub issue.

- `datasets` version: 1.18.4
- Platform: Linux-4.14.252-131.483.amzn1.x86_64-x86_64-with-glibc2.9
- Python version: 3.6.13
- PyArrow version: 6.0.1


You are likely trying to get the `features` from a `DatasetDict`, a dictionary containing `Datasets`. You probably first want to index into a particular split from your `DatasetDict` i.e. `dataset['train'].features`. 

For example 

```python 
ds = load_dataset('mnist')
ds.features
```
Returns 
```python
---------------------------------------------------------------------------

AttributeError                            Traceback (most recent call last)

[<ipython-input-39-791c1f9df6c2>](https://localhost:8080/#) in <module>()
----> 1 ds.features

AttributeError: 'DatasetDict' object has no attribute 'features'
```

If we look at the dataset variable, we see it is a `DatasetDict`:

```python 
print(ds)
DatasetDict({
    train: Dataset({
        features: ['image', 'label'],
        num_rows: 60000
    })
    test: Dataset({
        features: ['image', 'label'],
        num_rows: 10000
    })
})
```

We can grab the features from a split by indexing into `train`:
```python
ds['train'].features
{'image': Image(decode=True, id=None),
 'label': ClassLabel(num_classes=10, names=['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'], id=None)}
```

Hope that helps 

Yes, Thanks for that clarification,
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files

## Describe the bug
Can't load the dataset

## Steps to reproduce the bug
```python
# Sample code to reproduce the bug
```
from datasets import load_dataset

dataset = load_dataset('multi_news')
dataset_2=load_dataset("reddit_tifu", "long")


## Actual results
 raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/uc?export=download&id=1ffWfITKFMJeqjT8loC8aiCLRNJpc_XnF']

## Environment info
- `datasets` version: 1.18.4
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.8.0
- PyArrow version: 6.0.1


Hi @willowdong! These issues were fixed on master. We will have a new release of `datasets` later today. In the meantime, you can avoid these issues by installing `datasets` from master as follows:
```bash
pip install git+https://github.com/huggingface/datasets.git
```

You should force redownload:
```python
dataset = load_dataset("multi_news", download_mode="force_redownload")
dataset_2 = load_dataset("reddit_tifu", "long", download_mode="force_redownload")

Fixed by:
- #3787 
- #3843
Create README.md for GLUE

I still have a hesitation regarding the format of inputs -- whether it's a list or a list of lists? -- hopefully @lhoestq will be able to clarify.

Also tagging @yjernite for the Limitations section. Happy to hear your thoughts!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3916). All of your documentation changes will be reflected on that endpoint.
Use templates for doc-builidng jobs

This PR updates the jobs for all doc-building related things by using the templates introduced on `doc-builder`. By putting those once there, we make sure every repo gets the latest fixes on the doc-building github actions :-)

Note: all libraries must share the same docker image for those doc-building jobs. For now, all the one used (`huggingface/transformers-doc-builder`) contains all extra steps of the datasets install for docbuling (mainly libsndfile) but if in the future some additional steps are necessary on top of `pip install -e .[dev]`, this docker image will need to be updated with the extra deps.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3914). All of your documentation changes will be reflected on that endpoint.

You can ignore the CI failures btw, they're unrelated to this PR
Deterministic split order in DatasetDict.map

The order in which the splits are processed by `map` is not deterministic in `DatasetDict.map`. This can cause caching issues when the processing function is stateful and sensible to the order in which examples are processed

Close https://github.com/huggingface/datasets/issues/3847

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3913). All of your documentation changes will be reflected on that endpoint.

I'm surprised this is needed because the order of the `dict` keys is deterministic as of Python 3.6 (documented in 3.7). Is there a reproducer for this behavior? I wouldn't make this change unless it's absolutely needed because `sorted` modifies the initial order of the keys.

Indeed this doesn't fix the issue apparently. Actually this is probably because the tokenizer used to process the second split is in a state that has been modified by the first split.

Therefore after reloading the first split from the cache, then the second split can't be reloaded since the tokenizer hasn't seen the first split (and therefore is considered a different tokenizer).
Fix text loader to split only on universal newlines

Currently, `text` loader breaks on a superset of universal newlines, which also contains Unicode line boundaries. See: https://docs.python.org/3/library/stdtypes.html#str.splitlines

However, the expected behavior is to get the lines splitted only on universal newlines: "\n", "\r\n" and "\r".

See: oscar-corpus/corpus#18

Fix #3729.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3910). All of your documentation changes will be reflected on that endpoint.

Looks like the test needs to be updated for windows ^^'

I don't think this is the same issue as in https://github.com/oscar-corpus/corpus/issues/18, where the OSCAR metadata has line offsets that use only `\n` as the newline marker to count lines, not `\r\n` or `\r`.

It looks like the OSCAR data loader is opening the data files with `gzip.open` directly and I don't think this text loader is used, but I'm not familiar with a lot of `datasets` internals so I could be mistaken?

You are right @adrianeboyd.

This PR fixes #3729.

Additionally, this PR is somehow related to the OSCAR issue. However, the OSCAR issue have multiple root causes: one is the offset initialization (as you pointed out); other is similar to this case: Unicode newlines are not properly handled.

I will make a change proposal for OSCAR this afternoon.

@lhoestq I'm working on fixing the Windows tests on my Windows machine...

I finally changed the approach in order to avoid having "\r\n" and "\r" line breaks in Python `str` read from files on Windows/old Macintosh machines.
NonMatchingChecksumError on Spider dataset 

## Describe the bug
Failure to generate dataset ```spider``` because of checksums error for dataset source files.

## Steps to reproduce the bug
```
from datasets import load_dataset
spider = load_dataset("spider")
```


## Expected results
Checksums should match for files from url ['https://drive.google.com/uc?export=download&id=1_AckYkinAnhqmRQtGsQgUKAnTHxxX5J0']

## Actual results
```
>>> load_dataset("spider")
load_dataset("spider")
Downloading and preparing dataset spider/spider (download: 95.12 MiB, generated: 5.17 MiB, post-processed: Unknown size, total: 100.29 MiB) to /home/user/.cache/huggingface/datasets/spider/spider/1.0.0/79778ebea87c59b19411f1eb3eda317e9dd5f7788a556d837ef25c3ae6e5e8b7...
Traceback (most recent call last):
  File "/home/user/py3_env/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3441, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-5-d4cb54197348>", line 1, in <module>
    load_dataset("spider")
  File "/home/user/py3_env/lib/python3.8/site-packages/datasets/load.py", line 1702, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/user/py3_env/lib/python3.8/site-packages/datasets/builder.py", line 594, in download_and_prepare
    self._download_and_prepare(
  File "/home/user/py3_env/lib/python3.8/site-packages/datasets/builder.py", line 665, in _download_and_prepare
    verify_checksums(
  File "/home/user/py3_env/lib/python3.8/site-packages/datasets/utils/info_utils.py", line 40, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/uc?export=download&id=1_AckYkinAnhqmRQtGsQgUKAnTHxxX5J0']
```


## Environment info
datasets version: 1.18.3
Platform: Ubuntu 20 LTS
Python version: 3.8.10
PyArrow version: 6.0.1 





Hi @kolk, thanks for reporting.

Indeed, Google Drive service recently changed their service and we had to add a fix to our library to cope with that change:
- #3787 

We just made patch release last week: 1.18.4 https://github.com/huggingface/datasets/releases/tag/1.18.4

Please, feel free to update your local `datasets` version, so that you get the fix:
```shell
pip install -U datasets
```
Perplexity Metric Card

Add Perplexity metric card

Note that it is currently still missing the citation, but I plan to add it later today.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3905). All of your documentation changes will be reflected on that endpoint.

I'm wondering if we should add that perplexity can be used for analyzing datasets as well

Otherwise, looks good! Good job, @emibaylor !
Dataset viewer issue for IndicParaphrase- the preview doesn't show

## Dataset viewer issue for '*IndicParaphrase*'

**Link:** *[IndicParaphrase](https://huggingface.co/datasets/ai4bharat/IndicParaphrase/viewer/hi/validation)*

*The preview of the dataset doesn't come up. 
The error on the console is:
Status code:   400
Exception:     FileNotFoundError
Message:       [Errno 2] No such file or directory: '/home/hf/datasets-preview-backend/hi_IndicParaphrase_v1.0.tar'*

Am I the one who added this dataset ? Yes


It seems to have been fixed:

<img width="1534" alt="Capture d’écran 2022-04-12 à 14 10 07" src="https://user-images.githubusercontent.com/1676121/162959599-6b7fef7c-8411-4e03-8f00-90040a658079.png">
Create README.md for WER metric

Proposing a draft WER metric card, @lhoestq I'm not very certain about "Values from popular papers" -- I don't know ASR very well, what do you think of the examples I found?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3898). All of your documentation changes will be reflected on that endpoint.

For ASR you can probably ping @patrickvonplaten 

Ah only noticed now that ` # Values from popular papers` is from a template. @lhoestq @sashavor - not really sure if this section is useful in general really. 

IMO, it's more confusing/misleading than it helps. E.g. a value of 0.03 WER on a fake read-out audio dataset is not better than a WER of 0.3 on a real-world noisy, conversational audio dataset. I think the same holds true for other metrics no? I can think of very little metrics where a metric value is not dataset dependent. E.g. perplexity is super dataset dependent, summarization metrics like ROUGE as well, ...

Also, I don't really see what this section tries to achieve - is the idea here to give the reader some papers that use this metric to better understand in which context it is used? Should we maybe rename the section to `Popular papers making use of this metric` or something? 



I put "Values from popular papers" as a subsection of "Output values" -- I hope that's a compromise that works for everyone :hugs:
Align tqdm control/cache control with Transformers

This PR:
* aligns the `tqdm` logic with Transformers (follows https://github.com/huggingface/transformers/pull/15167) by moving the code to `utils/logging.py`, adding `enable_progres_bar`/`disable_progres_bar` and removing `set_progress_bar_enabled` (a note for @lhoestq: I'm not adding `logging.tqdm` to the public namespace in this PR to avoid the situation where `from datasets import *; tqdm` would overshadow the standard `tqdm` 
* aligns the cache control with the new `tqdm` logic by adding `enable_caching`/`disable_caching` to the  public namespace and deprecating `set_caching_enabled` (not fully removing it because it's used more often than `set_progress_bar_enabled` and has a dedicated example in the old docs)

Fix #3586 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3897). All of your documentation changes will be reflected on that endpoint.
Missing google file for `multi_news` dataset

## Dataset viewer issue for '*multi_news*'

**Link:** https://huggingface.co/datasets/multi_news

```
Server error

Status code:   400
Exception:     FileNotFoundError
Message:       https://drive.google.com/uc?export=download&id=1vRY2wM6rlOZrf9exGTm5pXj5ExlVwJ0C/multi-news-original/train.src
```

Am I the one who added this dataset ? No


reported by @abidlabs 

related to https://github.com/huggingface/datasets/pull/3843?

`datasets` 1.18.4 fixes the issue when you load the dataset with `load_dataset`.

When loading in streaming mode, the fix is indeed on https://github.com/huggingface/datasets/pull/3843 which will be merged soon :)

That is. The PR #3843 was just opened a bit later we had made our 1.18.4 patch release...
Once merged, that will fix this issue. 

OK. Should fix the viewer for 50 datasets

<img width="148" alt="Capture d’écran 2022-03-14 à 11 51 02" src="https://user-images.githubusercontent.com/1676121/158157853-6c544a47-2d6d-4ac4-964a-6f10951ec36b.png">
Fix code examples indentation

Some code examples are currently not rendered correctly. I think this is because they are over-indented

cc @mariosasko 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3895). All of your documentation changes will be reflected on that endpoint.

Still not rendered properly: https://moon-ci-docs.huggingface.co/docs/datasets/pr_3895/en/package_reference/main_classes#datasets.Dataset.align_labels_with_mapping

My last commit should have fixed it, I don't know why the dev doc build is not showing my last changes

Let me merge this and we can see on `master` how it renders, until the dev doc build is fixed
[docs] make dummy data creation optional

Related to #3507 : dummy data for datasets created on the Hugging Face Hub are optional.

We can discuss later to make them optional for datasets in this repository as well

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3894). All of your documentation changes will be reflected on that endpoint.

The dev doc build rendering doesn't seem to be updated with my last commit for some reason

Merging it anyway since I'd like to share this page with users 🙃
Add default branch for doc building

Since other libraries use `main` as their default branch and it's now the standard default, you have to specify a different name in the doc config if you're using `master` like datasets (`doc-builder` tries to guess it, but in the job, we have weird checkout of merge commits so it doesn't always manage to get it right).

This PR makes sure it will always use master for the dev doc (until you decide to switchto main)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3893). All of your documentation changes will be reflected on that endpoint.

Yes! And when we discovered on the Transformers side that this check fails on the GitHub actions, we added a config attribute to have a default. Setting in Transformers fixed the issue of the doc being deployed to main, so porting the fix here too :-)
Fix CLI test checksums

Previous PR:
- #3796 

introduced a side effect: `datasets-cli test` generates `dataset_infos.json` with `None` checksum values.

See:
- #3805 

This PR introduces a way for `datasets-cli test` to force to record infos, even if `verify_infos=False`

Close #3848.

CC: @craffel 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3892). All of your documentation changes will be reflected on that endpoint.

Feel free to merge if it's good for you :)

I've added a test @lhoestq. Once all green, I'll merge. 

Last failing tests do not have nothing to do with this PR.
Fix race condition in doc build

Following https://github.com/huggingface/datasets/runs/5499386744 it seems that race conditions that create issues when updating the doc. I took the same approach as in `transformers` to fix race conditions

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3891). All of your documentation changes will be reflected on that endpoint.
Update beans download urls

Replace the old URLs with the Hub [URLs](https://huggingface.co/datasets/beans/tree/main/data).

Also reported by @stevhliu. 

Fix #3889 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_3890). All of your documentation changes will be reflected on that endpoint.

@albertvillanova Thanks for investigating and fixing that issue. I regenerated the `dataset_infos.json` file.
Cannot load beans dataset (Couldn't reach the dataset)

## Describe the bug
The beans dataset is unavailable to download.

## Steps to reproduce the bug
```python
from datasets import load_dataset
ds = load_dataset('beans')
```

## Expected results
The dataset would be downloaded with no issue.

## Actual results

```
ConnectionError: Couldn't reach https://storage.googleapis.com/ibeans/train.zip (error 403)
```

[It looks like the billing of this project has been disabled because it is associated with a delinquent account.](https://storage.googleapis.com/ibeans/train.zip )

## Environment info
Google Colab


Hi ! A pull request is open to fix the dataset, we'll release a patch soon with a new release of `datasets` :)