SergeiGKS/huggingface-dataset-issues huggingface dataset:
Issue: Pre-Processing Cache Fails when using a Factory pattern

Comment:
Ok - this is still an issue but I believe the root cause is different than I originally thought. I'm now able to get caching to work consistently with the above example as long as I fix the python hash seed `export PYTHONHASHSEED=1234`
Issue: Pre-Processing Cache Fails when using a Factory pattern

Comment:
Hi! 

Yes, our hasher should work with decorators. For instance, this dummy example:
```python
def f(arg):
    def f1(ex):
        return {"a": ex["col1"] + arg}
    return f1
```
gives the same hash across different Python sessions (`datasets.fingerprint.Hasher.hash(f("string1")` returns `"408c9059f89dbd6c"` on my machine).

Could you please make the example self-contained? This way, we can reproduce the bug. Additionally, you can try to find the problematic object yourself by testing their hash with `datasets.fingerprint.Hasher.hash(obj)`

This could be related to https://github.com/huggingface/datasets/issues/3638.
Issue: NonMatchingChecksumError in xcopa dataset

Comment:
Hi @afcruzs-ms, thanks for opening this separate issue for your problem.

The root problem in the other issue (#3792) was a change in the service of Google Drive.

But in your case, the `xcopa` dataset is not hosted on Google Drive. Therefore, the root cause should be a different one.

Let me look at it...
Issue: NonMatchingChecksumError in xcopa dataset

Comment:
@afcruzs-ms, I'm not able to reproduce the issue you reported:
```python
In [1]: from datasets import load_dataset
   ...: dataset = load_dataset("xcopa", "it")
Downloading builder script: 5.21kB [00:00, 2.75MB/s]                                                                                                                                                       
Downloading metadata: 28.6kB [00:00, 14.5MB/s]                                                                                                                                                             
Downloading and preparing dataset xcopa/it (download: 627.09 KiB, generated: 76.43 KiB, post-processed: Unknown size, total: 703.52 KiB) to .../.cache/huggingface/datasets/xcopa/it/1.0.0/e1fab65f984b24c8b66bcf7ac27a26a1182f84adfb2e74035861be65e214b9e6...
Downloading data: 642kB [00:00, 5.42MB/s]
Dataset xcopa downloaded and prepared to .../.cache/huggingface/datasets/xcopa/it/1.0.0/e1fab65f984b24c8b66bcf7ac27a26a1182f84adfb2e74035861be65e214b9e6. Subsequent calls will reuse this data.  
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 733.27it/s]

In [2]: dataset
Out[2]: 
DatasetDict({
    test: Dataset({
        features: ['premise', 'choice1', 'choice2', 'question', 'label', 'idx', 'changed'],
        num_rows: 500
    })
    validation: Dataset({
        features: ['premise', 'choice1', 'choice2', 'question', 'label', 'idx', 'changed'],
        num_rows: 100
    })
})
```

Maybe you have some issue with your cached data... Could you please try to force the redownload of the data?
```python
dataset = load_dataset("xcopa", "it", download_mode="force_redownload")
```
Issue: NonMatchingChecksumError in xcopa dataset

Comment:
unfortunately, i am having a similar problem with the irc_disentaglement dataset :/
my code:
```
from datasets import load_dataset

dataset = load_dataset("irc_disentangle", download_mode="force_redownload")
```

however, it produces the same error as @afcruzs-ms 
```
[38](file:///Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/datasets/utils/info_utils.py?line=37) if len(bad_urls) > 0:
     [39](file:///Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/datasets/utils/info_utils.py?line=38)     error_msg = "Checksums didn't match" + for_verification_name + ":\n"
---> [40](file:///Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/datasets/utils/info_utils.py?line=39)     raise NonMatchingChecksumError(error_msg + str(bad_urls))
     [41](file:///Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/datasets/utils/info_utils.py?line=40) logger.info("All the checksums matched successfully" + for_verification_name)

NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://github.com/jkkummerfeld/irc-disentanglement/tarball/master']
```

I attempted to use the `ignore_verifications' as such:
```
ds = datasets.load_dataset('irc_disentangle', download_mode="force_redownload", ignore_verifications=True)

```
```
Downloading builder script: 12.0kB [00:00, 5.92MB/s]                   
Downloading metadata: 7.58kB [00:00, 3.48MB/s]                   
No config specified, defaulting to: irc_disentangle/ubuntu
Downloading and preparing dataset irc_disentangle/ubuntu (download: 112.98 MiB, generated: 60.05 MiB, post-processed: Unknown size, total: 173.03 MiB) to /Users/laylabouzoubaa/.cache/huggingface/datasets/irc_disentangle/ubuntu/1.0.0/0f24ab262a21d8c1d989fa53ed20caa928f5880be26c162bfbc02445dbade7e5...
Downloading data: 118MB [00:09, 12.1MB/s] 
                                                                             
Dataset irc_disentangle downloaded and prepared to /Users/laylabouzoubaa/.cache/huggingface/datasets/irc_disentangle/ubuntu/1.0.0/0f24ab262a21d8c1d989fa53ed20caa928f5880be26c162bfbc02445dbade7e5. Subsequent calls will reuse this data.
100%|██████████| 3/3 [00:00<00:00, 675.38it/s]
```
but, this returns an empty set?

```
DatasetDict({
    train: Dataset({
        features: ['id', 'raw', 'ascii', 'tokenized', 'date', 'connections'],
        num_rows: 0
    })
    test: Dataset({
        features: ['id', 'raw', 'ascii', 'tokenized', 'date', 'connections'],
        num_rows: 0
    })
    validation: Dataset({
        features: ['id', 'raw', 'ascii', 'tokenized', 'date', 'connections'],
        num_rows: 0
    })
})
```

not sure what else to try at this point?
Thanks in advanced🤗
Issue: NonMatchingChecksumError in xcopa dataset

Comment:
Thanks @labouz for reporting: yes, better opening a new GitHub issue as you did. I'm addressing it:
- #4376
Issue: Text builder with custom separator line boundaries

Comment:
Hi ! Interresting :)

Could you give more details on what kind of separators you would like to use instead ?
Issue: Text builder with custom separator line boundaries

Comment:
Ok I see, maybe there can be a `sep` parameter to allow users to specify what line/paragraph separator they'd like to use
Issue: Text builder with custom separator line boundaries

Comment:
Thanks for requesting this enhancement. We have recently found a somehow related issue with another dataset:
- #3704

Let me make a PR proposal.
Issue: Checksums didn't match for dataset source

Comment:
Same issue with `dataset = load_dataset("dbpedia_14")`
```
NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/uc?export=download&id=0Bz8a_Dbh9QhbQ2Vic1kxMmZZQ1k']
Issue: Checksums didn't match for dataset source

Comment:
I think this is a side-effect of #3787. The checksums won't match because the URLs have changed. @rafikg @Y0mingZhang, while this is fixed, maybe you can load the datasets as such:

`data = datasets.load_dataset("wiki_lingua", name=language, split="train[:2000]", ignore_verifications=True)`
`dataset = load_dataset("dbpedia_14", ignore_verifications=True)`

This will, most probably, skip the verifications and integrity checks listed [here](https://huggingface.co/docs/datasets/loading_datasets.html#integrity-verifications)
Issue: Checksums didn't match for dataset source

Comment:
Hi! Installing the `datasets` package from master (`pip install git+https://github.com/huggingface/datasets.git`) and then redownloading the datasets with `download_mode` set to `force_redownload` (e.g. `dataset = load_dataset("dbpedia_14", download_mode="force_redownload")`) should fix the issue.
Issue: Checksums didn't match for dataset source

Comment:
Hi @rafikg and @Y0mingZhang, thanks for reporting.

Indeed it seems that Google Drive changed their way to access their data files. We have recently handled that change:
- #3787

but it will be accessible to users only in our next release of the `datasets` version.
- Note that our latest release (version 1.18.3) was made before this fix: https://github.com/huggingface/datasets/releases/tag/1.18.3

In the meantime, as @mariosasko explained, you can incorporate this "fix" by installing our library from the GitHub master branch:
```shell
pip install git+https://github.com/huggingface/datasets#egg=datasets
```
Then, you should force the redownload of the data (before the fix, you are just downloading/caching the virus scan warning page, instead of the data file):
```shell
data = datasets.load_dataset("wiki_lingua", name=language, split="train[:2000]", download_mode="force_redownload")
Issue: Checksums didn't match for dataset source

Comment:
@albertvillanova  by running:
```
pip install git+https://github.com/huggingface/datasets#egg=datasets
data = datasets.load_dataset("wiki_lingua", name=language, split="train[:2000]", download_mode="force_redownload", ignore_verifications=True)
```

I had a pickle error **UnpicklingError: invalid load key, '<'** in this part of code  both `locally and on google colab`:

```
"""Yields examples."""
with open(filepath, "rb") as f:
         data = pickle.load(f)
for id_, row in enumerate(data.items()):
         yield id_, {"url": row[0], "article": self._process_article(row[1])}
```
Issue: Checksums didn't match for dataset source

Comment:
This issue impacts many more datasets than the ones mention in this thread.  Can we post # of downloads for each dataset by day (by successes and failures)?  If so, it should be obvious which ones are failing.
Issue: Checksums didn't match for dataset source

Comment:
I can see this problem too in xcopa, unfortunately installing the latest master (1.18.4.dev0) doesn't work, @albertvillanova .

```
from datasets import load_dataset
dataset = load_dataset("xcopa", "it")
```

Throws

```
in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     38     if len(bad_urls) > 0:
     39         error_msg = "Checksums didn't match" + for_verification_name + ":\n"
---> 40         raise NonMatchingChecksumError(error_msg + str(bad_urls))
     41     logger.info("All the checksums matched successfully" + for_verification_name)
     42 

NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://github.com/cambridgeltl/xcopa/archive/master.zip']
```
Issue: Checksums didn't match for dataset source

Comment:
Hi @rafikg, I think that is another different issue. Let me check it... 

I guess maybe you are using a different Python version that the one the dataset owner used to create the pickle file...
Issue: Checksums didn't match for dataset source

Comment:
@kwchurch the datasets impacted for this specific issue are the ones which are hosted at Google Drive.
Issue: Checksums didn't match for dataset source

Comment:
@afcruzs-ms I think your issue is a different one, because that dataset is not hosted at Google Drive. Would you mind open another issue for that other problem, please? Thanks! :)
Issue: Checksums didn't match for dataset source

Comment:
@albertvillanova just to let you know that I tried it locally and on colab and it is the same error
Issue: Checksums didn't match for dataset source

Comment:
There are many many datasets on HugggingFace that are receiving this checksum error.  Some of these datasets are very popular.  There must be a way to track these errors, or to do regression testing.  We don't want to catch each of these errors on each dataset, one at a time.
Issue: Checksums didn't match for dataset source

Comment:
@rafikg I am sorry, but I can't reproduce your issue. For me it works OK for all languages. See: https://colab.research.google.com/drive/1yIcLw1it118-TYE3ZlFmV7gJcsF6UCsH?usp=sharing
Issue: Checksums didn't match for dataset source

Comment:
@kwchurch the PR #3787 fixes this issue (generated by a change in Google Drive service) for ALL datasets with this issue. Once we make our next library release (in a couple of days), the fix will be accessible to all users that update our library from PyPI.
Issue: Checksums didn't match for dataset source

Comment:
By the way, @rafikg, I discovered the URL for Spanish was wrong. I've created a PR to fix it:
- #3806
Issue: Checksums didn't match for dataset source

Comment:
I have the same problem with "wider_face" dataset. It seems that "load_dataset" function can not download the dataset from google drive.
Issue: Checksums didn't match for dataset source

Comment:
I think this has to be fixed on the google drive side, but you also have to delete the bad stuff from your local cache.  This is not a great design, but it is what it is.
Issue: Checksums didn't match for dataset source

Comment:
We have fixed the issues with the datasets:
- wider_face: by hosting their data files on the HuggingFace Hub (CC: @HosseynGT)
- fever: by updating to their new data URLs (CC: @MoritzLaurer)
Issue: Checksums didn't match for dataset source

Comment:
This is a super-common failure mode.  We really need to find a better workaround.  My solution was to wait until the owner of the dataset in question did the right thing, and then I had to delete my cached versions of the datasets with the bad checksums.  I don't understand why this happens.  Would it be possible to maintain a copy of the most recent version that was known to work, and roll back to that automatically if the checksums fail?  And if the checksums fail, couldn't the system automatically flush the cached versions with the bad checksums?  It feels like we are blaming the provider of the dataset, when in fact, there are things that the system could do to ease the pain.  Let's take these error messages seriously.  There are too many of them involving too many different datasets.
Issue: Checksums didn't match for dataset source

Comment:
Same for [DART dataset](https://huggingface.co/datasets/dart):
```
NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://raw.githubusercontent.com/Yale-LILY/dart/master/data/v1.1.1/dart-v1.1.1-full-train.json', 'https://raw.githubusercontent.com/Yale-LILY/dart/master/data/v1.1.1/dart-v1.1.1-full-dev.json', 'https://raw.githubusercontent.com/Yale-LILY/dart/master/data/v1.1.1/dart-v1.1.1-full-test.json']
```
Issue: Checksums didn't match for dataset source

Comment:
- @thesofakillers the issue with `exams` was fixed on 16 Aug by this PR:
  - #4853
- @Aktsvigun the issue with `dart` has been transferred to the Hub: https://huggingface.co/datasets/dart/discussions/1
  - and fixed by PR: https://huggingface.co/datasets/dart/discussions/2
- @Carol-gutianle the issue with `multi_news` have been transferred to the Hub as well: https://huggingface.co/datasets/multi_news/discussions/1
  - not reproducible: maybe you should try to update `datasets`

For information to everybody, we are removing the checksum verifications (that were creating a bad user experience). This will be in place in the following weeks.
Issue: Only-data dataset loaded unexpectedly as validation split

Comment:
I see two options:
1. drop the "dev" keyword since it can be considered too generic
2. improve the pattern to something more reasonable, e.g. asking for a separator before and after "dev"
```python
["*[ ._-]dev[ ._-]*", "dev[ ._-]*"]
```

I think 2. is nice. If we agree on this one we can even decide to require the separation for the other split keywords "train", "test" etc.
Issue: Only-data dataset loaded unexpectedly as validation split

Comment:
Yes, I had something like that on mind: "dev" not being part of a word.
```
"[^a-zA-Z]dev[^a-zA-Z]"
Issue: Only-data dataset loaded unexpectedly as validation split

Comment:
Is there a reason why we want that regex? It feels like something that'll still be an issue for some weird case. "my_dataset_dev" doesn't match your regex, "my_dataset_validation" doesn't either ... Why not always "train" unless specified?
Issue: Only-data dataset loaded unexpectedly as validation split

Comment:
The regex is needed as part of our effort to make datasets configurable without code. In particular we define some generic dataset repository structures that users can follow

> ```
> "[^a-zA-Z]*dev[^a-zA-Z]*"
> ```

unfortunately our glob doesn't support "^": 

https://github.com/fsspec/filesystem_spec/blob/3e739db7e53f5b408319dcc9d11e92bc1f938902/fsspec/spec.py#L465-L479
Issue: Only-data dataset loaded unexpectedly as validation split

Comment:
> "my_dataset_dev" doesn't match your regex, "my_dataset_validation" doesn't either ... Why not always "train" unless specified?

And `my_dataset_dev.foo` would match the pattern, and we also have the same pattern but for the "validation" keyword so `my_dataset_validation.foo` would work too
Issue: Only-data dataset loaded unexpectedly as validation split

Comment:
> The regex is needed as part of our effort to make datasets configurable without code

This feels like coding with the filename ^^'
Issue: Only-data dataset loaded unexpectedly as validation split

Comment:
This is still much easier than having to write a full dataset script right ? :p
Issue: Bug downloading Virus scan warning page from Google Drive URLs

Comment:
Once the PR merged into master and until our next `datasets` library release, you can get this fix by installing our library from the GitHub master branch:
```shell
pip install git+https://github.com/huggingface/datasets#egg=datasets
```
Then, if you had previously tried to load the data and got the checksum error, you should force the redownload of the data (before the fix, you just downloaded and cached the virus scan warning page, instead of the data file):
```shell
load_dataset("...", download_mode="force_redownload")
```
Issue: Unable to Download CNN-Dailymail Dataset 

Comment:
Glad to help @albertvillanova! Just fine-tuning the PR, will comment once I am able to get it up and running 😀
Issue: Not be able to download dataset - "Newsroom"

Comment:
Hi @Darshan2104, thanks for reporting.

Please note that at Hugging Face we do not host the data of this dataset, but just a loading script pointing to the host of the data owners.

Apparently the data owners changed their data host server. After googling it, I found their new website at: https://lil.nlp.cornell.edu/newsroom/index.html
- Download page: https://lil.nlp.cornell.edu/newsroom/download/index.html

I'm fixing the link in our Datasets library.
Issue: Allow download only some files from the Wikipedia dataset

Comment:
Hi @jvanz, thank you for your proposal.

In fact, we are aware that it is very common the problem you mention. Because of that, we are currently working in implementing a new version of wikipedia on the Hub, with all data preprocessed (no need to use Apache Beam), from where you will be able to use `data_files` to load only a specific subset of the data files.

See:
- #3401
Issue: Checksum mismatch for the reddit_tifu dataset

Comment:
@albertvillanova Thank you for the fast response! However I am still getting the same error:

Downloading: 2.23kB [00:00, ?B/s]
Traceback (most recent call last):
  File "C:\Users\Anna\PycharmProjects\summarization\main.py", line 17, in <module>
    dataset = load_dataset('reddit_tifu', 'long')
  File "C:\Users\Anna\Desktop\summarization\summarization_env\lib\site-packages\datasets\load.py", line 1702, in load_dataset
    builder_instance.download_and_prepare(
  File "C:\Users\Anna\Desktop\summarization\summarization_env\lib\site-packages\datasets\builder.py", line 594, in download_and_prepare
    self._download_and_prepare(
  File "C:\Users\Anna\Desktop\summarization\summarization_env\lib\site-packages\datasets\builder.py", line 665, in _download_and_prepare
    verify_checksums(
  File "C:\Users\Anna\Desktop\summarization\summarization_env\lib\site-packages\datasets\utils\info_utils.py", line 40, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/uc?export=download&id=1ffWfITKFMJeqjT8loC8aiCLRNJpc_XnF']

I have cleaned the cache/huggingface/datasets & cache/huggingface/modules files and also tried on another machine with a fresh installation of trasnformers & datasets. 
The reddit_tifu.py that gets downloaded still has the previous url on line 51, _URL = "https://drive.google.com/uc?export=download&id=1ffWfITKFMJeqjT8loC8aiCLRNJpc_XnF"
Issue: Checksum mismatch for the reddit_tifu dataset

Comment:
Hi @anna-kay, I'm sorry I didn't clearly explain the details to you:
- the error has been fixed in our `master` branch on GitHub: https://github.com/huggingface/datasets/commit/8ae21bf6a77175dc803ce2f1b93d18b8fbf45586
- the fix will not be accessible to users in PyPI until our next release of the `datasets` library
  - our latest release (version 1.18.3) was made 23 days ago: https://github.com/huggingface/datasets/releases/tag/1.18.3
- in the meantime, you can get the fix if you install datasets from our GitHub `master` branch:
  ```
  pip install git+https://github.com/huggingface/datasets#egg=datasets
  ```
Issue: `dataset = dataset.map()` causes faiss index lost

Comment:
Hi ! Indeed `map` is dropping the index right now, because one can create a dataset with more or fewer rows using `map` (and therefore the index might not be relevant anymore)

I guess we could check the resulting dataset length, and if the user hasn't changed the dataset size we could keep the index, what do you think ?
Issue: `dataset = dataset.map()` causes faiss index lost

Comment:
doing `.add_column("x",x_data)` also removes the index. the new column might be irrelevant to the index so I don't think it should drop. 

Minimal example

```python
from datasets import load_dataset
import numpy as np

data=load_dataset("ceyda/cats_vs_dogs_sample") #just a test dataset
data=data["train"]
embd_data=data.map(lambda x: {"emb":np.random.uniform(-1,0,50).astype(np.float32)})
embd_data.add_faiss_index(column="emb")
print(embd_data.list_indexes())
embd_data=embd_data.add_column("x",[0]*data.num_rows)
print(embd_data.list_indexes())
```
Issue: `dataset = dataset.map()` causes faiss index lost

Comment:
I agree `add_column` shouldn't drop the index indeed ! Is it something you'd like to contribute ? I think it's just a matter of copying the `self._indexes` dictionary to the output dataset
Issue: It's not possible download `20200501.pt` dataset

Comment:
Hi @jvanz, thanks for reporting.

Please note that Wikimedia website does not longer host Wikipedia dumps for so old dates.

For a list of accessible dump dates of `pt` Wikipedia, please see: https://dumps.wikimedia.org/ptwiki/

You can load for example `20220220` `pt` Wikipedia:
```python
dataset = load_dataset("wikipedia", language="pt", date="20220220", beam_runner="DirectRunner")
```
Issue: It's not possible download `20200501.pt` dataset

Comment:
> ```python
> dataset = load_dataset("wikipedia", language="pt", date="20220220", beam_runner="DirectRunner")
> ```

Thank you! I did not know that I can do this. I was following the example in the error message when I do not define which language dataset I'm trying to download.

I've tried something similar changing the date in the `load_dataset` call that I've shared in the bug description. Obviously, it did not work. I need to read the docs more carefully next time. My bad!

Thanks again and sorry for the noise.