SurAyush/github_issues huggingface dataset:
Issue: add books3
State: closed
Author: richarddwang

books3 is part of EleutherAI/The Pile, but AFAIK, The Pile dataset blend all sub datasets together thus we are not able to use just one of its sub dataset from The Pile data. So I create an independent dataset using The Pile preliminary components.

When I was creating dataset card. I found there is room for creating / editing dataset card. I've made it an issue. #2797 

Also I am wondering whether the import of The Pile dataset is actively undertaken (because I may need it recently)? #1675

Comments:
> When I was creating dataset card. I found there is room for creating / editing dataset card. I've made it an issue. #2797

Thanks for the message, we'll definitely improve this

> Also I am wondering whether the import of The Pile dataset is actively undertaken (because I may need it recently)? #1675

Well currently no, but I think @lewtun was about to do it (though he's currently on vacations)

> > Also I am wondering whether the import of The Pile dataset is actively undertaken (because I may need it recently)? #1675
> 
> Well currently no, but I think @lewtun was about to do it (though he's currently on vacations)

yes i plan to start working on this next week #2185 

one question for @richarddwang - do you know if eleutherai happened to also release the "existing" datasets like enron emails and opensubtitles? 

in appendix c of their paper, they provide details on how they extracted these datasets, but it would be nice if we could just point to a url so we can be as close as possible to original implementation.

@lewtun 

> yes i plan to start working on this next week

Nice! Looking forward to it.

> one question for @richarddwang - do you know if eleutherai happened to also release the "existing" datasets like enron emails and opensubtitles?

Sadly, I don't know any existing dataset of enron emails, but I believe opensubtitles dataset is hosted at here. https://the-eye.eu/public/AI/pile_preliminary_components/
![image](https://user-images.githubusercontent.com/17963619/130061667-8c17985a-1c2f-432f-89f0-66a5288611b8.png)


thanks for the link @richarddwang! i think that corpus is actually the youtube subtitles one and my impression is that eleutherai have only uploaded the 14 new datasets they created. i've contacted one of the authors so hopefully they can share some additional info for us :)

btw it might take a while to put together all the corpora if i also need to preprocess them (e.g. the open subtitles / enron email etc), but i expect no longer than a few weeks.
Issue: Fix streaming zip files
State: closed
Author: albertvillanova

Currently, streaming remote zip data files gives `FileNotFoundError` message:
```python
data_files = f"https://huggingface.co/datasets/albertvillanova/datasets-tests-compression/resolve/main/sample.zip"
ds = load_dataset("json", split="train", data_files=data_files, streaming=True)
next(iter(ds))
```

This PR fixes it by adding a glob string.

The corresponding test is implemented in PR #2786.

Comments:
Hi ! I don't fully understand this change @albertvillanova 
The `_extract` method used to return the compound URL that points to the root of the inside of the archive.
This way users can use the usual os.path.join or other functions to point to the relevant files. I don't see why you're using a glob pattern ?

This change is to allow this:
```python
data_files = f"https://huggingface.co/datasets/albertvillanova/datasets-tests-compression/resolve/main/sample.zip"
ds = load_dataset("json", split="train", data_files=data_files, streaming=True)
assert isinstance(ds, IterableDataset)
```
Note that in this case the user will not call os.path.join.

Before this PR it gave error because pointing to the root, without any subsequent join, gives error:
```python
fsspec.open("zip://::https://huggingface.co/datasets/albertvillanova/datasets-tests-compression/resolve/main/sample.zip")
```
Issue: Make creating/editing dataset cards easier, by editing on site and dumping info from test command.
State: open
Author: richarddwang

**Is your feature request related to a problem? Please describe.**

Creating and editing dataset cards should be but not that easy
- If other else know Some information I don't know (bias of dataset, dataset curation, supported dataset, ...), he/she should know the description on hf.co comes from README.md under github huggingface/datasets/datasets/the dataset, and willing to make a pr to add or fix information.
- Many information is also saved in `dataset_info.json` (citaion, description), but still need to write it down to README.md again.
- Contributor need to pip install and start a local server just for tagging the dataset's size. And contributor may be creating the dataset on lab's server, which can't open a browser.  
- if any one proposes a new tag, it doesn't show in the list that another creator see. (a stackoverflow way may be ideal)
- dataset card generator web app doesn't generate the necessary subsecion `Contributions` for us.

**Describe the solution you'd like**
- Everyone (or at least the author/contributor) can edit the description, information, tags of the dataset, on hf.co website. Just like wikipedia+stackoverflow
-  We can infer the actual data size, citation, data instance, ... from `dataset_info.json` and `dataset.arrow`  via `dataset-cli test`


Comments:
Issue: Warnings and documentation about pickling incorrect
State: open
Author: mbforbes

## Describe the bug

I have a docs bug and a closely related docs enhancement suggestion!

### Bug

The warning and documentation say "either `dill` or `pickle`" for fingerprinting. But it seems that `dill`, which is installed by `datasets` by default, _must_ work, or else the fingerprinting fails.

Warning:

https://github.com/huggingface/datasets/blob/450b9174765374111e5c6daab0ed294bc3d9b639/src/datasets/fingerprint.py#L262

Docs:

> For a transform to be hashable, it needs to be pickleable using dill or pickle.
> – [docs](https://huggingface.co/docs/datasets/processing.html#fingerprinting)

For my code, `pickle` works, but `dill` fails. The `dill` failure has already been reported in https://github.com/huggingface/datasets/issues/2643. However, the `dill` failure causes a hashing failure in the datasets library, without any backing off to `pickle`. This implies that it's not the case that either `dill` **or** `pickle` can work, but that `dill` must work if it is installed. I think this is more accurate wording, since it is installed and used by default:

https://github.com/huggingface/datasets/blob/c93525dc291346e54212567fa72d7d607befe937/setup.py#L83

... and the hashing will fail if it fails.

### Enhancement

I think it'd be very helpful to add to the documentation how to debug hashing failures. It took me a while to figure out how to diagnose this. There is a very nice two-liner by @lhoestq in https://github.com/huggingface/datasets/issues/2516#issuecomment-865173139:

```python
from datasets.fingerprint import Hasher
Hasher.hash(my_object)
```

I think add this to the docs will help future users quickly debug any hashing troubles of their own :-)

## Steps to reproduce the bug

`dill` but not `pickle` hashing failure in https://github.com/huggingface/datasets/issues/2643

## Expected results
If either `dill` or `pickle` can successfully hash, the hashing will succeed.

## Actual results
If `dill` or `pickle` cannot hash, the hashing fails.

## Environment info

- `datasets` version: 1.9.0
- Platform: Linux-5.8.0-1038-gcp-x86_64-with-glibc2.31
- Python version: 3.9.6
- PyArrow version: 4.0.1


Comments:
Issue: Update: GooAQ - add train/val/test splits
State: closed
Author: bhavitvyamalik

[GooAQ](https://github.com/allenai/gooaq) dataset was recently updated after splits were added for the same. This PR contains new updated GooAQ with train/val/test splits and updated README as well.

Comments:
@albertvillanova my tests are failing here:
```
dataset_name = 'gooaq'

    def test_load_dataset(self, dataset_name):
        configs = self.dataset_tester.load_all_configs(dataset_name, is_local=True)[:1]
>       self.dataset_tester.check_load_dataset(dataset_name, configs, is_local=True, use_local_dummy_data=True)

tests/test_dataset_common.py:234: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_dataset_common.py:187: in check_load_dataset
    self.parent.assertTrue(len(dataset[split]) > 0)
E   AssertionError: False is not true
```
When I try loading dataset on local machine it works fine. Any suggestions on how can I avoid this error?

Thanks for the help, @albertvillanova! All tests are passing now.
Issue: How to sample every file in a list of files making up a split in a dataset when loading?
State: closed
Author: brijow

I am loading a dataset with multiple train, test, and validation files like this:

```
data_files_dict = {
    "train": [train_file1, train_file2],
    "test": [test_file1, test_file2],
    "val": [val_file1, val_file2]
}
dataset = datasets.load_dataset(
    "csv",
    data_files=data_files_dict,
    split=['train[:8]', 'test[:8]', 'val[:8]']
)

```

However, this only selects the first 8 rows from train_file1, test_file1, val_file1, since they are the first files in the lists.

I'm trying to formulate a split argument that can sample from each file specified in my list of files that make up each split.

Is this type of splitting supported? If so, how can I do it?

Comments:
Hi ! This is not possible just with `load_dataset`.

You can do something like this instead:
```python
seed=42
data_files_dict = {
    "train": [train_file1, train_file2],
    "test": [test_file1, test_file2],
    "val": [val_file1, val_file2]
}
dataset = datasets.load_dataset(
    "csv",
    data_files=data_files_dict,
).shuffle(seed=seed)

sample_dataset = {splitname: split.select(range(8)) for splitname, split in dataset.items()}
```

Another alternative is loading each file separately with `split="train[:8]"` and then use `concatenate_datasets` to merge the sample of each file.
Issue: Add KS task to SUPERB
State: closed
Author: anton-l

Add the KS (keyword spotting) task as described in the [SUPERB paper](https://arxiv.org/abs/2105.01051).

- [s3prl instructions](https://github.com/s3prl/s3prl/blob/master/s3prl/downstream/README.md#ks-keyword-spotting)
- [s3prl implementation](https://github.com/s3prl/s3prl/blob/master/s3prl/downstream/speech_commands/dataset.py)
- [TFDS implementation](https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/audio/speech_commands.py)

Some notable quirks:
- The dataset is originally single-archive (train+val+test all in one), but the test set has a "canonical" distribution in a separate archive, which is also used here (see `_split_ks_files()`). 
- The `_background_noise_`/`_silence_` audio files are much longer than others, so they require some sort of slicing for downstream training. I decided to leave the implementation of that up to the users, since TFDS and s3prl take different approaches (either slicing wavs deterministically, or subsampling randomly at runtime)

Related to #2619.

Comments:
thanks a lot for implementing this @anton-l !!

i won't have time to review this while i'm away, so happy for @albertvillanova and @patrickvonplaten to decide when to merge :)

@albertvillanova thanks! Everything should be ready now :)

> The _background_noise_/_silence_ audio files are much longer than others, so they require some sort of slicing for downstream training. I decided to leave the implementation of that up to the users, since TFDS and s3prl take different approaches (either slicing wavs deterministically, or subsampling randomly at runtime)

@anton-l I was thinking that maybe we could give some hints in the dataset card (in a Usage section); something similar as for diarization: https://github.com/huggingface/datasets/blob/master/datasets/superb/README.md#example-of-usage
Note that for diarization it is not yet finished: we have to test it and then provide an end-to-end example: https://github.com/huggingface/datasets/pull/2661/files#r680224909 

@albertvillanova yeah, I'm not sure how to best implement it in pure `datasets` yet. It's something like this, where `sample_noise()` needs to be called from a pytorch batch collator or other framework-specific variant:

```python
def map_to_array(example):
    import soundfile as sf

    speech_array, sample_rate = sf.read(example["file"])
    example["speech"] = speech_array
    example["sample_rate"] = sample_rate
    return example


def sample_noise(example):
    # Use a version of this function in a stateless way to extract random 1 sec slices of background noise
    # on each epoch
    from random import randint

    # _silence_ audios are longer than 1 sec
    if example["label"] == "_silence_":
        random_offset = randint(0, len(example["speech"]) - example["sample_rate"] - 1)
        example["speech"] = example["speech"][random_offset : random_offset + example["sample_rate"]]

    return example
```

I see... Yes, not trivial indeed. Maybe for the moment you could add those functions above to the README (as it is the case for now in diarization)? What do you think?
Issue: Fix renaming of corpus_bleu args
State: closed
Author: albertvillanova

Last `sacrebleu` release (v2.0.0) has renamed `sacrebleu.corpus_bleu` args from `(sys_stream, ref_streams)` to `(hipotheses, references)`: https://github.com/mjpost/sacrebleu/pull/152/files#diff-2553a315bb1f7e68c9c1b00d56eaeb74f5205aeb3a189bc3e527b122c6078795L17-R15

This PR passes the args without parameter names, so that it is valid for all versions of `sacrebleu`.

This is a partial hotfix of #2781.

Close #2781.

Comments:
Issue: Latest v2.0.0 release of sacrebleu has broken some metrics
State: closed
Author: albertvillanova

## Describe the bug
After `sacrebleu` v2.0.0 release (see changes here: https://github.com/mjpost/sacrebleu/pull/152/files#diff-2553a315bb1f7e68c9c1b00d56eaeb74f5205aeb3a189bc3e527b122c6078795L17-R15), some of `datasets` metrics are broken:
- Default tokenizer `sacrebleu.DEFAULT_TOKENIZER` no longer exists:
  - #2739
  - #2778
- Bleu tokenizers are no longer accessible with `sacrebleu.TOKENIZERS`:
  - #2779
- `corpus_bleu` args have been renamed from `(sys_stream, ref_streams)` to `(hipotheses, references)`: 
  - #2782 

Comments:
Issue: Fix sacrebleu tokenizers
State: closed
Author: albertvillanova

Last `sacrebleu` release (v2.0.0) has removed `sacrebleu.TOKENIZERS`: https://github.com/mjpost/sacrebleu/pull/152/files#diff-2553a315bb1f7e68c9c1b00d56eaeb74f5205aeb3a189bc3e527b122c6078795L17-R15

This PR makes a hot fix of the bug by using a private function in `sacrebleu`: `sacrebleu.metrics.bleu._get_tokenizer()`.

Eventually, this should be further fixed in order to use only public functions.

This is a partial hotfix of #2781.

Comments:
Issue: Do not pass tokenize to sacrebleu
State: closed
Author: albertvillanova

Last `sacrebleu` release (v2.0.0) has removed `sacrebleu.DEFAULT_TOKENIZER`: https://github.com/mjpost/sacrebleu/pull/152/files#diff-2553a315bb1f7e68c9c1b00d56eaeb74f5205aeb3a189bc3e527b122c6078795L17-R15

This PR does not pass `tokenize` to `sacrebleu` (note that the user cannot pass it anyway) and `sacrebleu` will use its default, no matter where it is and how it is called.

Related to #2739.

This is a partial hotfix of #2781.

Comments:
Issue: document `config.HF_DATASETS_OFFLINE` and precedence
State: open
Author: stas00

https://github.com/huggingface/datasets/pull/1976 implemented `HF_DATASETS_OFFLINE`, but:
1. `config.HF_DATASETS_OFFLINE` is not documented
2. the precedence is not documented (env, config)

I'm thinking it probably should be similar to what it says https://huggingface.co/docs/datasets/loading_datasets.html#from-the-huggingface-hub about `datasets.config.IN_MEMORY_MAX_SIZE`:

Quote:
> The default in 🤗 Datasets is to memory-map the dataset on disk unless you set datasets.config.IN_MEMORY_MAX_SIZE different from 0 bytes (default). In that case, the dataset will be copied in-memory if its size is smaller than datasets.config.IN_MEMORY_MAX_SIZE bytes, and memory-mapped otherwise. This behavior can be enabled by setting either the configuration option datasets.config.IN_MEMORY_MAX_SIZE (higher precedence) or the environment variable HF_DATASETS_IN_MEMORY_MAX_SIZE (lower precedence) to nonzero.

Context: trying to use `config.HF_DATASETS_OFFLINE` here:
https://github.com/bigscience-workshop/Megatron-DeepSpeed/pull/48
but are uncertain if it's safe, since it's not documented as a public API.

Thank you!

@lhoestq, @albertvillanova 

Comments:
Issue: Remove dataset_infos.json
State: closed
Author: albertvillanova

**Is your feature request related to a problem? Please describe.**
As discussed, there are infos in the `dataset_infos.json` which are redundant and we could have them only in the README file.

Others could be migrated to the README, like: "dataset_size", "size_in_bytes", "download_size", "splits.split_name.[num_bytes, num_examples]",...

However, there are others that do not seem too meaningful in the README, like the checksums.

**Describe the solution you'd like**
Open a discussion to decide what to do with the `dataset_infos.json` files: which information to be migrated and/or which information to be kept.

cc: @julien-c @lhoestq 

Comments:
This was closed by:
- #4926
Issue: Remove returned feature constrain
State: open
Author: PosoSAgapo

In the current version, the returned value of the map function has to be list or ndarray. However, this makes it unsuitable for many tasks. In NLP, many features are sparse like verb words, noun chunks, if we want to assign different values to different words, which will result in a large sparse matrix if we only score useful words like verb words.  

Mostly, when using it on large scale, saving it as a whole takes a lot of disk storage and making it hard to read,  the normal method is saving it in sparse form. However, the NumPy does not support sparse, therefore I have to use PyTorch or scipy to transform a matrix into special sparse form, which is not a form that can be transformed into list or ndarry. This violates the feature constraints of the map function. 

I do appreciate the convenience of Datasets package, but I do not think the compulsory datatype constrain is necessary, in some cases, we just cannot transform it into a list or ndarray due to some reasons.  Any way to fix this? Or what I can do to disable the compulsory datatype constrain?


Comments:
Issue: BERTScore Error
State: closed
Author: gagan3012

## Describe the bug
A clear and concise description of what the bug is.

## Steps to reproduce the bug
```python
predictions = ["hello there", "general kenobi"]
references = ["hello there", "general kenobi"]
bert = load_metric('bertscore')
bert.compute(predictions=predictions, references=references,lang='en')
```

# Bug
`TypeError: get_hash() missing 1 required positional argument: 'use_fast_tokenizer'`


## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform: Colab 
- Python version:
- PyArrow version:


Comments:
Hi,

The `use_fast_tokenizer` argument has been recently added to the bert-score lib. I've opened a PR with the fix. In the meantime, you can try to downgrade the version of bert-score with the following command to make the code work:
```
pip uninstall bert-score
pip install "bert-score<0.3.10"
```
Issue: Add RVL-CDIP dataset
State: closed
Author: NielsRogge

## Adding a Dataset
- **Name:** RVL-CDIP
- **Description:** The RVL-CDIP (Ryerson Vision Lab Complex Document Information Processing) dataset consists of 400,000 grayscale images in 16 classes, with 25,000 images per class. There are 320,000 training images, 40,000 validation images, and 40,000 test images. The images are sized so their largest dimension does not exceed 1000 pixels.
- **Paper:** https://www.cs.cmu.edu/~aharley/icdar15/
- **Data:** https://www.cs.cmu.edu/~aharley/rvl-cdip/
- **Motivation:** I'm currently adding LayoutLMv2 and LayoutXLM to HuggingFace Transformers. LayoutLM (v1) already exists in the library. This dataset has a large value for document image classification (i.e. classifying scanned documents). LayoutLM models obtain SOTA on this dataset, so would be great to directly use it in notebooks.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Comments:
cc @nateraw 

#self-assign

[labels_only.tar.gz](https://docs.google.com/uc?authuser=0&id=0B0NKIRwUL9KYcXo3bV9LU0t3SGs&export=download) on the RVL-CDIP website does not work for me.

> 404. That’s an error. The requested URL was not found on this server.

I contacted the author ( Adam Harley) regarding this, and he told me that the link works for him. Not sure what the issue is. But Adam shared the file (labels_only.tar.gz) with me as an attachment.

Are we allowed to host this file(labels_only.tar.gz) elsewhere and use that link instead ?

Thank you.
Issue: Add Nuswide dataset
State: open
Author: shivangibithel

## Adding a Dataset
- **Name:** *NUSWIDE*
- **Description:** *[A Real-World Web Image Dataset from National University of Singapore](https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/NUS-WIDE.html)*
- **Paper:** *[here](https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/nuswide-civr2009.pdf)*
- **Data:** *[here](https://github.com/wenting-zhao/nuswide)*
- **Motivation:** *This dataset is a benchmark in the Text Retrieval task.*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Comments: