rachel521/github-issues-simple huggingface dataset:
Non-deterministic `Dataset.builder_name` value
Opened by harupy (closed)

### Describe the bug

I'm not sure if this is a bug, but `print(ds.builder_name)` in the following code sometimes prints out `rotten_tomatoes` instead of `parquet`:

```python
import datasets

for _ in range(100):
    ds = datasets.load_dataset("rotten_tomatoes", split="train")
    print(ds.builder_name)  # prints out "rotten_tomatoes" sometimes instead of "parquet"
```

Output:

```
...
parquet
parquet
parquet
rotten_tomatoes
parquet
parquet
parquet
...
```

Here's a reproduction using GitHub Actions:
https://github.com/mlflow/mlflow/actions/runs/8153247984/job/22284263613?pr=11329#step:12:241

One of our tests is flaky because `builder_name` is not deterministic.

### Steps to reproduce the bug

1. Run the code above.

### Expected behavior

Always prints out `parquet`?

### Environment info

```
Copy-and-paste the text below in your GitHub issue.
- `datasets` version: 2.18.0
- Platform: Linux-6.5.0-1015-azure-x86_64-with-glibc2.34
- Python version: 3.8.18
- `huggingface_hub` version: 0.21.3
- PyArrow version: 15.0.0
- Pandas version: 2.0.3
- `fsspec` version: 2024.2.0
```

Comments: When `rotten_tomatoes` is printed out, the following warning message is also printed out:

```
You can avoid this message in future by passing the argument `trust_remote_code=True`.
Passing `trust_remote_code=True` will be mandatory to load this dataset from the next major release of `datasets`.
```

Hi ! This behavior happens because the dataset was originakky created using a dataset script [rotten_tomatoes.py](https://huggingface.co/datasets/rotten_tomatoes/blob/26f40d324d7b281d8b3fb1c47f30f8b9957f206b/rotten_tomatoes.py) and because we added features recently allowing to download the dataset directly from Parquet files (parquet builder) without running the dataset script (rotten_tomatoes). The flakiness must come from the availability of the Parquet files (we automatically export them in the refs/convert/parquet branch and we recently had to move some files).

Anyway the easy fix on our side is to remove the dataset script completely, let me open a PR at https://huggingface.co/datasets/rotten_tomatoes

EDIT: opened https://huggingface.co/datasets/rotten_tomatoes/discussions/6, feel free to comment there if you're ok with that change

@lhoestq Thanks for the comment, explanation, and patch!

> we automatically export them in the refs/convert/parquet branch

When this operation is in progress, the parquet files become temporarily unavailable?

> When this operation is in progress, the parquet files become temporarily unavailable?

Yes correct. I just merged the patch btw :)

@lhoestq Thanks for merging the PR! I think this issue can be closed.
Fix task template reload from dict
Opened by lhoestq (closed)

Since #4926 the JSON dumps are simplified and it made task template dicts empty by default.

I fixed this by always including the task name which is needed to reload a task from a dict

Comments: _The documentation is not available anymore as the PR was closed or merged._

> Just wondering if there might be other data classes default values missed that could cause an issue... Apart from feature-like classes and tasks, I don't see any others though...

I think we're good ! `asdict` is used on the DatasetInfo attributes like features, tasks etc. and they all support dict conversion properly now

> And a question: but this information about the tasks is no longer being saved as YAML tags in the dataset card; won't be a problem with current datasets using task templates (with this information in their metadata JSON) once we replace the JSON by the YAML tags (which do not have this information about the task templates)?

In the long run we'll use the train_eval_index YAML tags instead, but I agree when removing the JSON files we should try to not break existing code that may rely on this
Prioritize `module.builder_kwargs` over defaults in `TestCommand`
Opened by lvwerra (closed)

This fixes a bug in the `TestCommand` where multiple kwargs for `name` were passed if it was set in both default and `module.builder_kwargs`. Example error:

```Python
Traceback (most recent call last):
  File "create_metadata.py", line 96, in <module>
    main(**vars(args))
  File "create_metadata.py", line 86, in main
    metadata_command.run()
  File "/opt/conda/lib/python3.7/site-packages/datasets/commands/test.py", line 144, in run
    for j, builder in enumerate(get_builders()):
  File "/opt/conda/lib/python3.7/site-packages/datasets/commands/test.py", line 141, in get_builders
    name=name, cache_dir=self._cache_dir, data_dir=self._data_dir, **module.builder_kwargs
TypeError: type object got multiple values for keyword argument 'name'
```

Let me know what you think.
#3337 Add typing overloads to Dataset.__getitem__ for mypy
Opened by Dref360 (closed)

Add typing overloads to Dataset.__getitem__ for mypy

Fixes #3337 

**Iterable**
Iterable from `collections` cannot have a type, so you can't do `Iterable[int]` for example. `typing` has a Generic version that builds upon the one from `collections`.

**Flake8**
I had to add `# noqa: F811`, this is a bug from Flake8.

datasets uses flake8==3.7.9 which released in October 2019 if I update flake8 (4.0.1), I no longer get these errors, but I did not want to make the update without your approval. (It also triggers other errors like no args in f-strings.)

Comments: Locally the `make quality` passes with the same dependencies. I would suggest upgrading flake8. (I can take care of it in another PR)
cc @lhoestq 

Thank you for fixing flake8! I think we are ready to merge then.
Streaming not supported in Theivaprakasham/wildreceipt
Opened by NitishkKarra (closed)

### Link

_No response_

### Description

_No response_

### Owner

_No response_

Comments: Thanks for reporting @NitishkKarra.

The root source of the issue is that streaming mode is not supported out-of-the-box for that dataset, because it contains a TAR file.

We have opened a discussion in the corresponding Hub dataset page, pointing out this issue: https://huggingface.co/datasets/Theivaprakasham/wildreceipt/discussions/1

I'm closing this issue here, so this discussion is transferred there instead.
Add SubjQA dataset
Opened by lewtun (closed)

Hello datasetters 🙂!

Here's an interesting dataset about extractive question-answering on _subjective_ product / restaurant reviews. It's quite challenging for models fine-tuned on SQuAD and provides a nice example of domain adaptation (i.e. fine-tuning a SQuAD model on this domain gives better performance).

I found a bug in the start/end indices that I've proposed a fix for here: https://github.com/megagonlabs/SubjQA/pull/2

Unfortunately, the dataset creators are unresponsive, so for now I am using my fork as the source. Will update the URL if/when the creators respond.

Comments: I'm not sure why the windows test fails, but looking at the logs it looks like some caching issue on one of the metrics ... maybe re-run and 🤞 ?

Hi @lewtun, thanks for adding this dataset!

If the dataset is going to be referenced heavily, I think it's worth spending some time to make the dataset card really great :) To start, the information that is currently in the `Data collection` paragraph should probably be organized in the `Dataset Creation` section.

Here's a link to the [relevant section of the guide](https://github.com/huggingface/datasets/blob/master/templates/README_guide.md#dataset-creation), let me know if you have any questions!

> If the dataset is going to be referenced heavily, I think it's worth spending some time to make the dataset card really great :) To start, the information that is currently in the `Data collection` paragraph should probably be organized in the `Dataset Creation` section.

great idea @yjernite! i've added some extra information / moved things as you suggest and will wrap up the rest tomorrow :)

hi @yjernite and @lhoestq, i've fleshed out the dataset card and think this is now ready for another round of review!
Add QuALITY dataset
Opened by lewtun (open)

## Adding a Dataset
- **Name:** QuALITY
- **Description:** A challenging question answering with very long contexts (Twitter [thread](https://twitter.com/sleepinyourhat/status/1471225421794529281?s=20))
- **Paper:** No ArXiv link yet, but draft is [here](https://github.com/nyu-mll/quality/blob/main/quality_preprint.pdf)
- **Data:** GitHub repo [here](https://github.com/nyu-mll/quality)
- **Motivation:** This dataset would serve as a nice way to benchmark long-range Transformer models like BigBird, Longformer and their descendants. In particular, it would be very interesting to see how the S4 model fares on this given it's impressive performance on the Long Range Arena

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Comments: I'll take this one if no one hasn't yet!
how can I combine 2 dataset with different/same features?
Opened by world2vec (closed)

to combine 2 dataset by one-one map like ds = zip(ds1, ds2):
ds1: {'text'}, ds2: {'text'}, combine ds:{'src', 'tgt'} 
or different feature:
ds1: {'src'}, ds2: {'tgt'}, combine ds:{'src', 'tgt'}

Comments: Hi ! Currently we don't have a way to `zip` datasets but we plan to add this soon :)
For now you'll need to use `map` to add the fields from one dataset to the other. See the comment here for more info : https://github.com/huggingface/datasets/issues/853#issuecomment-727872188

Good to hear.
Currently I did not use map , just fetch src and tgt from the 2 dataset and merge them.
It will be a release if you can deal with it at the backend.
Thanks.

Hi! You can rename the columns and concatenate the datasets along `axis=1` to get the desired result as follows:
```python
ds1 = ds1.rename_column("text", "src")
ds2 = ds2.rename_column("text", "tgt")
ds = datasets.concatenate_datasets(["ds1", "ds2"], axis=1)
```
CUDA out of memory
Opened by li1117heex (closed)

In your dataset ,cuda run out of memory as long as the trainer begins:
however, without changing any other element/parameter,just switch dataset to `LineByLineTextDataset`,everything becames OK.


Comments: Could you provide more details ? What's the code you ran ?

```python
tokenizer = FunnelTokenizer.from_pretrained('funnel-transformer/small')

def tokenize(batch):
    return tokenizer(batch['text'], padding='max_length', truncation=True,max_length=512)

dataset = load_dataset("bookcorpus",split='train[:1000]').shuffle()
dataset = dataset.map(tokenize, batched=True, batch_size=512)

# dataset = LineByLineTextDataset(
#     tokenizer=tokenizer,
#     file_path="./wiki1000.txt",
#     block_size=128
# )

data_collator = DataCollatorForLanguageModeling(
    tokenizer=tokenizer, mlm=True, mlm_probability=0.15
)

config=FunnelConfig(
    return_dict=True
)

model= FunnelForMaskedLM(config=config)

training_args = TrainingArguments(
    output_dir="./checkpoints",
    overwrite_output_dir=True,
    do_train=True,
    num_train_epochs=1,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    save_steps=10000,
    logging_dir='./ptlogs'
)

trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=dataset,
)
trainer.train()
```

`RuntimeError: CUDA out of memory. Tried to allocate 954.00 MiB (GPU 0; 15.90 GiB total capacity; 14.35 GiB already allocated; 753.75 MiB free; 14.39 GiB reserved in total by PyTorch)
Exception raised from malloc at /pytorch/c10/cuda/CUDACachingAllocator.cpp:272 (most recent call first):`

part of error output

from funnel model to bert model : error still happened

from your dataset to LineByLineTextDataset : error disapeared

notice i just loaded 1000 rows of data

the error happens when executing loss.backward()

Since you're using a data collator you don't need to tokenizer the dataset using `map`. Could you try not to use `map` and only the data collator instead ? The data collator is supposed to pad to the longest sequence in each batch afaik, instead of padding to 512.

Also cc @sgugger 

Closing this one.
Feel free to re-open if you have other questions about this issue
Add oversampling strategy iterable datasets interleave
Opened by ylacombe (closed)

Hello everyone,
Following the issue #4893 and the PR #4831, I propose here an oversampling strategy for a `IterableDataset` list.  
The `all_exhausted` strategy stops building the new dataset as soon as all samples in each dataset have been added at least once. 
It follows roughly the same logic behind #4831, namely:
- if ``probabilities`` is `None` and the strategy is `all_exhausted`, it simply performs a round robin interleaving that stops when the longest dataset is out of samples. Here the new dataset length will be $maxLengthDataset*nbDataset$.
- if ``probabilities`` is not `None` and the strategy is `all_exhausted`, it keeps trace of the datasets which were out of samples but continues to add them to the new dataset, and stops as soons as every dataset runs out of samples at least once.

In order to be consistent and also to align with the `Dataset` behavior, please note that the behavior of the default strategy (`first_exhausted`) has been changed. Namely, it really stops when a dataset is out of samples whereas it used to stop when receiving the `StopIteration` error.  
To give an example of the last note, with the following snippet:
```
>>> from tests.test_iterable_dataset import *
>>> d1 = IterableDataset(ExamplesIterable((lambda: (yield from [(i, {"a": i}) for i in [0, 1, 2]])), {}))
>>> d2 = IterableDataset(ExamplesIterable((lambda: (yield from [(i, {"a": i}) for i in [10, 11, 12, 13]])), {}))
>>> d3 = IterableDataset(ExamplesIterable((lambda: (yield from [(i, {"a": i}) for i in [20, 21, 22, 23, 24]])), {}))
>>> dataset = interleave_datasets([d1, d2, d3])
>>> [x["a"] for x in dataset]
```
The result here will then be `[10, 0, 11, 1, 2]` instead of `[10, 0, 11, 1, 2, 20, 12, 13]`.

I modified the behavior because I found it to be consistent with the under/oversampling approach and because it unified the undersampling and oversampling code, but I stay open to any suggestions.


Comments: _The documentation is not available anymore as the PR was closed or merged._
`sort` after `filter` unreasonably  slow
Opened by Tobin-rgb (closed)

### Describe the bug

as the tittle says ...

### Steps to reproduce the bug

`sort` seems to be normal.

```python
from datasets import Dataset
import random

nums = [{"k":random.choice(range(0,1000))} for _ in range(100000)]

ds = Dataset.from_list(nums)

print("start sort")

ds = ds.sort("k")

print("finish sort")
```

but `sort` after `filter` is extremely slow.

```python
from datasets import Dataset
import random

nums = [{"k":random.choice(range(0,1000))} for _ in range(100000)]

ds = Dataset.from_list(nums)

ds = ds.filter(lambda x:x > 100, input_columns="k")

print("start sort")

ds = ds.sort("k")

print("finish sort")
```


### Expected behavior

Is this a bug, or is it a misuse of the `sort` function?

### Environment info

- `datasets` version: 2.20.0
- Platform: Linux-3.10.0-1127.19.1.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.10.13
- `huggingface_hub` version: 0.23.4
- PyArrow version: 16.1.0
- Pandas version: 2.2.2
- `fsspec` version: 2023.10.0

Comments: `filter` add an indices mapping on top of the dataset, so `sort` has to gather all the rows that are kept to form a new Arrow table and sort the table. Gathering all the rows can take some time, but is a necessary step. You can try calling `ds = ds.flatten_indices()` before sorting to remove the indices mapping.

> `filter` add an indices mapping on top of the dataset, so `sort` has to gather all the rows that are kept to form a new Arrow table and sort the table. Gathering all the rows can take some time, but is a necessary step. You can try calling `ds = ds.flatten_indices()` before sorting to remove the indices mapping.

This worked, thank you so much.
[Tests] add slow tests
Opened by patrickvonplaten (closed)

This PR adds a slow test that downloads the "real" dataset. The test is decorated as "slow" so that it will not automatically run on circle ci.

Before uploading a dataset, one should test that this test passes, manually by running 

```
RUN_SLOW=1 pytest tests/test_dataset_common.py::DatasetTest::test_load_real_dataset_<your-dataset-script-name>
```

This PR should be merged after PR: #33
Couldn't reach CNN/DM dataset
Opened by astariul (closed)

I can't get CNN / DailyMail dataset.

```python
import nlp

assert "cnn_dailymail" in [dataset.id for dataset in nlp.list_datasets()]
cnn_dm = nlp.load_dataset('cnn_dailymail')
```

[Colab notebook](https://colab.research.google.com/drive/1zQ3bYAVzm1h0mw0yWPqKAg_4EUlSx5Ex?usp=sharing)

gives following error :

```
ConnectionError: Couldn't reach https://s3.amazonaws.com/datasets.huggingface.co/nlp/cnn_dailymail/cnn_dailymail.py
```

Comments: Installing from source (instead of Pypi package) solved the problem.
WIP: drop labels in Image and Audio folders by default
Opened by polinaeterna (closed)

will fix https://github.com/huggingface/datasets/issues/5153 and redundant labels displaying for most of the images datasets on the Hub (which are used just to store files)

TODO: discuss adding `drop_labels` (and `drop_metadata`) params to yaml

Comments: _The documentation is not available anymore as the PR was closed or merged._

close in favor of https://github.com/huggingface/datasets/pull/5192
Adding initial version of cord-19 dataset
Opened by ggdupont (closed)

Initial version only reading the metadata in CSV.

### Checklist:
- [x] Create the dataset script /datasets/my_dataset/my_dataset.py using the template
- [x] Fill the _DESCRIPTION and _CITATION variables
- [x] Implement _infos(), _split_generators() and _generate_examples()
- [x] Make sure that the BUILDER_CONFIGS class attribute is filled with the different configurations of the dataset and that the BUILDER_CONFIG_CLASS is specified if there is a custom config class.
- [x] Generate the metadata file dataset_infos.json for all configurations
- [x] Generate the dummy data dummy_data.zip files to have the dataset script tested and that they don't weigh too much (<50KB)
- [x] Add the dataset card README.md using the template and at least fill the tags
- [x] Both tests for the real data and the dummy data pass.

### TODO:
- [x] add more metadata
- [x] add full text
- [x] add pre-computed document embedding

Comments: Hi @ggdupont !
Have you had a chance to take a look at my suggestions ?
Feel free to ping me if you have questions or when you're ready for a review

> Hi @ggdupont !
> Have you had a chance to take a look at my suggestions ?
> Feel free to ping me if you have questions or when you're ready for a review

Yes I did, just busy period (and no time on weekend right now ;-) )

With some delay, reduced the dummy data and had t rebase

Thanks !

It looks like the rebase messed up the github diff for this PR (2.000+ files changed)
Could you create another branch and another PR please ?

Cleaned PR: https://github.com/huggingface/datasets/pull/1850
[Feature request] Export Arrow dataset to TFRecords
Opened by jarednielsen (closed)

The TFRecord generation process is error-prone and requires complex separate Python scripts to download and preprocess the data. I propose to combine the user-friendly features of `nlp` with the speed and efficiency of TFRecords. Sample API:

```python
# use these existing methods
ds = load_dataset("wikitext", "wikitext-2-raw-v1", split="train")
ds = ds.map(lambda ex: tokenizer(ex))
ds.set_format("tensorflow", columns=["input_ids", "token_type_ids", "attention_mask"])
# then add this method
ds.export(folder="/my/tfrecords", prefix="myrecord", num_shards=8, format="tfrecord")
```
which would create files like so:
```bash
/my/tfrecords/myrecord_1.tfrecord
/my/tfrecords/myrecord_2.tfrecord
...
```

I would be happy to contribute this method. We could use a similar approach for PyTorch. Thoughts?
Support multiple image/audio columns in ImageFolder/AudioFolder
Opened by severo (closed)

### Feature request

Have a metadata.csv file with multiple columns that point to relative image or audio files.

### Motivation

Currently, ImageFolder allows one column, called `file_name`, pointing to relative image files. On the same model, AudioFolder allows one column, called `file_name`, pointing to relative audio files.

But it's not possible to have two image columns, or to have two audio column, or to have one audio column and one image column.

### Your contribution

no specific contribution

Comments: A duplicate of https://github.com/huggingface/datasets/issues/5760
Better cached path
Opened by lhoestq (closed)

### Changes:
- The `cached_path` no longer returns None if the file is missing/the url doesn't work. Instead, it can raise `FileNotFoundError` (missing file), `ConnectionError` (no cache and unreachable url) or `ValueError` (parsing error)
- Fix requests to firebase API that doesn't handle HEAD requests...
- Allow custom download in datasets script: it allows to use `tf.io.gfile.copy` for example, to download from google storage. I added an example: the `boolq` script

Comments: I should have read this PR before doing my own: https://github.com/huggingface/nlp/pull/62 :D 
will close mine. Looks great :-) 

> Awesome, this is really nice!
> 
> By the way, we should improve the `cached_path` method of the `transformers` repo similarly, don't you think (@patrickvonplaten in particular).

Yeah, we should do the same in `transformers` I think - will note it down.
Missing google file for `multi_news` dataset
Opened by severo (closed)

## Dataset viewer issue for '*multi_news*'

**Link:** https://huggingface.co/datasets/multi_news

```
Server error

Status code:   400
Exception:     FileNotFoundError
Message:       https://drive.google.com/uc?export=download&id=1vRY2wM6rlOZrf9exGTm5pXj5ExlVwJ0C/multi-news-original/train.src
```

Am I the one who added this dataset ? No


Comments: reported by @abidlabs 

related to https://github.com/huggingface/datasets/pull/3843?

`datasets` 1.18.4 fixes the issue when you load the dataset with `load_dataset`.

When loading in streaming mode, the fix is indeed on https://github.com/huggingface/datasets/pull/3843 which will be merged soon :)

That is. The PR #3843 was just opened a bit later we had made our 1.18.4 patch release...
Once merged, that will fix this issue. 

OK. Should fix the viewer for 50 datasets

<img width="148" alt="Capture d’écran 2022-03-14 à 11 51 02" src="https://user-images.githubusercontent.com/1676121/158157853-6c544a47-2d6d-4ac4-964a-6f10951ec36b.png">