rachel521/github-issues-simple huggingface dataset:
Improve `get_dataset_infos`?
Opened by severo (closed)

Using the dedicated function `get_dataset_infos` on a dataset that has no dataset-info.json file returns an empty info:

```
>>> from datasets import get_dataset_infos
>>> get_dataset_infos('wit')
{}
```

While it's totally possible to get it (regenerate it) with:

```
>>> from datasets import load_dataset_builder
>>> builder = load_dataset_builder('wit')
>>> builder.info
DatasetInfo(description='Wikipedia-based Image Text (WIT) Dataset is a large multimodal multilingual dataset. WIT is composed of a curated set\n of 37.6 million entity rich image-text examples with 11.5 million unique images across 108 Wikipedia languages. Its\n size enables WIT to be used as a pretraining dataset for multimodal machine learning models.\n', citation='@article{srinivasan2021wit,\n  title={WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning},\n  author={Srinivasan, Krishna and Raman, Karthik and Chen, Jiecao and Bendersky, Michael and Najork, Marc},\n  journal={arXiv preprint arXiv:2103.01913},\n  year={2021}\n}\n', homepage='https://github.com/google-research-datasets/wit', license='', features={'b64_bytes': Value(dtype='string', id=None), 'embedding': Sequence(feature=Value(dtype='float64', id=None), length=-1, id=None), 'image_url': Value(dtype='string', id=None), 'metadata_url': Value(dtype='string', id=None), 'original_height': Value(dtype='int32', id=None), 'original_width': Value(dtype='int32', id=None), 'mime_type': Value(dtype='string', id=None), 'caption_attribution_description': Value(dtype='string', id=None), 'wit_features': Sequence(feature={'language': Value(dtype='string', id=None), 'page_url': Value(dtype='string', id=None), 'attribution_passes_lang_id': Value(dtype='string', id=None), 'caption_alt_text_description': Value(dtype='string', id=None), 'caption_reference_description': Value(dtype='string', id=None), 'caption_title_and_reference_description': Value(dtype='string', id=None), 'context_page_description': Value(dtype='string', id=None), 'context_section_description': Value(dtype='string', id=None), 'hierarchical_section_title': Value(dtype='string', id=None), 'is_main_image': Value(dtype='string', id=None), 'page_changed_recently': Value(dtype='string', id=None), 'page_title': Value(dtype='string', id=None), 'section_title': Value(dtype='string', id=None)}, length=-1, id=None)}, post_processed=None, supervised_keys=None, task_templates=None, builder_name='wit', config_name='default', version=0.0.0, splits=None, download_checksums=None, download_size=None, post_processing_size=None, dataset_size=None, size_in_bytes=None)
```

Should we test if info is empty, and in that case regenerate it? Or always generate it?

Comments: To keeps things simple maybe we should use `load_dataset_builder` in `get_dataset_infos`.
`load_dataset_builder` instantiates a builder and runs the _infos() method in order to give you the most up-to-date infos, even if the dataset_infos.json is outdated or missing.
Can't Read List of JSON Files Properly
Opened by lordsoffallen (open)

### Describe the bug

Trying to read a bunch of JSON files into Dataset class but default approach doesn't work. I don't get why it works when I read it one by one but not when I pass as a list :man_shrugging: 

The code fails with 

```
ArrowInvalid: JSON parse error: Invalid value. in row 0  
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte 
DatasetGenerationError: An error occurred while generating the dataset
```

### Steps to reproduce the bug

This doesn't work

```
from datasets import Dataset

# dir contains 100 json files.
Dataset.from_json("/PUT SOME PATH HERE/*")
```

This works:
```
from datasets import concatenate_datasets

ls_ds = []

for file in list_of_json_files:
    ls_ds.append(Dataset.from_json(file))

ds = concatenate_datasets(ls_ds)

```





### Expected behavior

I expect this to read json files properly as error is not clear

### Environment info

- `datasets` version: 2.17.0
- Platform: Linux-6.5.0-15-generic-x86_64-with-glibc2.35
- Python version: 3.10.13
- `huggingface_hub` version: 0.20.2
- PyArrow version: 15.0.0
- Pandas version: 2.2.0
- `fsspec` version: 2023.10.0


Comments: Found the issue, if there are other files in the directory, it gets caught into this `*` so essentially it should be `*.json`. Could we possibly to check for list of files to make sure the pattern matches json files and raise error if not?

I don't think we should filter for `*.json` as this might silently remove desired files for many users. And this could be a major breaking change for many organizations.

You could do the globbing yourself which would keep the code clean.

```python
from glob import glob

Dataset.from_json(glob('folder/*.json'))
```

I think it should still be fine to log a warning message in case the folder contains different files? I also don't get why would this be breaking as in the end using `from_FILE_TYPE` should be able to read a specific file type only. Maybe some other use case I am not aware of but since globbing or this case not mentioned anywhere in the doc, I spent quite a bit of time trying to figure out where the issue was. Just making sure it's clear for users.
Update loading.mdx to include "jsonl" file loading.
Opened by mosheber (open)

* A small update to the documentation, noting the ability to load jsonl files. 

Comments: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_6647). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Thanks for adding the explicit loading command.
> 
> However, I would move it just below, where we present the JSON-Lines example.
> 
> * Maybe adding that this format is called JSON-Lines
> * Add the example after the JSON-Lines data example
> 
> https://github.com/huggingface/datasets/blob/14d9afbb7ae1b787c450261ca0ff374551993031/docs/source/loading.mdx#L135-L138

Thank you @albertvillanova for the feedback! I moved the jsonl file loading example to a more appropriate location.
Dataset.from_generator consistently freezes at ~1000 rows
Opened by andreemic (closed)

### Describe the bug

Whenever I try to create a dataset which contains images using `Dataset.from_generator`, it freezes around 996 rows. I suppose it has something to do with memory consumption, but there's more memory available. I

Somehow it worked a few times but mostly this makes the datasets library much more cumbersome to work with because generators are the easiest way to turn an existing dataset into a Hugging Face dataset.

I've let it run in the frozen state for way longer than it can possibly take to load the actual dataset.

Let me know if you have ideas how to resolve it!

### Steps to reproduce the bug

```python
from datasets import Dataset
import numpy as np

def gen():
    for row in range(10000):
        yield {"i": np.random.rand(512, 512, 3)}
        
Dataset.from_generator(gen)
# -> 90% of the time gets stuck around 1000 rows
```

### Expected behavior

Should continue and go through all the examples yielded by the generator, or at least throw an error or somehow communicate what's going on.

### Environment info

- `datasets` version: 2.8.0
- Platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 12.0.1
- Pandas version: 1.5.1


Comments: By default, we write data to disk (so it can be memory-mapped) every 1000 rows/samples. You can control this with the `writer_batch_size` parameter. Also, when working with fixed-size arrays, the `ArrayXD` feature types yield better performance (e.g., in your case, `features=datasets.Features({"i": datasets.Array3D(shape=(512,512,3), dtype="float32")})` should be faster).

Our support for multi-dim arrays could be better, and we plan to improve it as part of https://github.com/huggingface/datasets/issues/5272.

> By default, we write data to disk (so it can be memory-mapped) every 1000 rows/samples. You can control this with the `writer_batch_size` parameter. Also, when working with fixed-size arrays, the `ArrayXD` feature types yield better performance (e.g., in your case, `features=datasets.Features({"i": datasets.Array3D(shape=(512,512,3), dtype="float32")})` should be faster).
> 
> Our support for multi-dim arrays could be better, and we plan to improve it as part of #5272.

Thanks for the explanation! The Image array was just for demonstration, I use PIL Images in practice. Does that make a difference? What's the best approach for a dataset with PIL Images as rows?

It's best to use the `datasets.Image()` feature type for PIL images (to save space) :)
adds early exit if url is `PathLike`
Opened by vvvm23 (open)

Closes #4864 

Should fix errors thrown when attempting to load `json` dataset using `pathlib.Path` in `data_files` argument.

Comments: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5630). All of your documentation changes will be reflected on that endpoint.
A module that was compiled using NumPy 1.x cannot be run in NumPy 2.0.2 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
Opened by jamessdixon (closed)

### Describe the bug

Following this tutorial: https://huggingface.co/docs/diffusers/en/tutorials/basic_training and running it locally using VSCode on my MacBook.  The first line in the tutorial fails:  from datasets import load_dataset
dataset = load_dataset('huggan/smithsonian_butterflies_subset', split="train"). with this error: 
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.2 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2. and ImportError: numpy.core.multiarray failed to import.   

Does from datasets import load_dataset really use NumPy 1.x?  

### Steps to reproduce the bug

Open VSCode.  create a new venv.  Create a new ipynb file.  Import pip install diffusers[training] try to run this line of code: from datasets import load_dataset

### Expected behavior

data is loaded

### Environment info

ran this: datasets-cli env
and got A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.2 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Comments: recreated .venv and run this: pip install diffusers[training]==0.11.1
🐛 [Metrics] ROUGE is non-deterministic
Opened by astariul (closed)

If I run the ROUGE metric 2 times, with same predictions / references, the scores are slightly different.

Refer to [this Colab notebook](https://colab.research.google.com/drive/1wRssNXgb9ldcp4ulwj-hMJn0ywhDOiDy?usp=sharing) for reproducing the problem.

Example of F-score for ROUGE-1, ROUGE-2, ROUGE-L in 2 differents run :

> ['0.3350', '0.1470', '0.2329']
['0.3358', '0.1451', '0.2332']

---

Why ROUGE is not deterministic ?

Comments: Hi, can you give a full self-contained example to reproduce this behavior?

> Hi, can you give a full self-contained example to reproduce this behavior?

There is a notebook in the post ;)

> If I run the ROUGE metric 2 times, with same predictions / references, the scores are slightly different.
> 
> Refer to [this Colab notebook](https://colab.research.google.com/drive/1wRssNXgb9ldcp4ulwj-hMJn0ywhDOiDy?usp=sharing) for reproducing the problem.
> 
> Example of F-score for ROUGE-1, ROUGE-2, ROUGE-L in 2 differents run :
> 
> > ['0.3350', '0.1470', '0.2329']
> > ['0.3358', '0.1451', '0.2332']
> 
> Why ROUGE is not deterministic ?

This is because of rouge's `BootstrapAggregator` that uses sampling to get confidence intervals (low, mid, high).
You can get deterministic scores per sentence pair by using
```python
score = rouge.compute(rouge_types=["rouge1", "rouge2", "rougeL"], use_aggregator=False)
```
Or you can set numpy's random seed if you still want to use the aggregator.

Maybe we can set all the random seeds of numpy/torch etc. while running `metric.compute` ?

We should probably indeed!

Now if you re-run the notebook, the two printed results are the same @colanim
```
['0.3356', '0.1466', '0.2318']
['0.3356', '0.1466', '0.2318']
```
However across sessions, the results may change (as numpy's random seed can be different). You can prevent that by setting your seed:
```python
rouge = nlp.load_metric('rouge', seed=42)
```

> 

Minor nit: Note that "aggregator" is misspelled in this command. Should be `use_aggregator=False`.  

Thanks, I fixed the code snippet
Add SV-Ident dataset
Opened by e-tornike (closed)


Comments: Hi @e-tornike, thanks a lot for adding this interesting dataset.

Recently at Hugging Face, we have decided to give priority to adding datasets directly on the Hub. Would you mind to transfer your loading script to the Hub? You could create a dedicated org namespace, so that your dataset would be accessible using `vadis/sv_ident` or `sdproc/sv_ident` or `coling/sv_ident` (as you prefer).

You have an example here: https://huggingface.co/datasets/projecte-aina/catalan_textual_corpus

Additionally, please feel free to ping us if you need assistance/help in creating this dataset.

You could put the link to your Hub dataset here in this Issue discussion page, so that we can follow the progress. :)

Hi @albertvillanova, thanks for the feedback! Uploading via the Hub is a lot easier! 

I've uploaded the dataset here: https://huggingface.co/datasets/vadis/sv-ident, but it's reporting a "Status400Error". Is there any way to see the logs of the dataset script and what is causing the error?

Hi @e-tornike, good job at https://huggingface.co/datasets/vadis/sv-ident.

Normally, you can run locally the loading of the dataset by passing `streaming=True` (as the previewer does):
```python
ds = load_dataset("path/to/sv_ident.py, split="train", streaming=True)
item = next(iter(ds))
item
```

Let me have a look and open a discussion on your Hub repo! ;)

I've opened an Issue: 
- #4527
Fix emotion description
Opened by lewtun (closed)

Fixes the description of the emotion dataset to reflect the class names observed in the data, not the ones described in the paper.

I also took the liberty to make use of `ClassLabel` for the emotion labels.

Comments: Hello, I probably have a silly question but the labels of the emotion dataset are in the form of numbers and not string, so I can not use the function classification_report because it mixes numbers and string (prediction). How can I access the label in the form of a string and not a number? 
Thank you in advance.
load_dataset gives FileNotFoundError on imagenet-1k if license is not accepted on the hub
Opened by buoi (closed)

### Describe the bug

When calling ```load_dataset('imagenet-1k')``` FileNotFoundError is raised, if not logged in and if logged in with huggingface-cli but not having accepted the licence on the hub. There is no error once accepting.

### Steps to reproduce the bug

```
from datasets import load_dataset

imagenet = load_dataset("imagenet-1k", split="train", streaming=True)

FileNotFoundError: Couldn't find a dataset script at /content/imagenet-1k/imagenet-1k.py or any data file in the same directory. Couldn't find 'imagenet-1k' on the Hugging Face Hub either: FileNotFoundError: Dataset 'imagenet-1k' doesn't exist on the Hub
```
tested on a colab notebook.

### Expected behavior

I would expect a specific error indicating that I have to login then accept the dataset licence.

I find this bug very relevant as this code is on a guide on the [Huggingface documentation for Datasets](https://huggingface.co/docs/datasets/about_mapstyle_vs_iterable)

### Environment info

google colab cpu-only instance

Comments: Hi, thanks for the feedback! Would it help to add a tip or note saying the dataset is gated and you need to accept the license before downloading it?

The error is now more informative:
```
FileNotFoundError: Couldn't find a dataset script at /content/imagenet-1k/imagenet-1k.py or any data file in the same directory. Couldn't find 'imagenet-1k' on the Hugging Face Hub either: FileNotFoundError: Dataset 'imagenet-1k' doesn't exist on the Hub. If the repo is private or gated, make sure to log in with `huggingface-cli login`.
```
Not support links with 302 redirect 
Opened by chameleonTK (closed)

I have an issue adding this download link https://github.com/jitkapat/thailitcorpus/releases/download/v.2.0/tlc_v.2.0.tar.gz

it might be because it is not a direct link (it returns 302 and redirects to aws that returns 403 for head requests). 

```
r.head("https://github.com/jitkapat/thailitcorpus/releases/download/v.2.0/tlc_v.2.0.tar.gz", allow_redirects=True)                                      
# <Response [403]>
```

Comments: Hi !
This kind of links is now supported by the library since #1316

> Hi !
> This kind of links is now supported by the library since #1316

I updated links in TLC datasets to be the github links in this pull request 
 https://github.com/huggingface/datasets/pull/1737

Everything works now. Thank you.
Add metrics
Opened by mariamabarham (closed)

This PR is a draft for adding metrics (sacrebleu and seqeval are added)

use case examples:
`import nlp`
**sacrebleu:**
```
refs = [['The dog bit the man.', 'It was not unexpected.', 'The man bit him first.'],
        ['The dog had bit the man.', 'No one was surprised.', 'The man had bitten the dog.']]
sys = ['The dog bit the man.', "It wasn't surprising.", 'The man had just bitten him.']
sacrebleu = nlp.load_metrics('sacrebleu')
print(sacrebleu.score)
```

**seqeval:**
```
y_true = [['O', 'O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
y_pred = [['O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
seqeval = nlp.load_metrics('seqeval')
print(seqeval.accuracy_score(y_true, y_pred)
print(seqeval.f1_score(y_true, y_pred)
```
_examples are taken from the corresponding web page_

your comments and suggestions are more than welcomed
Best practices for training tokenizers with nlp
Opened by moskomule (closed)

Hi, thank you for developing this library. 

What do you think are the best practices for training tokenizers using `nlp`?  In the document and examples, I could only find pre-trained tokenizers used.

Comments: Docs that explain how to train a tokenizer with `datasets` are available here: https://huggingface.co/docs/tokenizers/training_from_memory#using-the-datasets-library
JSON lines with empty struct raise ArrowTypeError
Opened by albertvillanova (closed)

JSON lines with empty struct raise ArrowTypeError: struct fields don't match or are in the wrong order

See example: https://huggingface.co/datasets/wikimedia/structured-wikipedia/discussions/5

> ArrowTypeError: struct fields don't match or are in the wrong order: Input fields: struct<> output fields: struct<pov_count: int64, update_count: int64, citation_needed_count: int64>

Related to:
- #7159
Set builder name from module instead of class
Opened by albertvillanova (closed)

Now the builder name attribute is set from from the builder class name.

This PR sets the builder name attribute from the module name instead. Some motivating reasons:
- The dataset ID is relevant and unique among all datasets and this is directly related to the repository name, i.e., the name of the directory containing the dataset
- The name of the module (i.e. the file containing the loading loading script) is already relevant for loading: it must have the same name as its containing directory (related to the dataset ID), as we search for it using its directory name
- On the other hand, the name of the builder class is not relevant for loading: in our code, we just search for a class which is subclass of `DatasetBuilder` (independently of its name). We do not put any constraint on the naming of the builder class and indeed it can have a name completely different from its module/direcotry/dataset_id

IMO it makes more sense to align the caching directory name with the dataset_id/directory/module name instead of the builder class name.

Fix #4381.

Comments: _The documentation is not available anymore as the PR was closed or merged._
How to convert videos to images?
Opened by Loki-Lu (open)

### Feature request

Does someone know how to return the images from videos?

### Motivation

I am trying to use openpi(https://github.com/Physical-Intelligence/openpi) to finetune my Lerobot dataset(V2.0 and V2.1). I find that although the codedaset is v2.0, they are different. It seems like Lerobot V2.0 has two version, one is data include images infos and another one is separate to data and videos.

Does someone know how to return the images from videos?





Comments: Hi ! there is some documentation here on how to read video frames: https://huggingface.co/docs/datasets/video_load
social_i_qa wrong format of labels
Opened by ghost (closed)

Hi,
there is extra "\n" in labels of social_i_qa datasets, no big deal, but I was wondering if you could remove it to make it consistent.
so label is 'label': '1\n', not '1'
thanks

```
>>> import datasets 
>>> from datasets import load_dataset
>>> dataset = load_dataset(
...    'social_i_qa')
cahce dir  /julia/cache/datasets
Downloading: 4.72kB [00:00, 3.52MB/s]                                                                                                  
cahce dir /julia/cache/datasets
Downloading: 2.19kB [00:00, 1.81MB/s]                                                                                                  
Using custom data configuration default
Reusing dataset social_i_qa (/julia/datasets/social_i_qa/default/0.1.0/4a4190cc2d2482d43416c2167c0c5dccdd769d4482e84893614bd069e5c3ba06)
>>> dataset['train'][0]
{'answerA': 'like attending', 'answerB': 'like staying home', 'answerC': 'a good friend to have', 'context': 'Cameron decided to have a barbecue and gathered her friends together.', 'label': '1\n', 'question': 'How would Others feel as a result?'}

```



Comments: @lhoestq, should I raise a PR for this? Just a minor change while reading labels text file

Sure feel free to open a PR thanks !
Align logging with Transformers (again)
Opened by mariosasko (closed)

Fix #2832 

Comments: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4686). All of your documentation changes will be reflected on that endpoint.

I wasn't aware of https://github.com/huggingface/datasets/pull/1845 before opening this PR. This issue seems much more complex now ...
Nlp cli fix endpoints
Opened by lhoestq (closed)

With this PR users will be able to upload their own datasets and metrics.

As mentioned in #181, I had to use the new endpoints and revert the use of dataclasses (just in case we have changes in the API in the future).

We now distinguish commands for datasets and commands for metrics:
```bash
nlp-cli upload_dataset <path/to/dataset>
nlp-cli upload_metric <path/to/metric>
nlp-cli s3_datasets {rm, ls}
nlp-cli s3_metrics {rm, ls}
```

Does it sound good to you @julien-c @thomwolf ?

Comments: LGTM 👍