liuqingwen/github-issues huggingface dataset:
label_column='labels' in datasets.TextClassification and 'label' or 'label_ids' in transformers.DataColator

### Describe the bug

When preparing a dataset for a task using `datasets.TextClassification`, the output feature is named `labels`. When preparing the trainer using the `transformers.DataCollator` the default column name is `label` if binary or `label_ids` if multi-class problem.

It is required to rename the column accordingly to the expected name : `label` or `label_ids`

### Steps to reproduce the bug

```python
from datasets import TextClassification, AutoTokenized, DataCollatorWithPadding
ds_prepared = my_dataset.prepare_for_task(TextClassification(text_column='TEXT', label_column='MY_LABEL_COLUMN_1_OR_0'))
print(ds_prepared)

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")
ds_tokenized = ds_prepared.map(lambda x: tokenizer(x['text'], truncation=True), batched=True)
print(ds_tokenized)

data_collator = DataCollatorWithPadding(tokenizer=tokenizer, return_tensors="tf")

tf_data = model.prepare_tf_dataset(ds_tokenized, shuffle=True, batch_size=16, collate_fn=data_collator)

print(tf_data)
```

### Expected behavior

Without renaming the the column, the  target column is not in the final tf_data since it is not in the column name expected by the data_collator.

To correct this, we have to rename the column:

```python
ds_prepared = my_dataset.prepare_for_task(TextClassification(text_column='TEXT', label_column='MY_LABEL_COLUMN_1_OR_0')).rename_column('labels', 'label')
```

### Environment info

- `datasets` version: 2.8.0
- Platform: Linux-5.15.79.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
- Python version: 3.10.6
- PyArrow version: 10.0.1
- Pandas version: 1.5.2

- `transformers` version: 4.26.0.dev0
- Platform: Linux-5.15.79.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
- Python version: 3.10.6
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.11.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

Comments:
Hi! Thanks for pointing out this inconsistency. Changing the default value at this point is probably not worth it, considering we've started discussing the state of the task API internally - we will most likely deprecate the current one and replace it with a more robust solution that relies on the `train_eval_index` field stored in the YAML section of the dataset cards.
Add ProgressBar for `to_parquet`
Labels: name: enhancement
description: New feature or request

### Feature request

Add a progress bar for `Dataset.to_parquet`, similar to how `to_json` works.

### Motivation

It's a bit frustrating to not know how long a dataset will take to write to file and if it's stuck or not without a progress bar

### Your contribution

Sure I can help if needed

Comments:
Thanks for your proposal, @zanussbaum. Yes, I agree that would definitely be a nice feature to have!

@albertvillanova I’m happy to make a quick PR for the feature! let me know 

That would be awesome ! You can comment `#self-assign` to assign you to this issue and open a PR :) Will be happy to review

Closing as this has been merged @lhoestq
Better integration of BIG-bench
Labels: name: enhancement
description: New feature or request

### Feature request

Ideally, it would be nice to have a maintained PyPI package for `bigbench`.

### Motivation

We'd like to allow anyone to access, explore and use any task.

### Your contribution

@lhoestq has opened an issue in their repo:
- https://github.com/google/BIG-bench/issues/906

Comments:
Hi, I made my version : https://huggingface.co/datasets/tasksource/bigbench
IterableDataset missing column_names, differs from Dataset interface
Labels: name: enhancement
description: New feature or request

name: good first issue
description: Good for newcomers

### Describe the bug

The documentation on [Stream](https://huggingface.co/docs/datasets/v1.18.2/stream.html) seems to imply that IterableDataset behaves just like a Dataset. However, examples like
```
dataset.map(augment_data, batched=True, remove_columns=dataset.column_names, ...)
```
will not work because `.column_names` does not exist on IterableDataset. I cannot find any clear explanation on why this is not available, is it an oversight? We do have `iterable_ds.features` available.

### Steps to reproduce the bug

See above

### Expected behavior

Dataset and IterableDataset would be expected to have the same interface, with any differences noted in the documentation.

### Environment info

n/a

Comments:
Another example is that `IterableDataset.map` does not have `fn_kwargs`, among other arguments. It makes it harder to convert code from Dataset to IterableDataset.

Hi! `fn_kwargs` was added to `IterableDataset.map` in `datasets 2.5.0`, so please update your installation (`pip install -U datasets`) to use it.

Regarding `column_names`, I agree we should add this property to `IterableDataset`. In the meantime, you can use `list(dataset.features.keys())` instead.

Thanks! That's great news.

On Thu, Dec 22, 2022, 07:48 Mario Šaško ***@***.***> wrote:

> Hi! fn_kwargs was added to IterableDataset.map in datasets 2.5.0, so
> please update your installation (pip install -U datasets) to use it.
>
> Regarding column_names, I agree we should add this property to
> IterableDataset. In the meantime, you can use
> list(dataset.features.keys()) instead.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/5383#issuecomment-1362993633>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHD6N2EQUFEOUFDW3VHSILWORZ45ANCNFSM6AAAAAATGKWVGM>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>


I'm marking this issue as a "good first issue", as it makes sense to have `IterableDataset.column_names` in the API. Besides the case when `features` are `None` (e.g., `features` are `None` after `map`), in which we can also return `column_names` as `None`, adding this property should be straightforward,

Hi @mariosasko, I can work on this if that's ok?

Yes! I've assigned you the issue.
Add a robustness benchmark dataset for vision
Labels: name: dataset request
description: Requesting to add a new dataset

### Name

ImageNet-C

### Paper

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

### Data

https://github.com/hendrycks/robustness

### Motivation

It's a known fact that vision models are brittle when they meet with slightly corrupted and perturbed data. This is also correlated to the robustness aspects of vision models. 

Researchers use different benchmark datasets to evaluate the robustness aspects of vision models. ImageNet-C is one of them. 

Having this dataset in 🤗 Datasets would allow researchers to evaluate and study the robustness aspects of vision models. Since the metric associated with these evaluations is top-1 accuracy, researchers should be able to easily take advantage of the evaluation benchmarks on the Hub and perform comprehensive reporting. 

ImageNet-C is a large dataset. Once it's in, it can act as a reference and we can also reach out to the authors of the other robustness benchmark datasets in vision, such as ObjectNet, WILDS, Metashift, etc. These datasets cater to different aspects. For example, ObjectNet is related to assessing how well a model performs under sub-population shifts. 

Related thread: https://huggingface.slack.com/archives/C036H4A5U8Z/p1669994598060499

Comments:
Ccing @nazneenrajani @lvwerra @osanseviero
Support remote file systems for `Audio`
Labels: name: enhancement
description: New feature or request

### Feature request

Hi there!

It would be super cool if `Audio()`, and potentially other features, could read files from a remote file system.

### Motivation

Large amounts of data is often stored in buckets. `load_from_disk` is able to retrieve data from cloud storage but to my knowledge actually copies the datasets across first, so if you're working off a system with smaller disk specs (like a VM), you can run out of space very quickly.

### Your contribution

Something like this (for Google Cloud Platform in this instance):



```python
from datasets import Dataset, Audio
import gcsfs

fs = gcsfs.GCSFileSystem()

list_of_audio_fp = {'audio': ['1', '2', '3']}
ds = Dataset.from_dict(list_of_audio_fp)

ds = ds.cast_column("audio", Audio(sampling_rate=16000, fs=fs))
```

Under the hood:
```python

import librosa
from io import BytesIO

def load_audio(fp, sampling_rate=None, fs=None):
    if fs is not None:
        with fs.open(fp, 'rb') as f:
            arr, sr = librosa.load(BytesIO(f), sr=sampling_rate)
    else:
        # Perform existing io operations
```

Written from memory so some things could be wrong.

Comments:
Just seen https://github.com/huggingface/datasets/issues/5281
Emotion dataset cannot be downloaded
Labels: name: duplicate
description: This issue or pull request already exists

### Describe the bug

The emotion dataset gives a FileNotFoundError. The full error is: `FileNotFoundError: Couldn't find file at https://www.dropbox.com/s/1pzkadrvffbqw6o/train.txt?dl=1`.

It was working yesterday (December 7, 2022), but stopped working today (December 8, 2022).

### Steps to reproduce the bug

```python
from datasets import load_dataset
dataset = load_dataset("emotion")
```

### Expected behavior

The dataset should load properly.

### Environment info

- `datasets` version: 2.7.1
- Platform: Windows-10-10.0.19045-SP0
- Python version: 3.9.13
- PyArrow version: 10.0.1
- Pandas version: 1.5.1

Comments:
Hi @cbarond there's already an open issue at https://github.com/dair-ai/emotion_dataset/issues/5, as the data seems to be missing now, so check that issue instead 👍🏻 

Thanks @cbarond for reporting and @alvarobartt for pointing to the issue we opened in the author's repo.

Indeed, this issue was first raised in the "emotion" dataset Community tab: https://huggingface.co/datasets/emotion/discussions/3

I'm closing this issue and leave the issue above for the subsequent updates.

Duplicate of: https://huggingface.co/datasets/emotion/discussions/3

try using "SetFit/emotion" instead

> try using "SetFit/emotion" instead

I' replaced "emotion" with "SetFit/Emotion", but the code is getting stuck at

`emotions = load_dataset("SetFit/emotion")`

I pause execution using the debugger, and it takes me to filelock.py:226

`with self._thread_lock:`

Do you know a way to get past this issue?

thanks @honeyimholm - worked for me

> try using "SetFit/emotion" instead

It really helps a lot, thank you!

The dataset loading script has been fixed: https://huggingface.co/datasets/emotion/discussions/4
Add Video feature, videofolder, and video-classification task

This PR does the following:

- Adds `Video` feature (Resolves #5225 )
- Adds `video-classification` task
- Adds `videofolder` packaged module for easy loading of local video classification datasets

TODO:
  - [ ] add tests
  - [ ] add docs

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5339). All of your documentation changes will be reflected on that endpoint.

@lhoestq I think I need some serious help with the tests 😅...I started this locally but it got too time consuming.

One issue I remember running into is with lossless audio encoding/decoding. I started thinking of using the underlying Audio feature instead of PyAV so I didn't have to rewrite similar logic here...but assumed that would turn into a mess w/ underlying logic
`map()` stops every 1000 steps

### Describe the bug

I am passing the following `prepare_dataset` function to `Dataset.map` (code is inspired from [here](https://github.com/huggingface/community-events/blob/main/whisper-fine-tuning-event/run_speech_recognition_seq2seq_streaming.py#L454))
```python3
def prepare_dataset(batch):
    # load and resample audio data from 48 to 16kHz
    audio = batch["audio"]

    # compute log-Mel input features from input audio array 
    batch["input_features"] = feature_extractor(audio["array"], sampling_rate=audio["sampling_rate"]).input_features[0]

    # encode target text to label ids 
    batch["labels"] = tokenizer(batch[text_column]).input_ids
    return batch
...
train_ds = train_ds.map(prepare_dataset)
```

Here is the exact code I am running https://github.com/bayartsogt-ya/whisper-multiple-hf-datasets/blob/main/train.py#L70-L71

It starts using all the cores (I am not sure why because I did not pass `num_proc`)
then progress bar stops at every 1k steps. (starts using a single core)
then come back to using all the cores again.

link to [screen record](https://youtu.be/jPQpQQGp6Gc)

Can someone explain this process and maybe provide a way to improve this pipeline? cc: @lhoestq 

### Steps to reproduce the bug

1. load the dataset
2. create a Whisper processor
3. create a `prepare_dataset` function
4. pass the function to `dataset.map(prepare_dataset)`

### Expected behavior

- Use a single core per a function
- not to stop at some point?

### Environment info

- `datasets` version: 2.7.1.dev0
- Platform: Linux-5.4.0-109-generic-x86_64-with-glibc2.27
- Python version: 3.8.10
- PyArrow version: 10.0.1
- Pandas version: 1.5.2

Comments:
Hi !

> It starts using all the cores (I am not sure why because I did not pass num_proc)

The tokenizer uses Rust code that is multithreaded. And maybe the `feature_extractor` might run some things in parallel as well - but I'm not super familiar with its internals.

> then progress bar stops at every 1k steps. (starts using a single core)

Every 1000 examples we flush the processed examples to disk. It is this way because Arrow is a columnar format: you must write data chunk by chunk. The processing in on hold while writing right now - maybe this can be improved in the future.

Hi @lhoestq 
Thanks for the explanation! it was so helpful! Let me check why `feature_extractor` is running on multiple cpus.
Support webdataset format

Webdataset is an efficient format for iterable datasets. It would be nice to support it in `datasets`, as discussed in https://github.com/rom1504/img2dataset/issues/234.

In particular it would be awesome to be able to load one using `load_dataset` in streaming mode (either from a local directory, or from a dataset on the Hugging Face Hub). Some datasets on the Hub are already in webdataset format.

It terms of implementation, we can have something similar to the Parquet loader.
I also think it's fine to have webdataset as an optional dependency.

Comments:
I like the idea of having `webdataset` as an optional dependency to ensure our loader generates web datasets the same way as the main project.

Webdataset is the one of the most popular dataset formats for large scale computer vision tasks. Upvote for this issue. 

Any updates on this?

We haven't had the bandwidth to implement it so far, but if someone wants to give it a shot please don't hesitate ^^
Clean up docstrings
Labels: name: documentation
description: Improvements or additions to documentation

As raised by @polinaeterna in #5324, some of the docstrings are a bit of a mess because it has both Markdown and Sphinx syntax. This PR fixes the docstring for `DatasetBuilder`. 

I'll start working on cleaning up the rest of the docstrings and removing the old Sphinx syntax (let me know if you prefer one big PR with all the cleaned changes or multiple smaller ones)! 🧼 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks ! Let us know if we can help :)

Small pref for having multiple PRs

Awesome, thanks! Sorry this one is a little big, I'll open some smaller ones next :)
No documentation for main branch is built
Labels: name: bug
description: Something isn't working

Since:
- #5250
  - Commit: 703b84311f4ead83c7f79639f2dfa739295f0be6

the docs for main branch are no longer built.

The change introduced only triggers the docs building for releases.

Comments:
map(...batch_size=None) for IterableDataset
Labels: name: enhancement
description: New feature or request

name: good first issue
description: Good for newcomers

### Feature request

Dataset.map(...) allows batch_size to be None. It would be nice if IterableDataset did too.

### Motivation

Although it may seem a bit of a spurious request given that `IterableDataset` is meant for larger than memory datasets, but there are a couple of reasons why this might be nice.

One is that load_dataset(...) can return either IterableDataset or Dataset. mypy will then complain if batch_size=None even if we know it is Dataset. Of course we can do:

    assert isinstance(d, datasets.DatasetDict)

But it is a mild inconvenience. What's more annoying is that whenever we use something like e.g. `combine_datasets(...)`, we end up with the union again, and so have to do the assert again.

Another is that we could actually end up with an IterableDataset small enough for memory in normal/correct usage, e.g. by filtering a massive IterableDataset.

For practical usages, an alternative to this would be to convert from an iterable dataset to a map-style dataset, but it is not obvious how to do this.

### Your contribution

Not this time.

Comments:
Hi! I agree it makes sense for `IterableDataset.map` to support the `batch_size=None` case. This should be super easy to fix.

@mariosasko as this is something simple maybe I can include it as part of https://github.com/huggingface/datasets/pull/5311? Let me know :+1:

#self-assign

Feel free to close this @lhoestq as part of https://github.com/huggingface/datasets/pull/5336 :hugs:

Thanks again :)

> For practical usages, an alternative to this would be to convert from an iterable dataset to a map-style dataset, but it is not obvious how to do this.

This is interesting as well, if anyone wants to explore
Fix docstrings and types in documentation that appears on the website
Labels: name: documentation
description: Improvements or additions to documentation

While I was working on https://github.com/huggingface/datasets/pull/5313 I've noticed that we have a mess in how we annotate types and format args and return values in the code. And some of it is displayed in the [Reference section](https://huggingface.co/docs/datasets/package_reference/builder_classes) of the documentation on the website.
 
Would be nice someday, maybe before releasing datasets 3.0.0, to unify it......

Comments:
I agree we have a mess with docstrings...

Ok, I believe we've cleaned up most of the old syntax we were using for the user-facing docs! There are still a couple of `:obj:`'s and `:class:` floating around in the docstrings we don't expose that I'll track down :)
Bug in xopen with Windows pathnames
Labels: name: bug
description: Something isn't working

Currently, `xopen` function has a bug with local Windows pathnames:

From its implementation:
```python
def xopen(file: str, mode="r", *args, **kwargs):
    file = _as_posix(PurePath(file))
    main_hop, *rest_hops = file.split("::")
    if is_local_path(main_hop):
        return open(file, mode, *args, **kwargs)
```

On a Windows machine, if we pass the argument:
```python
xopen("C:\\Users\\USERNAME\\filename.txt")
```
it returns
```python
open("C:/Users/USERNAME/filename.txt")
```

Comments:
Bug in xjoin with Windows pathnames
Labels: name: bug
description: Something isn't working

Currently, `xjoin` function has a bug with local Windows pathnames: instead of returning the OS-dependent join pathname, it always returns it in POSIX format.

```python
from datasets.download.streaming_download_manager import xjoin

path = xjoin("C:\\Users\\USERNAME", "filename.txt")
```

Join path should be:
```python
"C:\\Users\\USERNAME\\filename.txt"
```
However it is:
```python
"C:/Users/USERNAME/filename.txt"
```

Comments:
Support streaming datasets with pathlib.Path.with_suffix
Labels: name: enhancement
description: New feature or request

Extend support for streaming datasets that use `pathlib.Path.with_suffix`.

This feature will be useful e.g. for datasets containing text files and annotated files with the same name but different extension.

Comments:
Missing documentation build for versions 2.7.1 and 2.6.2 
Labels: name: maintenance
description: Maintenance tasks

After the patch releases [2.7.1](https://github.com/huggingface/datasets/releases/tag/2.7.1) and [2.6.2](https://github.com/huggingface/datasets/releases/tag/2.6.2), the online docs were not properly built (the build_documentation workflow was not triggered).

There was a fix by:
- #5291

However, both documentations were built from main branch, instead of their corresponding version branch.

We are rebuilding them.

Comments:
- Build docs for 2.6.2:
  - Commit: a6a5a1cf4cdf1e0be65168aed5a327f543001fe8
  - Build docs GH Action: https://github.com/huggingface/datasets/actions/runs/3539470622/jobs/5941404044
- Build docs for 2.7.1:
  - Commit: 5ef1ab1cc06c2b7a574bf2df454cd9fcb071ccb2
  - Build docs GH Action: https://github.com/huggingface/datasets/actions/runs/3539574442/jobs/5941636792
Shell completions
Labels: name: enhancement
description: New feature or request

### Feature request

Like <https://github.com/huggingface/huggingface_hub/issues/1197>, datasets-cli maybe need it, too.

### Motivation

See above.

### Your contribution

Maybe.

Comments:
I don't think we need completion on the datasets-cli, since we're mainly developing huggingface-cli

I see.
Get an IterableDataset from a map-style Dataset
Labels: name: enhancement
description: New feature or request

name: streaming

This is useful to leverage iterable datasets specific features like:
- fast approximate shuffling
- lazy map, filter etc.

Iterating over the resulting iterable dataset should be at least as fast at iterating over the map-style dataset.

Here are some ideas regarding the API:
```python
# 1. 
# - consistency with load_dataset(..., streaming=True)
# - gives intuition that map/filter/etc. are done on-the-fly
ids = ds.stream()

# 2.
# - more explicit on the output type
# - but maybe sounds like a conversion tool rather than a step in a processing pipeline
ids = ds.as_iterable_dataset()
```

Comments:
I think `stream` could be misleading since the data is not being streamed from remote endpoints (one could think that's the case when they see `load_dataset` followed by `stream`). Hence, I prefer the second option.

PS: When we resolve https://github.com/huggingface/datasets/issues/4542, we could add `as_tf_dataset` to the API for consistency and deprecate `to_tf_dataset`.
Save a dataset in a determined number of shards
Labels: name: enhancement
description: New feature or request

This is useful to distribute the shards to training nodes.

This can be implemented in `save_to_disk` and can also leverage multiprocessing to speed up the process

Comments:
Add PubTables-1M
Labels: name: dataset request
description: Requesting to add a new dataset

### Name

PubTables-1M

### Paper

https://openaccess.thecvf.com/content/CVPR2022/html/Smock_PubTables-1M_Towards_Comprehensive_Table_Extraction_From_Unstructured_Documents_CVPR_2022_paper.html

### Data

https://github.com/microsoft/table-transformer

### Motivation

Table Transformer is now available in 🤗 Transformer, and it was trained on PubTables-1M. It's a large dataset for table extraction and structure recognition in unstructured documents.

Comments:
cc @albertvillanova the author would like to add this dataset to the hub: https://github.com/microsoft/table-transformer/issues/68#issuecomment-1319114621. Could you help him out?