Minimartzz/github-issues huggingface dataset:
CI benchmarks are broken: Unknown arguments: runnerPath, path
State: closed
Labels: description: Maintenance tasks
name: maintenance

Our CI benchmarks are broken, raising `Unknown arguments` error: https://github.com/huggingface/datasets/actions/runs/3932397079/jobs/6724905161
```
Unknown arguments: runnerPath, path
```

Stack trace:
```
100%|██████████| 500/500 [00:01<00:00, 338.98ba/s]
Updating lock file 'dvc.lock'

To track the changes with git, run:

	git add dvc.lock

To enable auto staging, run:

	dvc config core.autostage true
Use `dvc push` to send your updates to remote storage.
cml send-comment <markdown file>

Global Options:
  --log                    Logging verbosity
          [string] [choices: "error", "warn", "info", "debug"] [default: "info"]
  --driver                 Git provider where the repository is hosted
    [string] [choices: "github", "gitlab", "bitbucket"] [default: infer from the
                                                                    environment]
  --repo                   Repository URL or slug
                                  [string] [default: infer from the environment]
  --driver-token, --token  CI driver personal/project access token (PAT)
                                  [string] [default: infer from the environment]
  --help                   Show help                                   [boolean]

Options:
  --target                    Comment type (`commit`, `pr`, `commit/f00bar`,
                              `pr/42`, `issue/1337`),default is automatic (`pr`
                              but fallback to `commit`).                [string]
  --watch                     Watch for changes and automatically update the
                              comment                                  [boolean]
  --publish                   Upload any local images found in the Markdown
                              report                   [boolean] [default: true]
  --publish-url               Self-hosted image server URL
                                     [string] [default: "https://asset.cml.dev/"]
  --publish-native, --native  Uses driver's native capabilities to upload assets
                              instead of CML's storage; not available on GitHub
                                                                       [boolean]
  --watermark-title           Hidden comment marker (used for targeting in
                              subsequent `cml comment update`); "{workflow}" &
                              "{run}" are auto-replaced   [string] [default: ""]

Unknown arguments: runnerPath, path
Error: Process completed with exit code 1.
```

Issue reported to iterative/cml:
- iterative/cml#1319

Comments:
Support Apache Beam >= 2.44.0
State: closed
Labels: description: New feature or request
name: enhancement

Once we find out the root cause of:
- #5426 

we should revert the temporary pin on apache-beam introduced by:
- #5429

Comments:
Some of the shard files now have 0 number of rows.

We have opened an issue in the Apache Beam repo:
- https://github.com/apache/beam/issues/25041
Load/Save FAISS index using fsspec
State: closed
Labels: description: New feature or request
name: enhancement

### Feature request

From what I understand `faiss` already support this [link](https://github.com/facebookresearch/faiss/wiki/Index-IO,-cloning-and-hyper-parameter-tuning#generic-io-support)

I would like to use a stream as input to `Dataset.load_faiss_index` and `Dataset.save_faiss_index`.

### Motivation

In my case, I'm saving faiss index in cloud storage and use `fsspec` to load them. It would be ideal if I could send the stream directly instead of copying the file locally (or mounting the bucket) and then load the index.

### Your contribution

I can submit the PR

Comments:
Hi! Sure, feel free to submit a PR. Maybe if we want to be consistent with the existing API, it would be cleaner to directly add support for `fsspec` paths in `Dataset.load_faiss_index`/`Dataset.save_faiss_index` in the same manner as it was done in `Dataset.load_from_disk`/`Dataset.save_to_disk`.

That's a great idea! I'll do that instead.
Unable to download dataset id_clickbait
State: closed

### Describe the bug

I tried to download dataset `id_clickbait`, but receive this error message.

```
FileNotFoundError: Couldn't find file at https://md-datasets-cache-zipfiles-prod.s3.eu-west-1.amazonaws.com/k42j7x2kpn-1.zip
```

When i open the link using browser, i got this XML data.

```xml
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NoSuchBucket</Code><Message>The specified bucket does not exist</Message><BucketName>md-datasets-cache-zipfiles-prod</BucketName><RequestId>NVRM6VEEQD69SD00</RequestId><HostId>W/SPDxLGvlCGi0OD6d7mSDvfOAUqLAfvs9nTX50BkJrjMny+X9Jnqp/Li2lG9eTUuT4MUkAA2jjTfCrCiUmu7A==</HostId></Error>
```

### Steps to reproduce the bug

Code snippet:

```
from datasets import load_dataset

load_dataset('id_clickbait', 'annotated')
load_dataset('id_clickbait', 'raw')
```

Link to Kaggle notebook: https://www.kaggle.com/code/ilosvigil/bug-check-on-id-clickbait-dataset

### Expected behavior

Successfully download and load `id_newspaper` dataset.

### Environment info

- `datasets` version: 2.8.0
- Platform: Linux-5.15.65+-x86_64-with-debian-bullseye-sid
- Python version: 3.7.12
- PyArrow version: 8.0.0
- Pandas version: 1.3.5

Comments:
Thanks for reporting, @ilos-vigil.

We have transferred this issue to the corresponding dataset on the Hugging Face Hub: https://huggingface.co/datasets/id_clickbait/discussions/1
When applying `ReadInstruction` to custom load it's not DatasetDict but list of Dataset?
State: closed

### Describe the bug

I am loading datasets from custom `tsv` files stored locally and applying split instructions for each split. Although the ReadInstruction is being applied correctly and I was expecting it to be `DatasetDict` but instead it is a list of `Dataset`. 

### Steps to reproduce the bug

Steps to reproduce the behaviour:

1. Import 
   `from datasets import load_dataset, ReadInstruction`
2. Instruction to load the dataset
  ```
instructions = [
      ReadInstruction(split_name="train", from_=0, to=10, unit='%', rounding='closest'),
      ReadInstruction(split_name="dev", from_=0, to=10, unit='%', rounding='closest'),
      ReadInstruction(split_name="test", from_=0, to=5, unit='%', rounding='closest')
  ]
```
3. Load 
`dataset = load_dataset('csv', data_dir="data/", data_files={"train":"train.tsv", "dev":"dev.tsv", "test":"test.tsv"}, delimiter="\t", split=instructions)`


### Expected behavior

**Current behaviour**

![Screenshot from 2023-01-16 10-45-27](https://user-images.githubusercontent.com/25720695/212614754-306898d8-8c27-4475-9bb8-0321bd939561.png)
:
 
**Expected behaviour**

![Screenshot from 2023-01-16 10-45-42](https://user-images.githubusercontent.com/25720695/212614813-0d336bf7-5266-482e-bb96-ef51f64de204.png)


### Environment info

``datasets==2.8.0
``
`Python==3.8.5
`
`Platform - Ubuntu 20.04.4 LTS`

Comments:
Hi! You can get a `DatasetDict` if you pass a dictionary with read instructions as follows:
```python
instructions = [
    ReadInstruction(split_name="train", from_=0, to=10, unit='%', rounding='closest'),
    ReadInstruction(split_name="dev", from_=0, to=10, unit='%', rounding='closest'),
    ReadInstruction(split_name="test", from_=0, to=5, unit='%', rounding='closest')
]

dataset = load_dataset('csv', data_dir="data/", data_files={"train":"train.tsv", "dev":"dev.tsv", "test":"test.tsv"}, delimiter="\t", split={inst.split_name: inst for inst in instructions})
```
Support case-insensitive Hub dataset name in load_dataset
State: closed
Labels: description: New feature or request
name: enhancement

### Feature request

The dataset name on the Hub is case-insensitive (see https://github.com/huggingface/moon-landing/pull/2399, internal issue), i.e., https://huggingface.co/datasets/GLUE redirects to https://huggingface.co/datasets/glue.

Ideally, we could load the glue dataset using the following:

```
from datasets import load_dataset
load_dataset('GLUE', 'cola')
```

It breaks because the loading script `GLUE.py` does not exist (`glue.py` should be selected instead).

Minor additional comment: in other cases without a loading script, we can load the dataset, but the automatically generated config name depends on the casing:
- `load_dataset('severo/danish-wit')` generates the config name `severo--danish-wit-e6fda5b070deb133`, while
- `load_dataset('severo/danish-WIT')` generates the config name `severo--danish-WIT-e6fda5b070deb133`

### Motivation

To follow the same UX on the Hub and in the datasets library.

### Your contribution

...

Comments:
Closing as case-insensitivity should be only for URL redirection on the Hub. In the APIs, we will only support the canonical name (https://github.com/huggingface/moon-landing/pull/2399#issuecomment-1382085611)
label_column='labels' in datasets.TextClassification and 'label' or 'label_ids' in transformers.DataColator
State: closed

### Describe the bug

When preparing a dataset for a task using `datasets.TextClassification`, the output feature is named `labels`. When preparing the trainer using the `transformers.DataCollator` the default column name is `label` if binary or `label_ids` if multi-class problem.

It is required to rename the column accordingly to the expected name : `label` or `label_ids`

### Steps to reproduce the bug

```python
from datasets import TextClassification, AutoTokenized, DataCollatorWithPadding
ds_prepared = my_dataset.prepare_for_task(TextClassification(text_column='TEXT', label_column='MY_LABEL_COLUMN_1_OR_0'))
print(ds_prepared)

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")
ds_tokenized = ds_prepared.map(lambda x: tokenizer(x['text'], truncation=True), batched=True)
print(ds_tokenized)

data_collator = DataCollatorWithPadding(tokenizer=tokenizer, return_tensors="tf")

tf_data = model.prepare_tf_dataset(ds_tokenized, shuffle=True, batch_size=16, collate_fn=data_collator)

print(tf_data)
```

### Expected behavior

Without renaming the the column, the  target column is not in the final tf_data since it is not in the column name expected by the data_collator.

To correct this, we have to rename the column:

```python
ds_prepared = my_dataset.prepare_for_task(TextClassification(text_column='TEXT', label_column='MY_LABEL_COLUMN_1_OR_0')).rename_column('labels', 'label')
```

### Environment info

- `datasets` version: 2.8.0
- Platform: Linux-5.15.79.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
- Python version: 3.10.6
- PyArrow version: 10.0.1
- Pandas version: 1.5.2

- `transformers` version: 4.26.0.dev0
- Platform: Linux-5.15.79.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
- Python version: 3.10.6
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.11.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

Comments:
Hi! Thanks for pointing out this inconsistency. Changing the default value at this point is probably not worth it, considering we've started discussing the state of the task API internally - we will most likely deprecate the current one and replace it with a more robust solution that relies on the `train_eval_index` field stored in the YAML section of the dataset cards.

The task templates API has been deprecated (will be removed in version 3.0), so I'm closing this issue.
Add ProgressBar for `to_parquet`
State: closed
Labels: description: New feature or request
name: enhancement

### Feature request

Add a progress bar for `Dataset.to_parquet`, similar to how `to_json` works.

### Motivation

It's a bit frustrating to not know how long a dataset will take to write to file and if it's stuck or not without a progress bar

### Your contribution

Sure I can help if needed

Comments:
Thanks for your proposal, @zanussbaum. Yes, I agree that would definitely be a nice feature to have!

@albertvillanova I’m happy to make a quick PR for the feature! let me know 

That would be awesome ! You can comment `#self-assign` to assign you to this issue and open a PR :) Will be happy to review

Closing as this has been merged @lhoestq
RuntimeError: Sharding is ambiguous for this dataset
State: closed

### Describe the bug

When loading some datasets, a RuntimeError is raised.

For example, for "ami" dataset: https://huggingface.co/datasets/ami/discussions/3
```
.../huggingface/datasets/src/datasets/builder.py in _prepare_split(self, split_generator, check_duplicate_keys, file_format, num_proc, max_shard_size)
   1415         fpath = path_join(self._output_dir, fname)
   1416 
-> 1417         num_input_shards = _number_of_shards_in_gen_kwargs(split_generator.gen_kwargs)
   1418         if num_input_shards <= 1 and num_proc is not None:
   1419             logger.warning(

.../huggingface/datasets/src/datasets/utils/sharding.py in _number_of_shards_in_gen_kwargs(gen_kwargs)
     10     lists_lengths = {key: len(value) for key, value in gen_kwargs.items() if isinstance(value, list)}
     11     if len(set(lists_lengths.values())) > 1:
---> 12         raise RuntimeError(
     13             (
     14                 "Sharding is ambiguous for this dataset: "

RuntimeError: Sharding is ambiguous for this dataset: we found several data sources lists of different lengths, and we don't know over which list we should parallelize:
    - key samples_paths has length 6
    - key ids has length 7
    - key verification_ids has length 6
To fix this, check the 'gen_kwargs' and make sure to use lists only for data sources, and use tuples otherwise. In the end there should only be one single list, or several lists with the same length.
```

This behavior was introduced when implementing multiprocessing by PR:
- #5107

### Steps to reproduce the bug

```python
ds = load_dataset("ami", "microphone-single", split="train", revision="2d7620bb7c3f1aab9f329615c3bdb598069d907a")
```

### Expected behavior

No error raised.

### Environment info

Since datasets 2.7.0

Comments:
concatenate_datasets fails when two dataset with shards > 1 and unequal shard numbers
State: closed

### Describe the bug

When using `concatenate_datasets([dataset1, dataset2], axis = 1)` to concatenate two datasets with shards > 1, it fails:

```
File "/home/xzg/anaconda3/envs/tri-transfer/lib/python3.9/site-packages/datasets/combine.py", line 182, in concatenate_datasets
    return _concatenate_map_style_datasets(dsets, info=info, split=split, axis=axis)
File "/home/xzg/anaconda3/envs/tri-transfer/lib/python3.9/site-packages/datasets/arrow_dataset.py", line 5499, in _concatenate_map_style_datasets
    table = concat_tables([dset._data for dset in dsets], axis=axis)
File "/home/xzg/anaconda3/envs/tri-transfer/lib/python3.9/site-packages/datasets/table.py", line 1778, in concat_tables
    return ConcatenationTable.from_tables(tables, axis=axis)
File "/home/xzg/anaconda3/envs/tri-transfer/lib/python3.9/site-packages/datasets/table.py", line 1483, in from_tables
    blocks = _extend_blocks(blocks, table_blocks, axis=axis)
File "/home/xzg/anaconda3/envs/tri-transfer/lib/python3.9/site-packages/datasets/table.py", line 1477, in _extend_blocks
    result[i].extend(row_blocks)
IndexError: list index out of range
```

### Steps to reproduce the bug

dataset = concatenate_datasets([dataset1, dataset2], axis = 1)

### Expected behavior

The datasets are correctly concatenated.

### Environment info

datasets==2.8.0

Comments:
Hi ! Thanks for reporting :)

I managed to reproduce the hub using
```python

from datasets import concatenate_datasets, Dataset, load_from_disk

Dataset.from_dict({"a": range(9)}).save_to_disk("tmp/ds1")
ds1 = load_from_disk("tmp/ds1")
ds1 = concatenate_datasets([ds1, ds1])

Dataset.from_dict({"b": range(6)}).save_to_disk("tmp/ds2")
ds2 = load_from_disk("tmp/ds2")
ds2 = concatenate_datasets([ds2, ds2, ds2])

concatenate_datasets([ds1, ds2], axis=1)
```
and I get
```python
Traceback (most recent call last):                                                                                                    
  File "test.py", line 98, in <module>
    dds = concatenate_datasets([ds1, ds2], axis=1)
  File "/Users/.../datasets/combine.py", line 182, in concatenate_datasets
    return _concatenate_map_style_datasets(dsets, info=info, split=split, axis=axis)
  File "/Users/.../datasets/arrow_dataset.py", line 5499, in _concatenate_map_style_datasets
    table = concat_tables([dset._data for dset in dsets], axis=axis)
  File "/Users/.../datasets/table.py", line 1778, in concat_tables
    return ConcatenationTable.from_tables(tables, axis=axis)
  File "/Users/.../datasets/table.py", line 1483, in from_tables
    blocks = _extend_blocks(blocks, table_blocks, axis=axis)
  File "/Users/.../datasets/table.py", line 1477, in _extend_blocks
    result[i].extend(row_blocks)
IndexError: list index out of range
```

It appears to happen when the two datasets have a number of shards that is not the same
Datasets.from_sql() generates deprecation warning
State: closed

### Describe the bug

Calling `Datasets.from_sql()` generates a warning:

`.../site-packages/datasets/builder.py:712: FutureWarning: 'use_auth_token' was deprecated in version 2.7.1 and will be removed in 3.0.0. Pass 'use_auth_token' to the initializer/'load_dataset_builder' instead.`

### Steps to reproduce the bug

Any valid call to `Datasets.from_sql()` will produce the deprecation warning.

### Expected behavior

No warning.

The fix should be simply to remove the parameter `use_auth_token` from the call to `builder.download_and_prepare()` at line 43 of `io/sql.py` (it is set to `None` anyway, and is not needed).

### Environment info

- `datasets` version: 2.8.0
- Platform: Linux-4.15.0-169-generic-x86_64-with-glibc2.27
- Python version: 3.9.15
- PyArrow version: 10.0.1
- Pandas version: 1.5.2


Comments:
Thanks for reporting @msummerfield. We are fixing it.
size_in_bytes the same for all splits
State: open

### Describe the bug

Hi, it looks like whenever you pull a dataset and get size_in_bytes, it returns the same size for all splits (and that size is the combined size of all splits). It seems like this shouldn't be the intended behavior since it is misleading. Here's an example:

```
>>> from datasets import load_dataset
>>> x = load_dataset("glue", "wnli")
Found cached dataset glue (/Users/breakend/.cache/huggingface/datasets/glue/wnli/1.0.0/dacbe3125aa31d7f70367a07a8a9e72a5a0bfeb5fc42e75c9db75b96da6053ad)
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 1097.70it/s]
>>> x["train"].size_in_bytes
186159
>>> x["validation"].size_in_bytes
186159
>>> x["test"].size_in_bytes
186159
>>> 
```

### Steps to reproduce the bug

```
>>> from datasets import load_dataset
>>> x = load_dataset("glue", "wnli")
>>> x["train"].size_in_bytes
186159
>>> x["validation"].size_in_bytes
186159
>>> x["test"].size_in_bytes
186159
```

### Expected behavior

The expected behavior is that it should return the separate sizes for all splits.

### Environment info

- `datasets` version: 2.7.1
- Platform: macOS-12.5-arm64-arm-64bit
- Python version: 3.10.8
- PyArrow version: 10.0.1
- Pandas version: 1.5.2

Comments:
Hi @Breakend,

Indeed, the attribute `size_in_bytes` refers to the size of the entire dataset configuration, for all splits (size of downloaded files + Arrow files), not the specific split.
This is also the case for `download_size` (downloaded files) and `dataset_size` (Arrow files).

The size of the Arrow files for a specific split can be accessed: e.g. size of the "test" split only
```python
ds["train"].info.splits["test"].num_bytes
```

I agree this is confusing and maybe we should improve it.
Better integration of BIG-bench
State: open
Labels: description: New feature or request
name: enhancement

### Feature request

Ideally, it would be nice to have a maintained PyPI package for `bigbench`.

### Motivation

We'd like to allow anyone to access, explore and use any task.

### Your contribution

@lhoestq has opened an issue in their repo:
- https://github.com/google/BIG-bench/issues/906

Comments:
Hi, I made my version : https://huggingface.co/datasets/tasksource/bigbench
Support Dataset conversion from/to Spark
State: open

This PR implements Spark integration by supporting `Dataset` conversion from/to Spark `DataFrame`.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5401). All of your documentation changes will be reflected on that endpoint.

Cool thanks !

Spark DataFrame are usually quite big, and I believe here `from_spark` would load everything in the driver node's RAM, which is quite limiting. Same for `to_spark` which would load everything in the driver node's RAM before sending the data to the executor. Maybe we can mention this in the docstring ?

To transfer big datasets from/into the HF ecosystem using Spark maybe we can just make sure that `pyspark` can read/write to the HF Hub, and that `datasets` can read from HDFS/S3/etc.

Yes @lhoestq , consider this as a first integration of the Datasets library with Spark.
- This PR implements the basic conversion between both.
  - And yes, we are using the Spark's `pandas` API (that uses `pyarrow` under the hood): everything is transferred to the driver.
  - Note that we are converting from/to a Datasets dataset: this is not distributed

The next step is to support the integration of the HF Hub with Spark, that I think should be done using `hffs`.

Thinking more about it I don't really see how those two methods help in practice, since one can already do `datasets` <-> pandas <-> spark and those two methods don't add value over this.

However I think it can be good documentation to explain that it's possible to do it and it's super simple