josteece/github-issues huggingface dataset:
Add tutorial for no-code dataset upload
Opened by stevhliu (closed)

This PR is for a tutorial for uploading a dataset to the Hub. It relies on the Hub UI elements to upload a dataset, introduces the online tagging tool for creating tags, and the Dataset card template to get a head start on filling it out. The addition of this tutorial should make it easier for beginners to upload a dataset without accessing the terminal or knowing Git.

Comments:
Cool, love it ! :)

Feel free to add a paragraph saying how to load the dataset:
```python
from datasets import load_dataset

dataset = load_dataset("stevhliu/demo")

# or to separate each csv file into several splits
data_files = {"train": "train.csv", "test": "test.csv"}
dataset = load_dataset("stevhliu/demo", data_files=data_files)
print(dataset["train"][0])
```

Perfect, feel free to mark this PR ready for review :)

cc @albertvillanova do you have any comment ? You can check the tutorial here:
https://47389-250213286-gh.circle-artifacts.com/0/docs/_build/html/no_code_upload.html

Maybe we can just add a list of supported file types:
- csv
- json
- json lines
- text
- parquet

I just added a mention of the login for private datasets. Don't hesitate to edit or comment.

Otherwise I think it's all good, feel free to merge it @stevhliu  if you don't have other changes to make :)
Fix conversion of multidim arrays in list to arrow
Opened by lhoestq (closed)

Arrow only supports 1-dim arrays. Previously we were converting all the numpy arrays to python list before instantiating arrow arrays to workaround this limitation.
However in #2361 we started to keep numpy arrays in order to keep their dtypes.
It works when we pass any multi-dim numpy array (the conversion to arrow has been added on our side), but not for lists of multi-dim numpy arrays.

In this PR I added two strategies:
- one that takes a list of multi-dim numpy arrays on returns an arrow array in an optimized way (more common case)
- one that takes a list of possibly very nested data (lists, dicts, tuples) containing multi-dim arrays. This one is less optimized since it converts all the multi-dim numpy arrays into lists of 1-d arrays for compatibility with arrow. This strategy is simpler that just trying to create the arrow array from a possibly very nested data structure, but in the future we can improve it if needed.

Fix https://github.com/huggingface/datasets/issues/2921

Comments:
Using a list of multi-dim numpy arrays raises an error "can only convert 1-dimensional array values"
Opened by lhoestq (closed)

This error has been introduced in https://github.com/huggingface/datasets/pull/2361

To reproduce:
```python
import numpy as np
from datasets import Dataset

d = Dataset.from_dict({"a": [np.zeros((2, 2))]})
```
raises
```python
Traceback (most recent call last):
  File "playground/ttest.py", line 5, in <module>
    d = Dataset.from_dict({"a": [np.zeros((2, 2))]}).with_format("torch")
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/arrow_dataset.py", line 458, in from_dict
    pa_table = InMemoryTable.from_pydict(mapping=mapping)
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/table.py", line 365, in from_pydict
    return cls(pa.Table.from_pydict(*args, **kwargs))
  File "pyarrow/table.pxi", line 1639, in pyarrow.lib.Table.from_pydict
  File "pyarrow/array.pxi", line 332, in pyarrow.lib.asarray
  File "pyarrow/array.pxi", line 223, in pyarrow.lib.array
  File "pyarrow/array.pxi", line 110, in pyarrow.lib._handle_arrow_array_protocol
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/arrow_writer.py", line 107, in __arrow_array__
    out = pa.array(self.data, type=type)
  File "pyarrow/array.pxi", line 306, in pyarrow.lib.array
  File "pyarrow/array.pxi", line 39, in pyarrow.lib._sequence_to_array
  File "pyarrow/error.pxi", line 143, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Can only convert 1-dimensional array values

Comments:
Unwanted progress bars when accessing examples
Opened by lhoestq (closed)

When accessing examples from a dataset formatted for pytorch, some progress bars appear when accessing examples:
```python
In [1]: import datasets as ds                                        

In [2]: d = ds.Dataset.from_dict({"a": [0, 1, 2]}).with_format("torch")                                                           

In [3]: d[0]                                                         
100%|████████████████████████████████| 1/1 [00:00<00:00, 3172.70it/s]
Out[3]: {'a': tensor(0)}
```

This is because the pytorch formatter calls `map_nested` that uses progress bars

cc @sgugger 

Comments:
doing a patch release now :)
windows download abnormal
Opened by wei1826676931 (closed)

## Describe the bug
The script clearly exists (accessible from the browser), but the script download fails on windows. Then I tried it again and it can be downloaded normally on linux. why??
## Steps to reproduce the bug
```python3.7 + windows
![image](https://user-images.githubusercontent.com/52347799/133436174-4303f847-55d5-434f-a749-08da3bb9b654.png)


# Sample code to reproduce the bug
```

## Expected results
It can be downloaded normally.

## Actual results
it cann't

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:1.11.0
- Platform:windows
- Python version:3.7
- PyArrow version:


Comments:
Hi ! Is there some kind of proxy that is configured in your browser that gives you access to internet ? If it's the case it could explain why it doesn't work in the code, since the proxy wouldn't be used

It is indeed an agency problem, thank you very, very much

Let me know if you have other questions :)

Closing this issue now
Add OpenAI's pass@k code evaluation metric
Opened by lvwerra (closed)

This PR introduces the `code_eval` metric which implements [OpenAI's code evaluation harness](https://github.com/openai/human-eval) introduced in the [Codex paper](https://arxiv.org/abs/2107.03374). It is heavily based on the original implementation and just adapts the interface to follow the `predictions`/`references` convention.

The addition of this metric should enable the evaluation against the code evaluation datasets added in #2897 and #2893.

A few open questions:

- The implementation makes heavy use of multiprocessing which this PR does not touch. Is this conflicting with multiprocessing natively integrated in `datasets`?
- This metric executes generated Python code and as such it poses dangers of executing malicious code. OpenAI addresses this issue by 1) commenting the `exec` call in the code so the user has to actively uncomment it and read the warning and 2) suggests using a sandbox environment (gVisor container). Should we add a similar safeguard? E.g. a prompt that needs to be answered when initialising the metric? Or at least a warning message?
- Naming: the implementation sticks to the `predictions`/`references` naming, however, the references are not reference solutions but unittest to test the solution. While reference solutions are also available they are not used. Should the naming be adapted?

Comments:
> The implementation makes heavy use of multiprocessing which this PR does not touch. Is this conflicting with multiprocessing natively integrated in datasets?

It should work normally, but feel free to test it.
There is some documentation about using metrics in a distributed setup that uses multiprocessing [here](https://huggingface.co/docs/datasets/loading.html?highlight=rank#distributed-setup)
You can test to spawn several processes where each process would load the metric. Then in each process you add some references/predictions to the metric. Finally you call compute() in each process and on process 0 it should return the result on all the references/predictions

Let me know if you have questions or if I can help

Is there a good way to debug the Windows tests? I suspect it is an issue with `multiprocessing`, but I can't see the error messages.

Indeed it has an issue on windows.
In your example it's supposed to output
```python
{'pass@1': 0.5, 'pass@2': 1.0}
```
but it gets
```python
{'pass@1': 0.0, 'pass@2': 0.0}
```

I'm not on my windows machine today so I can't take a look at it. I can dive into it early next week if you want

> I'm not on my windows machine today so I can't take a look at it. I can dive into it early next week if you want

That would be great - unfortunately I have no access to a windows machine at the moment. I am quite sure it is an issue with in exectue.py because of multiprocessing.
timit_asr dataset only includes one text phrase
Opened by margotwagner (closed)

## Describe the bug
The dataset 'timit_asr' only includes one text phrase. It only includes the transcription "Would such an act of refusal be useful?" multiple times rather than different phrases.

## Steps to reproduce the bug
Note: I am following the tutorial https://huggingface.co/blog/fine-tune-wav2vec2-english

1. Install the dataset and other packages
```python
!pip install datasets>=1.5.0
!pip install transformers==4.4.0
!pip install soundfile
!pip install jiwer
```
2. Load the dataset
```python
from datasets import load_dataset, load_metric

timit = load_dataset("timit_asr")
```
3. Remove columns that we don't want
```python
timit = timit.remove_columns(["phonetic_detail", "word_detail", "dialect_region", "id", "sentence_type", "speaker_id"])
```
4. Write a short function to display some random samples of the dataset.
```python
from datasets import ClassLabel
import random
import pandas as pd
from IPython.display import display, HTML

def show_random_elements(dataset, num_examples=10):
    assert num_examples <= len(dataset), "Can't pick more elements than there are in the dataset."
    picks = []
    for _ in range(num_examples):
        pick = random.randint(0, len(dataset)-1)
        while pick in picks:
            pick = random.randint(0, len(dataset)-1)
        picks.append(pick)
    
    df = pd.DataFrame(dataset[picks])
    display(HTML(df.to_html()))

show_random_elements(timit["train"].remove_columns(["file"]))
```

## Expected results
10 random different transcription phrases.

## Actual results
10 of the same transcription phrase "Would such an act of refusal be useful?"

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.4.1
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.5
- PyArrow version: not listed


Comments:
Hi @margotwagner, 
This bug was fixed in #1995. Upgrading the datasets should work (min v1.8.0 ideally)

Hi @margotwagner,

Yes, as @bhavitvyamalik has commented, this bug was fixed in `datasets` version 1.5.0. You need to update it, as your current version is 1.4.1:
> Environment info
> - `datasets` version: 1.4.1
fix anli splits
Opened by zaidalyafeai (closed)

I can't run the tests for dummy data, facing this error 

`ImportError while loading conftest '/home/zaid/tmp/fix_anli_splits/datasets/tests/conftest.py'.
tests/conftest.py:10: in <module>
    from datasets import config
E   ImportError: cannot import name 'config' from 'datasets' (unknown location)`

Comments:
feat: 🎸 add a function to get a dataset config's split names
Opened by severo (closed)

Also: pass additional arguments (use_auth_token) to get private configs + info of private datasets on the hub

Questions:

- [x] I'm not sure how the versions work: I changed 1.12.1.dev0 to 1.12.1.dev1, was it correct?
    -> no: reverted
- [x] Should I add a section in https://github.com/huggingface/datasets/blob/master/docs/source/load_hub.rst? (there is no section for get_dataset_infos)
    -> yes: added

Comments:
> Should I add a section in https://github.com/huggingface/datasets/blob/master/docs/source/load_hub.rst? (there is no section for get_dataset_infos)

Yes totally :) This tutorial should indeed mention this, given how fundamental it is
FORCE_REDOWNLOAD does not work
Opened by anoopkatti (open)

## Describe the bug
With GenerateMode.FORCE_REDOWNLOAD, the documentation says 
    +------------------------------------+-----------+---------+
    |                                    | Downloads | Dataset |
    +====================================+===========+=========+
    | `REUSE_DATASET_IF_EXISTS` (default)| Reuse     | Reuse   |
    +------------------------------------+-----------+---------+
    | `REUSE_CACHE_IF_EXISTS`            | Reuse     | Fresh   |
    +------------------------------------+-----------+---------+
    | `FORCE_REDOWNLOAD`                 | Fresh     | Fresh   |
    +------------------------------------+-----------+---------+

However, the old dataset is loaded even when FORCE_REDOWNLOAD is chosen.

## Steps to reproduce the bug
```python

import pandas as pd
from datasets import load_dataset, GenerateMode
pd.DataFrame(range(5), columns=['numbers']).to_csv('/tmp/test.tsv.gz', index=False)
ee = load_dataset('csv', data_files=['/tmp/test.tsv.gz'], delimiter='\t', split='train', download_mode=GenerateMode.FORCE_REDOWNLOAD)
print(ee)
pd.DataFrame(range(10), columns=['numerals']).to_csv('/tmp/test.tsv.gz', index=False)
ee = load_dataset('csv', data_files=['/tmp/test.tsv.gz'], delimiter='\t', split='train', download_mode=GenerateMode.FORCE_REDOWNLOAD)
print(ee)

```

## Expected results
Dataset({
    features: ['numbers'],
    num_rows: 5
})
Dataset({
    features: ['numerals'],
    num_rows: 10
})

## Actual results
Dataset({
    features: ['numbers'],
    num_rows: 5
})
Dataset({
    features: ['numbers'],
    num_rows: 5
})


## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.8.0
- Platform: Linux-4.14.181-108.257.amzn1.x86_64-x86_64-with-glibc2.10
- Python version: 3.7.10
- PyArrow version: 3.0.0


Comments:
Hi ! Thanks for reporting. The error seems to happen only if you use compressed files.

The second dataset is prepared in another dataset cache directory than the first - which is normal, since the source file is different. However, it doesn't uncompress the new data file because it finds the old uncompressed data in the extraction cache directory.

If we fix the extraction cache mechanism to uncompress a local file if it changed then it should fix the issue.
Currently the extraction cache mechanism only takes into account the path of the compressed file, which is an issue.

Facing the same issue, is there any way to overtake this issue until it will be fixed? 

You can clear your extraction cache in the meantime (by default at `~/.cache/huggingface/datasets/downloads/extracted`)
Add WIT Dataset
Opened by nateraw (closed)

## Adding a Dataset
- **Name:** *WIT*
- **Description:** *Wikipedia-based Image Text Dataset*
- **Paper:** *[WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning
](https://arxiv.org/abs/2103.01913)*
- **Data:** *https://github.com/google-research-datasets/wit*
- **Motivation:**  (excerpt from their Github README.md)

> - The largest multimodal dataset (publicly available at the time of this writing) by the number of image-text examples.
> - A massively multilingual dataset (first of its kind) with coverage for over 100+ languages.
> - A collection of diverse set of concepts and real world entities.
> - Brings forth challenging real-world test sets.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Comments:
@hassiahk is working on it  #2810 

WikiMedia is now hosting the pixel values directly which should make it a lot easier!
The files can be found here:
https://techblog.wikimedia.org/2021/09/09/the-wikipedia-image-caption-matching-challenge-and-a-huge-release-of-image-data-for-research/
https://analytics.wikimedia.org/published/datasets/one-off/caption_competition/training/image_pixels/

> @hassiahk is working on it #2810

Thank you @bhavitvyamalik! Added this issue so we could track progress 😄 . Just linked the PR as well for visibility. 

Hey folks, we are now hosting the merged pixel values + embeddings + metadata ourselves. I gave it a try - [nateraw/wit](https://huggingface.co/datasets/nateraw/wit)

**⚠️ - Make sure you add `streaming=True` unless you're prepared to download 400GB of data!**

```python
from datasets import load_dataset

ds = load_dataset('nateraw/wit', streaming=True)
example = next(iter(ds))
```

```python
>>> example = next(iter(ds['train']))
>>> example.keys()
dict_keys(['b64_bytes', 'original_width', 'image_url', 'wit_features', 'original_height', 'metadata_url', 'mime_type', 'caption_attribution_description', 'embedding'])
>>> example['wit_features'].keys()
dict_keys(['hierarchical_section_title', 'language', 'attribution_passes_lang_id', 'context_section_description', 'is_main_image', 'page_title', 'caption_title_and_reference_description', 'caption_alt_text_description', 'caption_reference_description', 'page_url', 'context_page_description', 'section_title', 'page_changed_recently'])
```

Hi! `datasets` now hosts two versions of the WIT dataset:
* [`google/wit`](https://huggingface.co/datasets/google/wit): Google's version with the image URLs
* [`wikimedia/wit_base`](https://huggingface.co/datasets/wikimedia/wit_base): Wikimedia's version with the images + ResNet embeddings, but with less data than Google's version

Does this dataset work with `data_files` parameter. I tried and it doesn't. it attempts to download ~600 GB data even when i specify single file
Dataset
Opened by rcacho172 (closed)

## Adding a Dataset
- **Name:** *name of the dataset*
- **Description:** *short description of the dataset (or link to social media or blog post)*
- **Paper:** *link to the dataset paper if available*
- **Data:** *link to the Github repository or current dataset location*
- **Motivation:** *what are some good reasons to have this dataset*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Comments:
Hug emoji
Opened by Jackg-08 (closed)

## Adding a Dataset
- **Name:** *name of the dataset*
- **Description:** *short description of the dataset (or link to social media or blog post)*
- **Paper:** *link to the dataset paper if available*
- **Data:** *link to the Github repository or current dataset location*
- **Motivation:** *what are some good reasons to have this dataset*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Comments:
Add OpenAI's HumanEval dataset
Opened by lvwerra (closed)

This PR adds OpenAI's [HumanEval](https://github.com/openai/human-eval) dataset. The dataset consists of 164 handcrafted programming problems with solutions and unittests to verify solution. This dataset is useful to evaluate code generation models.

Comments:
I just fixed the class name, and added `[More Information Needed]` in empty sections in case people want to complete the dataset card :)
add multi-proc in `to_csv`
Opened by bhavitvyamalik (closed)

This PR extends the multi-proc method used in #2747 for`to_json` to `to_csv` as well. 

Results on my machine post benchmarking on `ascent_kb` dataset (giving ~45% improvement when compared to num_proc = 1):
```
Time taken on 1 num_proc, 10000 batch_size  674.2055702209473
Time taken on 4 num_proc, 10000 batch_size  425.6553490161896

Time taken on 1 num_proc, 50000 batch_size  623.5897650718689
Time taken on 4 num_proc, 50000 batch_size  380.0402421951294

Time taken on 4 num_proc, 100000 batch_size  361.7168130874634
```
This is a WIP as writing tests is pending for this PR. 

I'm also exploring [this](https://arrow.apache.org/docs/python/csv.html#incremental-writing) approach for which I'm using `pyarrow-5.0.0`.


Comments:
I think you can just add a test `test_dataset_to_csv_multiproc` in `tests/io/test_csv.py` and we'll be good

Hi @lhoestq, 
I've added `test_dataset_to_csv` apart from `test_dataset_to_csv_multiproc` as no test was there to check generated CSV file when `num_proc=1`. Please let me know if anything is also required!
Use pyarrow.Table.replace_schema_metadata instead of pyarrow.Table.cast
Opened by arsarabi (closed)

This PR partially addresses #2252.

``update_metadata_with_features`` uses ``Table.cast`` which slows down ``load_from_disk`` (and possibly other methods that use it) for very large datasets. Since ``update_metadata_with_features`` is only updating the schema metadata, it makes more sense to use ``pyarrow.Table.replace_schema_metadata`` which is much faster. This PR adds a ``replace_schema_metadata`` method to all table classes, and modifies  ``update_metadata_with_features`` to use it instead of ``cast``.

Comments:
add mbpp dataset
Opened by lvwerra (closed)

This PR adds the mbpp dataset introduced by Google [here](https://github.com/google-research/google-research/tree/master/mbpp) as mentioned in #2816.

The dataset contain two versions: a full and a sanitized one. They have a slightly different schema and it is current state the loading preserves the original schema. An open question is whether to harmonize the two schemas when loading the dataset or to preserve the original one. Since not all fields are overlapping the schema will not be exactly the same.

Comments:
I think it's fine to have the original schema
Error when encoding a dataset with None objects with a Sequence feature
Opened by lhoestq (closed)

There is an error when encoding a dataset with None objects with a Sequence feature

To reproduce:
```python
from datasets import Dataset, Features, Value, Sequence
data = {"a": [[0], None]}
features = Features({"a": Sequence(Value("int32"))})
dataset = Dataset.from_dict(data, features=features)
```
raises

```python
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-24-40add67f8751> in <module>
      2 data = {"a": [[0], None]}
      3 features = Features({"a": Sequence(Value("int32"))})
----> 4 dataset = Dataset.from_dict(data, features=features)
[...]
~/datasets/features.py in encode_nested_example(schema, obj)
    888         if isinstance(obj, str):  # don't interpret a string as a list
    889             raise ValueError("Got a string but expected a list instead: '{}'".format(obj))
--> 890         return [encode_nested_example(schema.feature, o) for o in obj]
    891     # Object with special encoding:
    892     # ClassLabel will convert from string to int, TranslationVariableLanguages does some checks

TypeError: 'NoneType' object is not iterable
```

Instead, if should run without error, as if the `features` were not passed

Comments:
This has been fixed by https://github.com/huggingface/datasets/pull/2900
We're doing a new release 1.12 today to make the fix available :)
0x290B112ED1280537B24Ee6C268a004994a16e6CE
Opened by rcacho172 (closed)

## Adding a Dataset
- **Name:** *name of the dataset*
- **Description:** *short description of the dataset (or link to social media or blog post)*
- **Paper:** *link to the dataset paper if available*
- **Data:** *link to the Github repository or current dataset location*
- **Motivation:** *what are some good reasons to have this dataset*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Comments:
Coc
Opened by Bwiggity (closed)

## Adding a Dataset
- **Name:** *name of the dataset*
- **Description:** *short description of the dataset (or link to social media or blog post)*
- **Paper:** *link to the dataset paper if available*
- **Data:** *link to the Github repository or current dataset location*
- **Motivation:** *what are some good reasons to have this dataset*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Comments: