ahmedfarazsyk/datasets-github-issues huggingface dataset:
Dataset Viewer issue for vadis/sv-ident
Author: albertvillanova
State: closed
Labels: name: dataset-viewer

### Link

https://huggingface.co/datasets/vadis/sv-ident

### Description

The dataset preview does not work:
```
Server Error

Status code:   400
Exception:     Status400Error
Message:       The dataset does not exist.
```

However, the dataset is streamable and works locally:
```python
In [1]: from datasets import load_dataset; ds = load_dataset("sv-ident.py", split="train", streaming=True); item = next(iter(ds)); item
Using custom data configuration default
Out[1]: 
{'sentence': 'Our point, however, is that so long as downward (favorable) comparisons overwhelm the potential for unfavorable comparisons, system justification should be a likely outcome amongst the disadvantaged.',
 'is_variable': 1,
 'variable': ['exploredata-ZA5400_VarV66', 'exploredata-ZA5400_VarV53'],
 'research_data': ['ZA5400'],
 'doc_id': '73106',
 'uuid': 'b9fbb80f-3492-4b42-b9d5-0254cc33ac10',
 'lang': 'en'}
```

CC: @e-tornike

### Owner

No

Comments: Fixed, thanks!
![Uploading Capture d’écran 2022-06-21 à 18.42.40.png…]()
split cache used when processing different split
Author: gpucce
State: open
Labels: name: bug

## Describe the bug`
```
ds1 = load_dataset('squad', split='validation')
ds2 = load_dataset('squad', split='train')
ds1 = ds1.map(some_function)
ds2 = ds2.map(some_function)
assert ds1 == ds2
```
This happens when ds1 and ds2 are created in `pytorch_lightning.DataModule` through

```
class myDataModule:

    def train_dataloader(self):
         ds = load_dataset('squad', split='train')
         ds = ds.map(some_function)
         return [ds]

    def val_dataloader(self):
         ds = load_dataset('squad', split="validation")
         ds = ds.map(some_function)
         return [ds]
```
I don't know if it depends on `pytorch_lightning` or `datasets` but setting `ds.map(some_function, load_from_cache_file=False)` fixes the issue.

If this is not enough to replicate I will try and provide and MWE, I don't have time now so I thought I wuld open the issue first!

Comments: I was not able to reproduce this behavior (I tried without using pytorch lightning though, since I don't know what code you ran in pytorch lightning to get this).

If you can provide a MWE that would be perfect ! :)

Hi, I think the issue happened because I was loading datasets under an `if` ... `else` statement and the condition would change the dataset I would need to load but instead the cached one was always returned. However, I believe that is expected behaviour, if so I'll close the issue.

Otherwise I will try to provide a MWE
Try to reduce the number of datasets that require manual download
Author: severo
State: open

> Currently, 41 canonical datasets require manual download. I checked their scripts and I'm pretty sure this number can be reduced to ≈ 30 by not relying on bash scripts to download data, hosting data directly on the Hub when the license permits, etc. Then, we will mostly be left with datasets with restricted access, which we can ignore

from https://github.com/huggingface/datasets-server/issues/12#issuecomment-1026920432
Datasets method `.map` not hashing
Author: sanchit-gandhi
State: closed
Labels: name: bug

## Describe the bug
Datasets method `.map` not hashing, even with an empty no-op function

## Steps to reproduce the bug
```python
from datasets import load_dataset

# download 9MB dummy dataset
ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean")

def prepare_dataset(batch):
     return(batch)

ds = ds.map(
    prepare_dataset,
    num_proc=1,
    desc="preprocess train dataset",
)
```

## Expected results
Hashed and cached dataset preprocessing

## Actual results
Does not hash properly:
```
Parameter 'function'=<function prepare_dataset at 0x7fccb68e9280> of the transform datasets.arrow_dataset.Dataset._map_single couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting and caching to work. If you reuse this transform, the caching mechanism will consider it to be different from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed.
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.3.3.dev0
- Platform: Linux-5.11.0-1028-gcp-x86_64-with-glibc2.31
- Python version: 3.9.12
- PyArrow version: 8.0.0
- Pandas version: 1.4.2

cc @lhoestq 


Comments: Fix posted: https://github.com/huggingface/datasets/issues/4506#issuecomment-1157417219

Didn't realize it's a bug when I asked the question yesterday! Free free to post an answer if you are sure the cause has been addressed.

https://stackoverflow.com/questions/72664827/can-pickle-dill-foo-but-not-lambda-x-foox

Thank @nalzok . That works for me:

`pip install "dill<0.3.5"`
Allow .JPEG as a file extension
Author: DiGyt
State: closed
Labels: name: bug

## Describe the bug
When loading image data, HF datasets seems to recognize `.jpg` and `.jpeg` file extensions, but not e.g. .JPEG. As the naming convention .JPEG is used in important datasets such as imagenet, I would welcome if according extensions like .JPEG or .JPG would be allowed.

## Steps to reproduce the bug
```python
# use bash to create 2 sham datasets with jpeg and JPEG ext
!mkdir dataset_a
!mkdir dataset_b
!wget https://upload.wikimedia.org/wikipedia/commons/7/71/Dsc_%28179253513%29.jpeg -O example_img.jpeg
!cp example_img.jpeg ./dataset_a/
!mv example_img.jpeg ./dataset_b/example_img.JPEG

from datasets import load_dataset

# working
df1 = load_dataset("./dataset_a", ignore_verifications=True)

#not working
df2 = load_dataset("./dataset_b", ignore_verifications=True)

# show
print(df1, df2)
```

## Expected results
```
DatasetDict({
    train: Dataset({
        features: ['image', 'label'],
        num_rows: 1
    })
}) DatasetDict({
    train: Dataset({
        features: ['image', 'label'],
        num_rows: 1
    })
})
```

## Actual results
```
FileNotFoundError: Unable to resolve any data file that matches '['**']' at /..PATH../dataset_b with any supported extension ['csv', 'tsv', 'json', 'jsonl', 'parquet', 'txt', 'blp', 'bmp', 'dib', 'bufr', 'cur', 'pcx', 'dcx', 'dds', 'ps', 'eps', 'fit', 'fits', 'fli', 'flc', 'ftc', 'ftu', 'gbr', 'gif', 'grib', 'h5', 'hdf', 'png', 'apng', 'jp2', 'j2k', 'jpc', 'jpf', 'jpx', 'j2c', 'icns', 'ico', 'im', 'iim', 'tif', 'tiff', 'jfif', 'jpe', 'jpg', 'jpeg', 'mpg', 'mpeg', 'msp', 'pcd', 'pxr', 'pbm', 'pgm', 'ppm', 'pnm', 'psd', 'bw', 'rgb', 'rgba', 'sgi', 'ras', 'tga', 'icb', 'vda', 'vst', 'webp', 'wmf', 'emf', 'xbm', 'xpm', 'zip']
```

I know that it can be annoying to allow seemingly arbitrary numbers of file extensions. But I think this one would be really welcome.

Comments: Hi, thanks for reporting!  I've opened a PR with the fix.

Wow, that was quick! Thank you very much 🙏
cast_storage method from datasets.features
Author: romainremyb
State: closed
Labels: name: bug

## Describe the bug
A bug occurs when mapping a function to a dataset object. I ran the same code with the same data yesterday and it worked just fine. It works when i run locally on an old version of datasets.

## Steps to reproduce the bug
Steps are:
- load whatever datset
- write a preprocessing function such as "tokenize_and_align_labels" written in https://huggingface.co/docs/transformers/tasks/token_classification
- map the function on dataset and get "ValueError: Class label -100 less than -1" from cast_storage method from datasets.features

# Sample code to reproduce the bug
def tokenize_and_align_labels(examples):
    tokenized_inputs = tokenizer(examples["tokens"], truncation=True, is_split_into_words=True, max_length=38,padding="max_length")

    labels = []
    for i, label in enumerate(examples[f"labels"]):
        word_ids = tokenized_inputs.word_ids(batch_index=i)  # Map tokens to their respective word.
        previous_word_idx = None
        label_ids = []
        for word_idx in word_ids:  # Set the special tokens to -100.
            if word_idx is None:
                label_ids.append(-100)
            elif word_idx != previous_word_idx:  # Only label the first token of a given word.
                label_ids.append(label[word_idx])
            else:
                label_ids.append(-100)
            previous_word_idx = word_idx
        labels.append(label_ids)

    tokenized_inputs["labels"] = labels
    return tokenized_inputs

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")
dt = dataset.map(tokenize_and_align_labels, batched=True)

## Expected results
New dataset objects should load and do on older versions.

## Actual results
"ValueError: Class label -100 less than -1" from cast_storage method from datasets.features

## Environment info
everything works fine on older installations of datasets/transformers

Issue arises when installing datasets on google collab under python3.7
I can't manage to find the exact output you're requirering but version printed is datasets-2.3.2


Comments: Hi! We've recently added a check to the `ClassLabel` type to ensure the values are in the valid label range `-1, 0, ..., num_classes-1` (-1 is used for missing values). The error in your case happens only if the `labels` column is of type `Sequence(ClassLabel(...))` before the `map` call and can be avoided by calling `dataset = dataset.cast_column("labels", Sequence(Value("int")))` beforehand. The token-classification examples in Transformers introduce a new `labels` column, so their type is also `Sequence(Value("int"))`, which doesn't lead to an error as this type unbounded. 

I'm fine with re-adding support for all negative values for unknown/missing labels @mariosasko, wdyt ?
How to let `load_dataset` return a `Dataset` instead of `DatasetDict` in customized loading script
Author: liyucheng09
State: closed
Labels: name: enhancement

If the dataset does not need splits, i.e., no training and validation split, more like a table. How can I let the `load_dataset` function return a `Dataset` object directly rather than return a `DatasetDict` object with only one key-value pair.

Or I can paraphrase the question in the following way: how to skip `_split_generators` step in `DatasetBuilder` to let `as_dataset` gives a single `Dataset` rather than a list`[Dataset]`?

Many thanks for any help.

Comments: Hi @liyucheng09.

Users can pass the `split` parameter to `load_dataset`. For example, if your split name is "train",
```python
ds = load_dataset("dataset_name", split="train")
```
will return a `Dataset` instance.

@albertvillanova Thanks! I can't believe I didn't know this feature till now.
WER and CER > 1
Author: sadrasabouri
State: closed
Labels: name: bug

## Describe the bug
It seems that in some cases in which the `prediction` is longer than the `reference` we may have word/character error rate higher than 1 which is a bit odd.

If it's a real bug I think I can solve it with a PR changing [this](https://github.com/huggingface/datasets/blob/master/metrics/wer/wer.py#L105) line to
```python
return min(incorrect / total, 1.0)
```

## Steps to reproduce the bug
```python
from datasets import load_metric
wer = load_metric("wer")
wer_value = wer.compute(predictions=["Hi World vka"], references=["Hello"])
print(wer_value)
```

## Expected results
```
1.0
```

## Actual results
```
3.0
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.3.0
- Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.13
- PyArrow version: 6.0.1
- Pandas version: 1.3.5

Comments: WER can have values bigger than 1.0, this is expected when there are too many insertions

From [wikipedia](https://en.wikipedia.org/wiki/Word_error_rate):
> Note that since N is the number of words in the reference, the word error rate can be larger than 1.0
Patching fails for modules that are not installed or don't exist
Author: lhoestq
State: closed

Reported in https://github.com/huggingface/huggingface_hub/runs/6894703718?check_suite_focus=true

When trying to patch `scipy.io.loadmat`:

```python
ModuleNotFoundError: No module named 'scipy'
```

Instead it shouldn't raise an error and do nothing

We use patching to extend such functions to support remote URLs and work in streaming mode
Dataset Viewer issue for Pavithree/test
Author: Pavithree
State: closed
Labels: name: dataset-viewer

### Link

https://huggingface.co/datasets/Pavithree/test

### Description

I have extracted the subset of original eli5 dataset found at hugging face. However, while loading the dataset It throws ArrowNotImplementedError: Unsupported cast from string to null using function cast_null error. Is there anything missing from my end? Kindly help.

### Owner

_No response_

Comments: This issue can be resolved according to this post https://stackoverflow.com/questions/70566660/parquet-with-null-columns-on-pyarrow. It looks like first data entry in the json file must not have any null values as pyarrow uses this first file to infer schema for entire dataset.
Use `torch.nested_tensor` for arrays of varying length in torch formatter
Author: mariosasko
State: open
Labels: name: enhancement

Use `torch.nested_tensor` for arrays of varying length in `TorchFormatter`.

The PyTorch API of nested tensors is in the prototype stage, so wait for it to become more mature.

Comments: What's the current behavior?

Currently, we return a list of Torch tensors if their shapes don't match. If they do, we consolidate them into a single Torch tensor.
Dataset Viewer issue for fgrezes/WIESP2022-NER
Author: AshTayade
State: closed

### Link

_No response_

### Description

_No response_

### Owner

_No response_

Comments: https://huggingface.co/datasets/fgrezes/WIESP2022-NER

The error:

```
Message:       Couldn't find a dataset script at /src/services/worker/fgrezes/WIESP2022-NER/WIESP2022-NER.py or any data file in the same directory. Couldn't find 'fgrezes/WIESP2022-NER' on the Hugging Face Hub either: FileNotFoundError: Unable to resolve any data file that matches ['**test*', '**eval*'] in dataset repository fgrezes/WIESP2022-NER with any supported extension ['csv', 'tsv', 'json', 'jsonl', 'parquet', 'txt', 'blp', 'bmp', 'dib', 'bufr', 'cur', 'pcx', 'dcx', 'dds', 'ps', 'eps', 'fit', 'fits', 'fli', 'flc', 'ftc', 'ftu', 'gbr', 'gif', 'grib', 'h5', 'hdf', 'png', 'apng', 'jp2', 'j2k', 'jpc', 'jpf', 'jpx', 'j2c', 'icns', 'ico', 'im', 'iim', 'tif', 'tiff', 'jfif', 'jpe', 'jpg', 'jpeg', 'mpg', 'mpeg', 'msp', 'pcd', 'pxr', 'pbm', 'pgm', 'ppm', 'pnm', 'psd', 'bw', 'rgb', 'rgba', 'sgi', 'ras', 'tga', 'icb', 'vda', 'vst', 'webp', 'wmf', 'emf', 'xbm', 'xpm', 'zip']
```

I understand the issue is not related to the dataset viewer in itself, but with the autodetection of the data files without a loading script in the datasets library. cc @lhoestq @albertvillanova @mariosasko 

Apparently it finds `scoring-scripts/compute_seqeval.py` which matches `**eval*`, a regex that detects a test split. We should probably improve the regex because it's not supposed to catch this kind of files. It must also only check for files with supported extensions: txt, csv, png etc.
`to_pandas` doesn't take into account format.
Author: Dref360
State: closed
Labels: name: enhancement

**Is your feature request related to a problem? Please describe.**

I have a large dataset that I need to convert part of to pandas to do some further analysis. Calling `to_pandas` directly on it is expensive. So I thought I could simply select the columns that I want and then call `to_pandas`.

**Describe the solution you'd like**

```python
from datasets import Dataset

ds = Dataset.from_dict({'a': [1,2,3], 'b': [5,6,7], 'c': [8,9,10]})
pandas_df = ds.with_format(columns=['a', 'b']).to_pandas()

# I would expect `pandas_df` to only include a,b as column.
```


**Describe alternatives you've considered**
I could remove all columns that I don't want? But I don't know all of them in advance. 

**Additional context**
I can probably make a PR with some pointers.


Comments: Thanks for opening a discussion :)

Note that you can use `.remove_columns(...)` to keep only the ones you're interested in before calling `.to_pandas()`

Yes I can do that thank you!

Do you think that conceptually my example should work? If not, I'm happy to close this issue. 

If yes, I can start working on it.

Hi! Instead of `with_format(columns=['a', 'b']).to_pandas()`, use `with_format("pandas", columns=["a", "b"])` for easy conversion of the parts of the dataset to pandas via indexing/slicing.

The full code:
```python
from datasets import Dataset

ds = Dataset.from_dict({'a': [1,2,3], 'b': [5,6,7], 'c': [8,9,10]})
pandas_df = ds.with_format("pandas", columns=['a', 'b'])[:]
```

Ahhhh Thank you!

closing then :)
CI error with repo lhoestq/_dummy
Author: albertvillanova
State: closed
Labels: name: bug

## Describe the bug
CI is failing because of repo "lhoestq/_dummy". See: https://app.circleci.com/pipelines/github/huggingface/datasets/12461/workflows/1b040b45-9578-4ab9-8c44-c643c4eb8691/jobs/74269
```
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/api/datasets/lhoestq/_dummy?full=true
```

The repo seems to no longer exist: https://huggingface.co/api/datasets/lhoestq/_dummy
```
error: "Repository not found"
```

CC: @lhoestq 

Comments: fixed by https://github.com/huggingface/datasets/pull/4472
BigBench: NonMatchingSplitsSizesError when passing a dataset configuration parameter
Author: lhoestq
State: open
Labels: name: bug

As noticed in https://github.com/huggingface/datasets/pull/4125 when a dataset config class has a parameter that reduces the number of examples (e.g. named `max_examples`), then loading the dataset and passing `max_examples` raises `NonMatchingSplitsSizesError`.

This is because it will check for expected the number of examples of the config with the same name without taking into account the `max_examples` parameter. This can be fixed by checking the expected number of examples using the **config id** instead of name. Indeed the config id corresponds to the config name + an optional suffix that depends on the config parameters

Comments: Why not adding `max_examples` as part of the config name?

Yup it can also work, and maybe it's simpler this way. Opening a PR to fix bigbench instead of https://github.com/huggingface/datasets/pull/4463

Hi @lhoestq,

Thank you for taking a look at this issue, and proposing a solution. 
Unfortunately, after trying the fix in #4465 I still see the same issue.

I think there is some subtlety where the config name gets overwritten somewhere when `BUILDER_CONFIGS`[(link)](https://github.com/huggingface/datasets/blob/master/datasets/bigbench/bigbench.py#L126) is defined. 

If I print out the `self.config.name` in the current version (with the fix in #4465), I see just the task name, but if I comment out `BUILDER_CONFIGS`, the `num_shots` and `max_examples` gets appended as was meant by #4465.

I haven't managed to track down where this happens, but I thought you might know? 

(Another comment on your fix: the `name` variable is used to fetch the task from the bigbench API, so modifying it causes an error if it's actually called. This can easily be fixed by having `config_name` variable in addition to the `task_name`)
AttributeError: module 'datasets' has no attribute 'load_dataset'
Author: AlexNLP
State: closed
Labels: name: bug

## Describe the bug
I have piped install datasets, but this package doesn't have these attributes: load_dataset, load_metric.

## Environment info
- `datasets` version: 1.9.0
- Platform: Linux-5.13.0-44-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.6.13
- PyArrow version: 6.0.1



Comments: I'm having the same issue，Can you tell me how to solve it？

I have the same issue, can you tell me how to solve it? Thanks

I had a folder named 'datasets' so this is why it can't find the import, it's looking in the wrong place

@briandw your comment saved my day 👍
Workflow for Tabular data
Author: lhoestq
State: open
Labels: name: enhancement

name: generic discussion

Tabular data are treated very differently than data for NLP, audio, vision, etc. and therefore the worflow for tabular data in `datasets` is not ideal.

For example for tabular data, it is common to use pandas/spark/dask to process the data, and then load the data into X and y (X is an array of features and y an array of labels), then train_test_split and finally feed the data to a machine learning model.

In `datasets` the workflow is different: we use load_dataset, then map, then train_test_split (if we only have a train split) and we end up with columnar dataset splits, not formatted as X and y.

Right now, it is already possible to convert a dataset from and to pandas, but there are still many things that could improve the workflow for tabular data:
- be able to load the data into X and y
- be able to load a dataset from the output of spark or dask (as far as I know it's usually csv or parquet files on S3/GCS/HDFS etc.)
- support "unsplit" datasets explicitly, instead of putting everything in "train" by default

cc @adrinjalali @merveenoyan feel free to complete/correct this :)

Feel free to also share ideas of APIs that would be super intuitive in your opinion !

Comments: I use below to load a dataset:
```
dataset = datasets.load_dataset("scikit-learn/auto-mpg")
df = pd.DataFrame(dataset["train"])
```
TBH as said, tabular folk split their own dataset, they sometimes have two splits, sometimes three. Maybe somehow avoiding it for tabular datasets might be good for later. (it's just UX improvement) 

is very slow batch access of a dataset (tabular, csv) with many columns to be expected?

Define "many" ? x)

~20k! I was surprised batch loading with as few as 32 samples was really slow. I was speculating the columnar format was the cause -- or do you see good performance with this approx size of tabular data?

20k can be a lot for a columnar format but maybe we can optimize a few things.

It would be cool to profile the code to see if there's an unoptimized part of the code that slows everything down.

(it's also possible to kill the job when it accesses the batch, it often gives you the traceback at the location where the code was running)

FWIW I've worked with tabular data with 540k columns.

thats awesome, whats your secret? would love to see an example!

@wconnell I'm not sure what you mean by my secret, I load them into a numpy array 😁 

An example dataset is [here](https://portal.gdc.cancer.gov/repository?facetTab=files&filters=%7B%22content%22%3A%5B%7B%22content%22%3A%7B%22field%22%3A%22cases.project.project_id%22%2C%22value%22%3A%5B%22TCGA-CESC%22%5D%7D%2C%22op%22%3A%22in%22%7D%2C%7B%22content%22%3A%7B%22field%22%3A%22files.data_category%22%2C%22value%22%3A%5B%22DNA%20Methylation%22%5D%7D%2C%22op%22%3A%22in%22%7D%5D%2C%22op%22%3A%22and%22%7D&searchTableTab=files) which is a dataset of DNA methylation reads. This dataset is about 950 rows and 450k columns.
Dataset Viewer issue for Yaxin/SemEval2015
Author: WithYouTo
State: closed

### Link

_No response_

### Description

_No response_

### Owner

_No response_

Comments: I understand that the issue is that a remote file (URL) is being loaded as a local file. Right @albertvillanova @lhoestq?

```
Message:       [Errno 2] No such file or directory: 'https://raw.githubusercontent.com/YaxinCui/ABSADataset/main/SemEval2015Task12Corrected/train/restaurants_train.xml'
```

`xml.dom.minidom.parse` is not supported in streaming mode. I opened a PR here to fix it:
https://huggingface.co/datasets/Yaxin/SemEval2015/discussions/1

Please review the PR @WithYouTo and let me know if it works !

Additionally, I'm also patching our library, so that we support streaming datasets that use `xml.dom.minidom.parse`.