ahmedfarazsyk/datasets-github-issues huggingface dataset:
Unexpected behavior of _TempDirWithCustomCleanup
Author: JonasGeiping
State: open
Labels: name: bug

## Describe the bug
This is not 100% a bug in `datasets`, but behavior that surprised me and I think this could be made more robust on the `datasets`side. 

When using `datasets.disable_caching()`, cache files are written to a temporary directory. This directory should be based on the environment variable TMPDIR. I want to set TMPDIR at runtime using os.ENVIRON["TMPDIR"] = something, but depending on other imported modules this can fail to take effect. 

## Steps to reproduce the bug
`_TempDirWithCustomCleanup` relies on `tempfile` to generate a path to a temporary directory. However, `tempfile` generates the path only once. This can be a problem when trying to set TMPDIR at runtime whenever other code imports `tempfile` first and does something unexpected. 

For example (after too much trial and error) I found out that a different part of the code base I work with defines a class `PatchedDataCollatorForLanguageModeling(transformers.DataCollatorForLanguageModeling)` based on a `transformers` class.  This import is enough to trigger `tempfile` to generate `tempfile` to generate a temporary path and leading to the wrong path being cached in `tempfile.tempdir`.

## Suggestion:
I could file this also as bug with `transformers`, but I think fixing this on the datasets would be much more robust:
Datasets could recompute the temporary path once (technically possible via `tempfile._get_default_tempdir` or resetting 
 the global variable `tempfile.tmpdir` to None) before setting its own global  `_TEMP_DIR_FOR_TEMP_CACHE_FILES`.




Comments: Hi ! Would setting TMPDIR at the beginning of your python script/session work ? I mean, even before importing transformers, datasets, etc. and using them ? I think this would be the most robust solution given any library that uses `tempfile`. I don't think we aim to support environment variables to be changed at run time

Hi, yeah setting the environment variable before the imports / as environment variable outside is another way to fix this. I am just arguing that `datasets` already uses its own global variable to track temporary files: `_TEMP_DIR_FOR_TEMP_CACHE_FILES`, and the creation of this global variable should respect TMPDIR instead of relying on tempfile to do so.
Modify datatype from `int32` to `float` for pearsonr, spearmanr.
Author: woodywarhol9
State: closed
Labels: name: enhancement

**Is your feature request related to a problem? Please describe.**
- Now [Pearsonr](https://github.com/huggingface/datasets/blob/master/metrics/pearsonr/pearsonr.py) and [Spearmanr](https://github.com/huggingface/datasets/blob/master/metrics/spearmanr/spearmanr.py) both get input data as 'int32'. 


**Describe the solution you'd like**
- Considering that those metrics are widely used for the STS task(labels are in 'float' data type),
it would be better to modify datatype from 'int32' to 'float' for getting exact values of similarity.

Comments: @Woodywarhol9 good catch, thanks for reporting.

We are fixing this.
metric = metric_cls( TypeError: 'NoneType' object is not callable
Author: klyuhang9
State: closed

Hi, friend. I meet a problem.
When I run the code:
`metric = load_metric('glue', 'rte')`

There is a problem raising:
`metric = metric_cls(
TypeError: 'NoneType' object is not callable `

I don't know why. Thanks for your help!



Comments: Hi @klyuhang9,

I'm sorry but I can't reproduce your problem:
```python
In [2]: metric = load_metric('glue', 'rte')
Downloading builder script: 5.76kB [00:00, 2.40MB/s]
```

Could you please, retry to load the metric? Sometimes there are temporary connectivity issues.

Feel free to re-open this issue of the problem persists.
Split size error on `amazon_us_reviews` / `PC_v1_00` dataset
Author: trentonstrong
State: closed
Labels: name: bug

name: good first issue

## Describe the bug
When downloading this subset as of 3-28-2022 you will encounter a split size error after the dataset is extracted. The extracted dataset has roughly ~6m rows while the split expects <1m. 

Upon digging a little deeper, I downloaded the raw files from `https://s3.amazonaws.com/amazon-reviews-pds/tsv/amazon_reviews_us_PC_v1_00.tsv.gz` and extracted them. A line count via `wc -l` confirms the ~6m number that we see and the data looks valid at a glance (I did not check for duplicate rows). My guess is this file has either been updated in place or there is a bug in the dataset metadata.

Happy to submit a PR and fix this up if turns out to be a metadata issue but wanted to get some other :eyes: on it first.

## Steps to reproduce the bug
```python
load_dataset('amazon_us_reviews', 'PC_v1_00')
```

## Expected results
Dataset is downloaded and extracted successfully.

## Actual results
An split size exception is thrown.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.0.0
- Platform: Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 7.0.0
- Pandas version: 1.4.1


Comments: Follow-up: I have confirmed there are no duplicate lines via `sort amazon_reviews_us_PC_v1_00.tsv | uniq -cd` after extracting the raw file.

Hi @trentonstrong, thanks for reporting!

I confirm that loading this dataset configuration throws a `NonMatchingSplitsSizesError`:
```
NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=350242049, num_examples=785730, dataset_name='amazon_us_reviews'), 'recorded': SplitInfo(name='train', num_bytes=3982712078, num_examples=6908554, dataset_name='amazon_us_reviews')}]
```

Also thank you for your offer to fix this. You can find information about how to update the metadata JSON file here: https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#automatically-add-code-metadata
```shell
datasets-cli test datasets/amazon_us_reviews --save_infos --all_configs
```
Please, feel free to open a PR with this fix. And do not hesitate to ping me if you need any help.

No sweat. Will get it patched up ASAP.
Dataset.unique(column: str) -> ArrowNotImplementedError
Author: orkenstein
State: closed
Labels: name: bug

## Describe the bug
I'm trying to use `unique()` function, but it fails
## Steps to reproduce the bug
1. Get dataset
2. Call `unique`
3. Error

# Sample code to reproduce the bug
```python
!pip show datasets

from datasets import load_dataset

dataset = load_dataset('wikiann', 'en')

dataset['train'].column_names
dataset['train'].unique(dataset['train'].column_names[0])
```

## Expected results
It would be nice to actually see unique items

## Actual results
Error:
```python
---------------------------------------------------------------------------
ArrowNotImplementedError                  Traceback (most recent call last)
[<ipython-input-10-5e0de07ed42c>](https://s0qyv2vjaji-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20220324-060046-RC00_436956229#) in <module>()
      6 
      7 dataset['train'].column_names
----> 8 dataset['train'].unique(dataset['train'].column_names[0])

5 frames
/usr/local/lib/python3.7/dist-packages/pyarrow/error.pxi in pyarrow.lib.check_status()

ArrowNotImplementedError: Function unique has no kernel matching input types (array[list<item: string>])
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.0.0
- Platform: Google Collab
- Python version: 3.7.13
- PyArrow version: 6.0.1


Comments: Hi @orkenstein, thanks for reporting.

Please note that for this case, our `datasets` library uses under the hood the Apache Arrow `unique` function: https://arrow.apache.org/docs/python/generated/pyarrow.compute.unique.html#pyarrow.compute.unique

And currently the Apache Arrow `unique` function is only implemented for these input types (see info in their [docs](https://arrow.apache.org/docs/cpp/compute.html#array-wise-vector-functions)): Boolean, Null, Numeric, Temporal, Binary- and String-like.

However, the data types of the `wikiann` dataset are all `list<item: string>` (see its [dataset card](https://huggingface.co/datasets/wikiann#data-fields)), and thus, not yet supported by the Apache Arrow `unique` function.

As a workaround solution you can use pandas:
```python
from datasets import load_dataset

dataset = load_dataset('wikiann', 'en', split='train')
df = dataset.to_pandas()
unique_df = df[~df.tokens.apply(tuple).duplicated()]  # from https://stackoverflow.com/a/46958336/17517845
```

Note that pandas loads the dataset in memory (this one is small so it's fine).

@lhoestq thank you! I will fall back to this method for now
CLI dummy data generation is broken
Author: albertvillanova
State: closed
Labels: name: bug

## Describe the bug
We get a TypeError when running CLI dummy data generation:
```shell
datasets-cli dummy_data datasets/<your-dataset-folder> --auto_generate
```
gives:
```
  File ".../huggingface/datasets/src/datasets/commands/dummy_data.py", line 361, in _autogenerate_dummy_data
    dataset_builder._prepare_split(split_generator)
TypeError: _prepare_split() missing 1 required positional argument: 'check_duplicate_keys'
```
Error while building documentation
Author: albertvillanova
State: closed
Labels: name: bug

## Describe the bug
Documentation building is failing:
- https://github.com/huggingface/datasets/runs/5716300989?check_suite_focus=true

```
ValueError: There was an error when converting ../datasets/docs/source/package_reference/main_classes.mdx to the MDX format.
Unable to find datasets.filesystems.S3FileSystem in datasets. Make sure the path to that object is correct.
```



Comments: After some investigation, maybe the bug is in `doc-builder`.

I've opened an issue there:
- huggingface/doc-builder#160

Fixed by @lewtun (thank you):
- huggingface/doc-builder@31fe6c8bc7225810e281c2f6c6cd32f38828c504
can't download cats_vs_dogs dataset
Author: RRaphaell
State: closed
Labels: name: bug

## Describe the bug
can't download cats_vs_dogs dataset. error: Checksums didn't match for dataset source files

## Steps to reproduce the bug
```python
from datasets import load_dataset
dataset = load_dataset("cats_vs_dogs")
```

## Expected results
loaded successfully.

## Actual results
NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_3367a.zip']

## Environment info
fresh google colab notebook


Comments: Thnaks for reporting @RRaphaell.

We are fixing it.
Cannot load the dataset conll2012_ontonotesv5
Author: cathyxl
State: closed
Labels: name: bug

## Describe the bug
Cannot load the dataset conll2012_ontonotesv5

## Steps to reproduce the bug
```python
# Sample code to reproduce the bug
from datasets import load_dataset
dataset = load_dataset('conll2012_ontonotesv5', 'english_v4', split="test")
print(dataset)

```

## Expected results
The datasets should be downloaded successfully

## Actual results
raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://md-datasets-cache-zipfiles-prod.s3.eu-west-1.amazonaws.com/zmycy7t9h9-1.zip']

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.0.0
- Platform: Linux-5.4.0-88-generic-x86_64-with-glibc2.31
- Python version: 3.9.7
- PyArrow version: 7.0.0



Comments: Hi @cathyxl, thanks for reporting.

Indeed, we have recently updated the loading script of that dataset (and fixed that bug as well):
- #4002

That fix will be available in our next `datasets` library release. In the meantime, you can incorporate that fix by:
- installing `datasets` from our GitHub repo:
```bash
pip install git+https://github.com/huggingface/datasets#egg=datasets
```
- forcing the data files to be redownloaded
```python
ds = load_dataset('conll2012_ontonotesv5', 'english_v4', split="test", download_mode="force_redownload")
```

Feel free to re-open this issue if the problem persists.
Add FAISS .range_search() method for retrieving all texts from dataset above similarity threshold
Author: MoritzLaurer
State: closed
Labels: name: enhancement

**Is your feature request related to a problem? Please describe.**
I would like to retrieve all texts from a dataset, which are semantically similar to a specific input text (query), above a certain (cosine) similarity threshold. My dataset is very large (Wikipedia), so I need to use Datasets and FAISS for this. I would like to be able to repeat many different queries on the dataset quickly. 

**Describe the solution you'd like**
dataset objects currently have the .get_nearest_examples() method for text retrieval via FAISS. But this only allows retrieving a specific number of K texts instead of everything above a specified similarity threshold. 
It would be great if HF Datasets would also support the FAISS method .range_search() for retrieving texts above a certain similarity threshold. 
see details here: https://github.com/facebookresearch/faiss/issues/1273

**Describe alternatives you've considered**
I've considered using native FAISS, but doing this via HF datasets would be better. My assumption is that Dataset features like dataset streaming make it easier to work with large datasets


**Additional context**
The concrete use-case is: I have a large dataset (wikipedia) and I would like to retrieve all paragraphs which are similar to a query. I will use sentence-transformers for encoding the texts. 


Comments: Hi ! You can access the faiss index with
```python
faiss_index = my_dataset.get_index("my_index_name").faiss_index
```
and then do whatever you want with it, e.g. query it using range_search:
```python
threshold = 0.95
limits, distances, indices = faiss_index.range_search(x=xq, thresh=threshold)

texts = dataset[indices]
```

wow, that's great, thank you for the explanation. (if that's not already in the documentation, could be worth adding it)

which type of faiss index is Datasets using? I looked into faiss recently and I understand that there are several different types of indexes and the choice is important, e.g. regarding which distance metric you use (euclidian vs. cosine/dot product), the size of my dataset etc. can I chose the type of index somehow as well?

`Dataset.add_faiss_index` has a `string_factory` parameter, used to set the type of index (see the faiss documentation about [index factory](https://github.com/facebookresearch/faiss/wiki/The-index-factory)). Alternatively, you can pass an index you've defined yourself using faiss with the `custom_index` parameter of `Dataset.add_faiss_index` 

Here is the full documentation of `Dataset.add_faiss_index`: https://huggingface.co/docs/datasets/v2.0.0/en/package_reference/main_classes#datasets.Dataset.add_faiss_index

great thanks, I will try it out
ElasticSearch Indexing example: TypeError: __init__() missing 1 required positional argument: 'scheme'
Author: MoritzLaurer
State: closed
Labels: name: bug

name: duplicate

## Describe the bug
I am following the example in the documentation for elastic search step by step (on google colab): https://huggingface.co/docs/datasets/faiss_es#elasticsearch

```
from datasets import load_dataset
squad = load_dataset('crime_and_punish', split='train[:1000]')
```

When I run the line: 
`squad.add_elasticsearch_index("context", host="localhost", port="9200")`

I get the error: 
`TypeError: __init__() missing 1 required positional argument: 'scheme'`


## Expected results
No error message

## Actual results

```
TypeError                                 Traceback (most recent call last)

[<ipython-input-23-9205593edef3>](https://localhost:8080/#) in <module>()
      1 import elasticsearch
----> 2 squad.add_elasticsearch_index("text", host="localhost", port="9200")


6 frames

[/usr/local/lib/python3.7/dist-packages/elasticsearch/_sync/client/utils.py](https://localhost:8080/#) in host_mapping_to_node_config(host)
    209         options["path_prefix"] = options.pop("url_prefix")
    210 
--> 211     return NodeConfig(**options)  # type: ignore
    212 
    213 

TypeError: __init__() missing 1 required positional argument: 'scheme'
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.2.0
- Platform: Linux, Google Colab
- Python version: Google Colab (probably 3.7)
- PyArrow version: ?


Comments: Hi, @MoritzLaurer, thanks for reporting.

Normally this is due to a mismatch between the versions of your Elasticsearch client and server:
- your ES client is passing only keyword arguments to your ES server
- whereas your ES server expects a positional argument called 'scheme'

In order to fix this, you should align the major versions of both Elasticsearch client and server.

You can have more info:
- on this other issue page: https://github.com/huggingface/datasets/issues/3956#issuecomment-1072115173
- Elasticsearch client docs: https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/overview.html#_compatibility

Feel free to re-open this issue if the problem persists.

Duplicate of:
- #3956

1. Check elasticsearch version
```
import elasticsearch
print(elasticsearch.__version__)
```
Ex: 7.9.1
2. Uninstall current elasticsearch package
`pip uninstall elasticsearch`
3. Install elasticsearch 7.9.1 package
`pip install elasticsearch==7.9.1`
Missing argument in precision/recall
Author: Dref360
State: closed
Labels: name: enhancement

**Is your feature request related to a problem? Please describe.**
[`sklearn.metrics.precision_score`](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_score.html) accepts an argument `zero_division`, but it is not available in [precision Metric](https://github.com/huggingface/datasets/blob/master/metrics/precision/precision.py#L117)

Same issue is present for Recall.

**Describe the solution you'd like**
Support for **kwargs or adding a new field for `zero_division`.

**Describe alternatives you've considered**
I could filter the warnings myself, but that is not ideal.

**Additional context**
I can make the requested changes if this is approved.

Comments: Thanks for the suggestion, @Dref360.

We are adding that argument.
Can not correctly parse the classes with imagefolder
Author: YiSyuanChen
State: closed
Labels: name: bug

## Describe the bug
I try to load my own image dataset with imagefolder, but the parsing of classes is incorrect.

## Steps to reproduce the bug
I organized my dataset (ImageNet) in the following structure:
```
  - imagenet/
      - train/
          - n01440764/
              - ILSVRC2012_val_00000293.jpg
              - ...... 
          - n01695060/
          - ......
      - val/ 
          - n01440764/
          - n01695060/
          - ......
```

At first, I followed the instructions from the Huggingface [example](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification#using-your-own-data) to load my data as:
```
from datasets import load_dataset
data_files = {'train': 'imagenet/train', 'val': 'imagenet/val'}
ds = load_dataset("nateraw/image-folder", data_files=data_files, task="image-classification")
```
but it resulted following error (I mask my personal path as <PERSONAL_PATH>):
```
FileNotFoundError: Unable to find 'https://huggingface.co/datasets/nateraw/image-folder/resolve/main/imagenet/train' at <PERSONAL_PATH>/ImageNet/https:/huggingface.co/datasets/nateraw/image-folder/resolve/main
```

Next, I followed a recent issue #3960 to load data as:
```
from datasets import load_dataset
data_files = {'train': ['imagenet/train/**'], 'val': ['imagenet/val/**']}
ds = load_dataset("imagefolder", data_files=data_files, task="image-classification")
```
and the data can be loaded without error as: (I copy val folder to train folder for illustration)
```
>>> ds
DatasetDict({
    train: Dataset({
        features: ['image', 'labels'],
        num_rows: 50000
    })
    val: Dataset({
        features: ['image', 'labels'],
        num_rows: 50000
    })
})
```
However, the parsed classes is wrong (should be 1000 classes):
```
>>> ds["train"].features
{'image': Image(decode=True, id=None), 'labels': ClassLabel(num_classes=1, names=['val'], id=None)}
```

## Expected results
I expect that the "labels" in ds["train"].features should contain 1000 classes.

## Actual results
The "labels" in ds["train"].features contains only 1 wrong class.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.0.0
- Platform: Ubuntu 18.04
- Python version: Python 3.7.12
- PyArrow version: 7.0.0


Comments: I found that the problem arises because the image files in my folder are actually symbolic links (for my own reasons).  After modifications, the classes can now be correctly parsed. Therefore, I close this issue.

HI, I have a question. How much time did you load the ImageNet data files?
Cannot preview "hazal/Turkish-Biomedical-corpus-trM"
Author: hazalturkmen
State: closed

## Dataset viewer issue for '*hazal/Turkish-Biomedical-corpus-trM'

**Link:** *https://huggingface.co/datasets/hazal/Turkish-Biomedical-corpus-trM*

*I cannot see the dataset preview.*

```
Server Error

Status code:   400
Exception:     HTTPError
Message:       403 Client Error: Forbidden for url: https://huggingface.co/api/datasets/hazal/Turkish-Biomedical-corpus-trM?full=true
```


Am I the one who added this dataset ? Yes


Comments: Hi @hazalturkmen, thanks for reporting.

Note that your dataset repository does not contain any loading script; it only contains a data file named `tr_article_2`.

When there is no loading script but only data files, the `datasets` library tries to infer how to load the data by looking at the data file extensions. However, your data file does not have any extension.

Note that current supported data file extensions are: 'csv', 'tsv', 'json', 'jsonl', 'parquet', 'txt', 'blp', 'bmp', 'dib', 'bufr', 'cur', 'pcx', 'dcx', 'dds', 'ps', 'eps', 'fit', 'fits', 'fli', 'flc', 'ftc', 'ftu', 'gbr', 'gif', 'grib', 'h5', 'hdf', 'png', 'apng', 'jp2', 'j2k', 'jpc', 'jpf', 'jpx', 'j2c', 'icns', 'ico', 'im', 'iim', 'tif', 'tiff', 'jfif', 'jpe', 'jpg', 'jpeg', 'mpg', 'mpeg', 'msp', 'pcd', 'pxr', 'pbm', 'pgm', 'ppm', 'pnm', 'psd', 'bw', 'rgb', 'rgba', 'sgi', 'ras', 'tga', 'icb', 'vda', 'vst', 'webp', 'wmf', 'emf', 'xbm', 'xpm', 'zip'.

You have more info on our docs: [How to share a dataset](https://huggingface.co/docs/datasets/share).

thanks for reply :)