Minimartzz/github-issues huggingface dataset:
Fix multiprocessing in map_nested 
State: closed

As previously discussed:

Before, multiprocessing was not used in `map_nested` if `num_proc` was greater than or equal to `len(iterable)`.
- Multiprocessing was not used e.g. when passing `num_proc=20` but having 19 files to download
- As by default, `DownloadManager` sets `num_proc=16`, before multiprocessing was only used when `len(iterable)>16` by default

Now, if `num_proc` is greater than or equal to ``len(iterable)``, `num_proc` is set to ``len(iterable)`` and multiprocessing is used.
- We pass the variable `parallel_min_length=16`, so that multiprocessing is only used if at least 16 files to be downloaded
- ~As by default, `DownloadManager` sets `num_proc=16`, now multiprocessing is used when `len(iterable)>1` by default~

See discussion below.

~After having had to fix some tests (87602ac), I am wondering:~
- ~do we want to have multiprocessing by default?~
  - ~please note that `DownloadManager.download` sets `num_proc=16` by default~
- ~or would it be better to ask the user to set it explicitly if they want multiprocessing (and default to `num_proc=1`)?~

Fix #4636.

CC: @nateraw 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@lhoestq as a workaround to preserve previous behavior, the parameter `multiprocessing_min_length=16` is passed from `download` to `map_nested`, so that multiprocessing is only used if at least 16 files to be downloaded.

Note that there is a small breaking change (I think previously it was unintended behavior, so that I have fixed it):
- Before (with default `num_proc=16`) if there were 16 files to be downloaded, multiprocessing was not used
- Now (with default `num_proc=16`) if there are 16 files to be downloaded, multiprocessing is used

Thanks for the workaround !
Deprecate metrics
State: closed

Deprecate metrics:
- deprecate public functions: `load_metric`, `list_metrics` and `inspect_metric`: docstring and warning
- test deprecation warnings are issues
- deprecate metrics in all docs
- remove mentions to metrics in docs and README
- deprecate internal functions/classes

Maybe we should also stop testing metrics?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I mark this as Draft because the deprecated version number needs being updated after the latest release.

Perhaps now is the time to also update the `inspect_metric` from `evaluate` with the changes introduced in https://github.com/huggingface/datasets/pull/4433 (cc @lvwerra) 

What do you think of including what changes users have to do to switch to `evaluate` in the warning message ?
(basically replace `datasets.load_metric` by `evaluate.load`)

I think it can help users migrate to `evaluate` and silence the warnings
Use CI unit/integration tests
State: closed

This PR:
- Implements separate unit/integration tests
- A fail in integration tests does not cancel the rest of the jobs
  - We should implement more robust integration tests: work in progress in a subsequent PR
- For the moment, test involving network requests are marked as integration: to be evolved

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I think this PR can be merged. Willing to see it in action.

CC: @lhoestq
Download error on scene_parse_150
State: closed
Labels: description: Something isn't working
name: bug

```
from datasets import load_dataset
dataset = load_dataset("scene_parse_150", "scene_parsing")

FileNotFoundError: Couldn't find file at http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip
```


Comments:
Hi! The server with the data seems to be down. I've reported this issue (https://github.com/CSAILVision/sceneparsing/issues/34) in the dataset repo. 

The URL seems to work now, and therefore the script as well.
Dataset Viewer issue for deepklarity/huggingface-spaces-dataset
State: closed
Labels: description: Related to the dataset viewer on huggingface.co
name: dataset-viewer

### Link

https://huggingface.co/datasets/deepklarity/huggingface-spaces-dataset/viewer/deepklarity--huggingface-spaces-dataset/train

### Description

Hi Team, 
I'm getting the following error on a uploaded dataset. I'm getting the same status for a couple of hours now. The dataset size is `<1MB` and the format is csv, so I'm not sure if it's supposed to take this much time or not. 
```
Status code:   400
Exception:     Status400Error
Message:       The split is being processed. Retry later.
```

Is there any explicit step to be taken to get the viewer to work? 

### Owner

Yes

Comments:
Thanks for reporting. You're right, workers were under-provisioned due to a manual error, and the job queue was full. It's fixed now.
Package rouge-score cannot be imported
State: closed
Labels: description: Something isn't working
name: bug

## Describe the bug
After the today release of `rouge_score-0.0.7` it seems no longer importable. Our CI fails: https://github.com/huggingface/datasets/runs/7463218591?check_suite_focus=true
```
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_builder_class_bigbench
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_builder_configs_bigbench
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_bigbench
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_rouge
```
with errors:
```
>   from rouge_score import rouge_scorer
E   ModuleNotFoundError: No module named 'rouge_score'
```
```
E           ImportError: To be able to use rouge, you need to install the following dependency: rouge_score.
E           Please install it using 'pip install rouge_score' for instance'
```


Comments:
We have added a comment on an existing issue opened in their repo: https://github.com/google-research/google-research/issues/1212#issuecomment-1192267130
- https://github.com/google-research/google-research/issues/1212
rouge metric
State: closed
Labels: description: Something isn't working
name: bug

## Describe the bug
A clear and concise description of what the bug is.
Loading  Rouge metric gives error after latest rouge-score==0.0.7 release.
Downgrading rougemetric==0.0.4 works fine.

## Steps to reproduce the bug
```python
# Sample code to reproduce the bug
```

## Expected results
A clear and concise description of the expected results.
from rouge_score import rouge_scorer, scoring 
should run

## Actual results
Specify the actual results or traceback.
File "/root/.cache/huggingface/modules/datasets_modules/metrics/rouge/0ffdb60f436bdb8884d5e4d608d53dbe108e82dac4f494a66f80ef3f647c104f/rouge.py", line 21, in <module>
    from rouge_score import rouge_scorer, scoring
ImportError: cannot import name 'rouge_scorer' from 'rouge_score' (unknown location)

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform: Linux
- Python version:3.9
- PyArrow version:


Comments:
Fixed by:
- #4735
Document better that loading a dataset passing its name does not use the local script
State: closed
Labels: description: Improvements or additions to documentation
name: documentation

As reported by @TrentBrick here https://github.com/huggingface/datasets/issues/4725#issuecomment-1191858596, it could be more clear that loading a dataset by passing its name does not use the (modified) local script of it.

What he did:
- he installed `datasets` from source
- he modified locally `datasets/the_pile/the_pile.py` loading script
- he tried to load it but using `load_dataset("the_pile")` instead of `load_dataset("datasets/the_pile")`
  - as explained here https://github.com/huggingface/datasets/issues/4725#issuecomment-1191040245:
    - the former does not use the local script, but instead it downloads a copy of `the_pile.py` from our GitHub, caches it locally (inside `~/.cache/huggingface/modules`) and uses that.

He suggests adding a more clear explanation about this. He suggests adding it maybe in [Installation > source](https://huggingface.co/docs/datasets/installation))

CC: @stevhliu 

Comments:
Thanks for the feedback!

I think since this issue is closely related to loading, I can add a clearer explanation under [Load > local loading script](https://huggingface.co/docs/datasets/main/en/loading#local-loading-script).

That makes sense but I think having a line about it under https://huggingface.co/docs/datasets/installation#source the "source" header here would be useful. My mental model of `pip install -e .` does not include the fact that the source files aren't actually being used. 

Thanks for sharing your perspective. I think the `load_dataset` function is the only one that pulls from GitHub, and since this use-case is very specific, I don't think we need to include such a broad clarification in the Installation section.

Feel free to check out the linked PR and let me know if it needs any additional explanation 😊
Loading imagenet-1k validation split takes much more RAM than expected
State: closed
Labels: description: Something isn't working
name: bug

## Describe the bug

Loading into memory the validation split of imagenet-1k takes much more RAM than expected. Assuming ImageNet-1k is 150 GB, split is 50000 validation images and 1,281,167 train images, I would expect only about 6 GB loaded in RAM.


## Steps to reproduce the bug
```python
from datasets import  load_dataset

dataset = load_dataset("imagenet-1k", split="validation")

print(dataset)

"""prints
Dataset({
    features: ['image', 'label'],
    num_rows: 50000
})
"""

pipe_inputs = dataset["image"]
# and wait :-)
```

## Expected results
Use only < 10 GB RAM when loading the images.

## Actual results
![image](https://user-images.githubusercontent.com/9808326/180249183-62f75ca4-d127-402a-9330-f12825a22b0a.png)

```
Using custom data configuration default
Reusing dataset imagenet-1k (/home/fxmarty/.cache/huggingface/datasets/imagenet-1k/default/1.0.0/a1e9bfc56c3a7350165007d1176b15e9128fcaf9ab972147840529aed3ae52bc)
Killed
```

## Environment info
- `datasets` version: 2.3.3.dev0
- Platform: Linux-5.15.0-41-generic-x86_64-with-glibc2.35
- Python version: 3.9.12
- PyArrow version: 7.0.0
- Pandas version: 1.3.5
- datasets commit: 4e4222f1b6362c2788aec0dd2cd8cede6dd17b80


Comments:
My bad, `482 * 418 * 50000 * 3 / 1000000 = 30221 MB` ( https://stackoverflow.com/a/42979315 ).

Meanwhile `256 * 256 * 50000 * 3 / 1000000 = 9830 MB`. We are loading the non-cropped images and that is why we take so much RAM.
Refactor Hub tests
State: closed

This PR refactors `test_upstream_hub` by removing unittests and using the following pytest Hub fixtures:
- `ci_hub_config`
- `set_ci_hub_access_token`: to replace setUp/tearDown
- `temporary_repo` context manager: to replace `try... finally`
- `cleanup_repo`: to delete repo accidentally created if one of the tests fails

This is a preliminary work done to manage unit/integration tests separately.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
load_dataset gives "403" error when using Financial Phrasebank
State: closed

I tried both codes below to download the financial phrasebank dataset (https://huggingface.co/datasets/financial_phrasebank) with the sentences_allagree subset. However, the code gives a 403 error when executed from multiple machines locally or on the cloud.

```
from datasets import load_dataset, DownloadMode
load_dataset(path='financial_phrasebank',name='sentences_allagree',download_mode=DownloadMode.FORCE_REDOWNLOAD)
```

```
from datasets import load_dataset, DownloadMode
load_dataset(path='financial_phrasebank',name='sentences_allagree')
```

**Error**
ConnectionError: Couldn't reach https://www.researchgate.net/profile/Pekka_Malo/publication/251231364_FinancialPhraseBank-v10/data/0c96051eee4fb1d56e000000/FinancialPhraseBank-v10.zip (error 403)


Comments:
Hi @rohitvincent, thanks for reporting.

Unfortunately I'm not able to reproduce your issue:
```python
In [2]: from datasets import load_dataset, DownloadMode
   ...: load_dataset(path='financial_phrasebank',name='sentences_allagree', download_mode="force_redownload")
Downloading builder script: 6.04kB [00:00, 2.87MB/s]                                                                                                                                                       
Downloading metadata: 13.7kB [00:00, 7.24MB/s]                                                                                                                                                             
Downloading and preparing dataset financial_phrasebank/sentences_allagree (download: 665.91 KiB, generated: 296.26 KiB, post-processed: Unknown size, total: 962.17 KiB) to .../.cache/huggingface/datasets/financial_phrasebank/sentences_allagree/1.0.0/550bde12e6c30e2674da973a55f57edde5181d53f5a5a34c1531c53f93b7e141...
Downloading data: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 682k/682k [00:00<00:00, 7.66MB/s]
Dataset financial_phrasebank downloaded and prepared to .../.cache/huggingface/datasets/financial_phrasebank/sentences_allagree/1.0.0/550bde12e6c30e2674da973a55f57edde5181d53f5a5a34c1531c53f93b7e141. Subsequent calls will reuse this data.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 918.80it/s]
Out[2]: 
DatasetDict({
    train: Dataset({
        features: ['sentence', 'label'],
        num_rows: 2264
    })
})
```

Are you able to access the link? https://www.researchgate.net/profile/Pekka-Malo/publication/251231364_FinancialPhraseBank-v10/data/0c96051eee4fb1d56e000000/FinancialPhraseBank-v10.zip

Yes was able to download from the link manually. But still, get the same error when I use load_dataset.

Fixed once data files are hosted on the Hub:
- #4598
Dataset Viewer issue for TheNoob3131/mosquito-data
State: closed
Labels: description: Related to the dataset viewer on huggingface.co
name: dataset-viewer

### Link

https://huggingface.co/datasets/TheNoob3131/mosquito-data/viewer/TheNoob3131--mosquito-data/test

### Description

Dataset preview not showing with large files. Says 'split cache is empty' even though there are train and test splits.

### Owner

_No response_

Comments:
The preview is working OK:

![Screenshot from 2022-07-21 09-46-09](https://user-images.githubusercontent.com/8515462/180158929-bd8faad4-6392-4fc1-8d9c-df38aa9f8438.png)
Fix broken link to the Hub
State: closed

The Markdown link fails to render if it is in the same line as the `<span>`. This PR implements @mishig25's fix by using `<a href=" ">` instead.

![Screen Shot 2022-07-20 at 3 53 05 PM](https://user-images.githubusercontent.com/59462357/180096412-7fbb33be-abb0-4e54-a52d-201b3b58e0f9.png)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
the_pile datasets URL broken. 
State: closed
Labels: description: Something isn't working
name: bug

https://github.com/huggingface/datasets/pull/3627 changed the Eleuther AI Pile dataset URL from https://the-eye.eu/ to https://mystic.the-eye.eu/ but the latter is now broken and the former works again. 

Note that when I git clone the repo and use `pip install -e .` and then edit the URL back the codebase doesn't seem to use this edit so the mystic URL is also cached somewhere else that I can't find? 

Comments:
Thanks for reporting, @TrentBrick. We are addressing the change with their data host server.

On the meantime, if you would like to work with your fixed local copy of the_pile script, you should use:
```python
load_dataset("path/to/your/local/the_pile/the_pile.py",...
```
instead of just `load_dataset("the_pile",...`.

The latter downloads a copy of `the_pile.py` from our GitHub, caches it locally (inside `~/.cache/huggingface/modules`) and uses that.

@TrentBrick, I have checked the URLs and both hosts work, the original (https://the-eye.eu/) and the mirror (https://mystic.the-eye.eu/). See e.g.:
- https://mystic.the-eye.eu/public/AI/pile/
- https://mystic.the-eye.eu/public/AI/pile_preliminary_components/

Please, let me know if you still find any issue loading this dataset by using current server URLs.

Great this is working now. Re the download from GitHub... I'm sure thought went into doing this but could it be made more clear maybe here? https://huggingface.co/docs/datasets/installation for example under installing from source? I spent over an hour questioning my sanity as I kept trying to edit this file, uninstall and reinstall the repo, git reset to previous versions of the file etc.

Thanks for the quick reply and help too


Thanks @TrentBrick for the suggestion about improving our docs: we should definitely do this if you find they are not clear enough.

Currently, our docs explain how to load a dataset from a local loading script here: [Load > Local loading script](https://huggingface.co/docs/datasets/loading#local-loading-script)

I've opened an issue here:
- #4732

Feel free to comment on it any additional explanation/suggestion/requirement related to this problem.
Refactor conftest fixtures
State: closed

Previously, fixture modules `hub_fixtures` and `s3_fixtures`:
- were both at the root test directory
- were imported using `import *`
  - as a side effect, the modules `os` and `pytest` were imported from `s3_fixtures` into `conftest`

This PR:
- puts both fixture modules in a dedicated directory `fixtures`
- renames both to: `fixtures.hub` and `fixtures.s3`
- imports them into `conftest` as plugins, using the `pytest_plugins`: this avoids the `import *`
- additionally creates a new fixture module `fixtures.files` with all file-related fixtures

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Docs: Fix same-page haslinks
State: closed

`href="/docs/datasets/quickstart#audio"` implicitly goes to `href="/docs/datasets/{$LATEST_STABLE_VERSION}/quickstart#audio"`. Therefore, https://huggingface.co/docs/datasets/quickstart#audio #audio hashlink does not work since the new docs were not added to v2.3.2 (LATEST_STABLE_VERSION)

to preserve the version, it should be just `href="#audio"`, which will implicilty go to curren_page + #audio element

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Issue loading TheNoob3131/mosquito-data dataset
State: closed

![image](https://user-images.githubusercontent.com/53668030/179815591-d75fa7d3-3122-485f-a852-b06a68909066.png)

So my dataset is public in the Huggingface Hub, but when I try to load it using the load_dataset command, it shows that it is downloading the files, but throws a ValueError. When I went to my directory to see if the files were downloaded, the folder was blank.

Here is the error below:
ValueError                                Traceback (most recent call last)
Input In [8], in <cell line: 3>()
      1 from datasets import load_dataset
----> 3 dataset = load_dataset("TheNoob3131/mosquito-data", split="train")

File ~\Anaconda3\lib\site-packages\datasets\load.py:1679, in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, **config_kwargs)
   1676 try_from_hf_gcs = path not in _PACKAGED_DATASETS_MODULES
   1678 # Download and prepare data
-> 1679 builder_instance.download_and_prepare(
   1680     download_config=download_config,
   1681     download_mode=download_mode,
   1682     ignore_verifications=ignore_verifications,
   1683     try_from_hf_gcs=try_from_hf_gcs,
   1684     use_auth_token=use_auth_token,
   1685 )
   1687 # Build dataset for splits
   1688 keep_in_memory = (
   1689     keep_in_memory if keep_in_memory is not None else is_small_dataset(builder_instance.info.dataset_size)
   1690 )

Is the dataset in the wrong format or is there some security permission that I should enable?

Comments:
I am also getting a ValueError: 'Couldn't cast' at the bottom. Is this because of some delimiter issue? My dataset is on the Huggingface Hub. If you could look at it, that would be greatly appreciated.

Hi @thenerd31, thanks for reporting.

Please note that your issue is not caused by the Hugging Face Datasets library, but it has to do with the specific implementation of your dataset on the Hub.

Therefore, I'm transferring this discussion to your own dataset Community tab: https://huggingface.co/datasets/TheNoob3131/mosquito-data/discussions/1
Dataset Viewer issue for LawalAfeez/englishreview-ds-mini
State: closed
Labels: description: Related to the dataset viewer on huggingface.co
name: dataset-viewer

### Link

_No response_

### Description

Unable to view the split data

### Owner

_No response_

Comments:
It's currently working, as far as I understand

https://huggingface.co/datasets/LawalAfeez/englishreview-ds-mini/viewer/LawalAfeez--englishreview-ds-mini/train

<img width="1556" alt="Capture d’écran 2022-07-19 à 09 24 01" src="https://user-images.githubusercontent.com/1676121/179761130-2d7980b9-c0f6-4093-8b1d-f0a3872fef3f.png">

---

What was your issue?
Support "tags" yaml tag
State: closed

Added the "tags" YAML tag, so that users can specify data domain/topics keywords for dataset search

Comments:
_The documentation is not available anymore as the PR was closed or merged._

IMO `DatasetMetadata` shouldn't crash with attributes that it doesn't know, btw

Yea this PR is mostly to have a validation that this field contains a list of strings.

Regarding unknown fields, the tagging app currently returns an error if a field is unknown using the `DatasetMetadata`. We can change that though
Fix POS tags
State: closed

We're now using `part-of-speech` and not `part-of-speech-tagging`, see discussion here: https://github.com/huggingface/datasets/commit/114c09aff2fa1519597b46fbcd5a8e0c0d3ae020#r78794777

Comments:
_The documentation is not available anymore as the PR was closed or merged._

CI failures are about missing content in the dataset cards or bad tags, and this is unrelated to this PR. Merging :)
Fix named split sorting and remove unnecessary casting
State: closed

This PR:
- makes `NamedSplit` sortable: so that `sorted()` can be called on them
- removes unnecessary `sorted()` on `dict.keys()`: `dict_keys` view is already like a `set`
- removes unnecessary casting of `NamedSplit` to `str`

Comments:
_The documentation is not available anymore as the PR was closed or merged._

hahaha what a timing, I added my comment right after you merged x)

you can ignore my (nit), it's fine

Sorry, just too sync... :sweat_smile:
Document how to create a dataset loading script for audio/vision
State: closed
Labels: description: Improvements or additions to documentation
name: documentation

Currently, in our docs for Audio/Vision/Text, we explain how to:
- Load data
- Process data

However we only explain how to *Create a dataset loading script* for text data.

I think it would be useful that we add the same for Audio/Vision as these have some specificities different from Text.

See, for example:
- #4697
  - and comment there: https://github.com/huggingface/datasets/issues/4697#issuecomment-1191502492

CC: @stevhliu 


Comments:
I'm closing this issue as both the Audio and Image sections now have a "Create dataset" page that contains the info about writing the loading script version of a dataset.
Add object detection processing tutorial
State: closed

The following adds a quick guide on how to process object detection datasets with `albumentations`. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Great idea! Now that we have more than one task, it makes sense to separate image classification and object detection so it'll be easier for users to follow.

@lhoestq do we want to do that in this PR, or should we merge it and let @stevhliu reorganize separately?