josteece/github-issues huggingface dataset:
Dummy data nowhere to be found
Opened by BramVanroy (closed)

## Describe the bug
To finalize my dataset, I wanted to create dummy data as per the guide and I ran 

```shell
 datasets-cli dummy_data datasets/hebban-reviews --auto_generate
```

where hebban-reviews is [this repo](https://huggingface.co/datasets/BramVanroy/hebban-reviews). And even though the scripts runs and shows a message at the end that it succeeded, I cannot find the dummy data anywhere. Where is it?

## Expected results

To see the dummy data in the datasets' folder or in the folder where I ran the command.

## Actual results

I see the following message but I cannot find the dummy data anywhere.

```
Dummy data generation done and dummy data test succeeded for config 'filtered''.
Automatic dummy data generation succeeded for all configs of '.\datasets\hebban-reviews\'
```

## Environment info
- `datasets` version: 2.4.1.dev0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.8
- PyArrow version: 8.0.0
- Pandas version: 1.4.3



Comments:
Hi @BramVanroy, thanks for reporting.

First of all, please note that you do not need the dummy data: this was the case when we were adding datasets to the `datasets` library (on this GitHub repo), so that we could test the correct loading of all datasets with our CI. However, this is no longer the case for datasets on the Hub.
- We should definitely update our docs.

Second, the dummy data is generated locally:
- in your case, the dummy data will be generated inside the directory:  `./datasets/hebban-reviews/dummy`
- please note the preceding `./datasets` directory: the reason for this is that the command to generate the dummy data was specifically created for our `datasets` library, and therefore assumes our directory structure: commands are run from the root directory of our GitHub repo, and datasets scripts are under `./datasets` 


 

I have opened an Issue to update the instructions on dummy data generation:
- #4744

Dummy data generation is deprecated now, so I think we can close this issue.
Fix multiprocessing in map_nested 
Opened by albertvillanova (closed)

As previously discussed:

Before, multiprocessing was not used in `map_nested` if `num_proc` was greater than or equal to `len(iterable)`.
- Multiprocessing was not used e.g. when passing `num_proc=20` but having 19 files to download
- As by default, `DownloadManager` sets `num_proc=16`, before multiprocessing was only used when `len(iterable)>16` by default

Now, if `num_proc` is greater than or equal to ``len(iterable)``, `num_proc` is set to ``len(iterable)`` and multiprocessing is used.
- We pass the variable `parallel_min_length=16`, so that multiprocessing is only used if at least 16 files to be downloaded
- ~As by default, `DownloadManager` sets `num_proc=16`, now multiprocessing is used when `len(iterable)>1` by default~

See discussion below.

~After having had to fix some tests (87602ac), I am wondering:~
- ~do we want to have multiprocessing by default?~
  - ~please note that `DownloadManager.download` sets `num_proc=16` by default~
- ~or would it be better to ask the user to set it explicitly if they want multiprocessing (and default to `num_proc=1`)?~

Fix #4636.

CC: @nateraw 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@lhoestq as a workaround to preserve previous behavior, the parameter `multiprocessing_min_length=16` is passed from `download` to `map_nested`, so that multiprocessing is only used if at least 16 files to be downloaded.

Note that there is a small breaking change (I think previously it was unintended behavior, so that I have fixed it):
- Before (with default `num_proc=16`) if there were 16 files to be downloaded, multiprocessing was not used
- Now (with default `num_proc=16`) if there are 16 files to be downloaded, multiprocessing is used

Thanks for the workaround !
Deprecate metrics
Opened by albertvillanova (closed)

Deprecate metrics:
- deprecate public functions: `load_metric`, `list_metrics` and `inspect_metric`: docstring and warning
- test deprecation warnings are issues
- deprecate metrics in all docs
- remove mentions to metrics in docs and README
- deprecate internal functions/classes

Maybe we should also stop testing metrics?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I mark this as Draft because the deprecated version number needs being updated after the latest release.

Perhaps now is the time to also update the `inspect_metric` from `evaluate` with the changes introduced in https://github.com/huggingface/datasets/pull/4433 (cc @lvwerra) 

What do you think of including what changes users have to do to switch to `evaluate` in the warning message ?
(basically replace `datasets.load_metric` by `evaluate.load`)

I think it can help users migrate to `evaluate` and silence the warnings
Use CI unit/integration tests
Opened by albertvillanova (closed)

This PR:
- Implements separate unit/integration tests
- A fail in integration tests does not cancel the rest of the jobs
  - We should implement more robust integration tests: work in progress in a subsequent PR
- For the moment, test involving network requests are marked as integration: to be evolved

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I think this PR can be merged. Willing to see it in action.

CC: @lhoestq
Download error on scene_parse_150
Opened by juliensimon (closed)

```
from datasets import load_dataset
dataset = load_dataset("scene_parse_150", "scene_parsing")

FileNotFoundError: Couldn't find file at http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip
```


Comments:
Hi! The server with the data seems to be down. I've reported this issue (https://github.com/CSAILVision/sceneparsing/issues/34) in the dataset repo. 

The URL seems to work now, and therefore the script as well.
Dataset Viewer issue for deepklarity/huggingface-spaces-dataset
Opened by dk-crazydiv (closed)

### Link

https://huggingface.co/datasets/deepklarity/huggingface-spaces-dataset/viewer/deepklarity--huggingface-spaces-dataset/train

### Description

Hi Team, 
I'm getting the following error on a uploaded dataset. I'm getting the same status for a couple of hours now. The dataset size is `<1MB` and the format is csv, so I'm not sure if it's supposed to take this much time or not. 
```
Status code:   400
Exception:     Status400Error
Message:       The split is being processed. Retry later.
```

Is there any explicit step to be taken to get the viewer to work? 

### Owner

Yes

Comments:
Thanks for reporting. You're right, workers were under-provisioned due to a manual error, and the job queue was full. It's fixed now.
Package rouge-score cannot be imported
Opened by albertvillanova (closed)

## Describe the bug
After the today release of `rouge_score-0.0.7` it seems no longer importable. Our CI fails: https://github.com/huggingface/datasets/runs/7463218591?check_suite_focus=true
```
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_builder_class_bigbench
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_builder_configs_bigbench
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_bigbench
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_rouge
```
with errors:
```
>   from rouge_score import rouge_scorer
E   ModuleNotFoundError: No module named 'rouge_score'
```
```
E           ImportError: To be able to use rouge, you need to install the following dependency: rouge_score.
E           Please install it using 'pip install rouge_score' for instance'
```


Comments:
We have added a comment on an existing issue opened in their repo: https://github.com/google-research/google-research/issues/1212#issuecomment-1192267130
- https://github.com/google-research/google-research/issues/1212
rouge metric
Opened by asking28 (closed)

## Describe the bug
A clear and concise description of what the bug is.
Loading  Rouge metric gives error after latest rouge-score==0.0.7 release.
Downgrading rougemetric==0.0.4 works fine.

## Steps to reproduce the bug
```python
# Sample code to reproduce the bug
```

## Expected results
A clear and concise description of the expected results.
from rouge_score import rouge_scorer, scoring 
should run

## Actual results
Specify the actual results or traceback.
File "/root/.cache/huggingface/modules/datasets_modules/metrics/rouge/0ffdb60f436bdb8884d5e4d608d53dbe108e82dac4f494a66f80ef3f647c104f/rouge.py", line 21, in <module>
    from rouge_score import rouge_scorer, scoring
ImportError: cannot import name 'rouge_scorer' from 'rouge_score' (unknown location)

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform: Linux
- Python version:3.9
- PyArrow version:


Comments:
Fixed by:
- #4735
Document better that loading a dataset passing its name does not use the local script
Opened by albertvillanova (closed)

As reported by @TrentBrick here https://github.com/huggingface/datasets/issues/4725#issuecomment-1191858596, it could be more clear that loading a dataset by passing its name does not use the (modified) local script of it.

What he did:
- he installed `datasets` from source
- he modified locally `datasets/the_pile/the_pile.py` loading script
- he tried to load it but using `load_dataset("the_pile")` instead of `load_dataset("datasets/the_pile")`
  - as explained here https://github.com/huggingface/datasets/issues/4725#issuecomment-1191040245:
    - the former does not use the local script, but instead it downloads a copy of `the_pile.py` from our GitHub, caches it locally (inside `~/.cache/huggingface/modules`) and uses that.

He suggests adding a more clear explanation about this. He suggests adding it maybe in [Installation > source](https://huggingface.co/docs/datasets/installation))

CC: @stevhliu 

Comments:
Thanks for the feedback!

I think since this issue is closely related to loading, I can add a clearer explanation under [Load > local loading script](https://huggingface.co/docs/datasets/main/en/loading#local-loading-script).

That makes sense but I think having a line about it under https://huggingface.co/docs/datasets/installation#source the "source" header here would be useful. My mental model of `pip install -e .` does not include the fact that the source files aren't actually being used. 

Thanks for sharing your perspective. I think the `load_dataset` function is the only one that pulls from GitHub, and since this use-case is very specific, I don't think we need to include such a broad clarification in the Installation section.

Feel free to check out the linked PR and let me know if it needs any additional explanation 😊
Loading imagenet-1k validation split takes much more RAM than expected
Opened by fxmarty (closed)

## Describe the bug

Loading into memory the validation split of imagenet-1k takes much more RAM than expected. Assuming ImageNet-1k is 150 GB, split is 50000 validation images and 1,281,167 train images, I would expect only about 6 GB loaded in RAM.


## Steps to reproduce the bug
```python
from datasets import  load_dataset

dataset = load_dataset("imagenet-1k", split="validation")

print(dataset)

"""prints
Dataset({
    features: ['image', 'label'],
    num_rows: 50000
})
"""

pipe_inputs = dataset["image"]
# and wait :-)
```

## Expected results
Use only < 10 GB RAM when loading the images.

## Actual results
![image](https://user-images.githubusercontent.com/9808326/180249183-62f75ca4-d127-402a-9330-f12825a22b0a.png)

```
Using custom data configuration default
Reusing dataset imagenet-1k (/home/fxmarty/.cache/huggingface/datasets/imagenet-1k/default/1.0.0/a1e9bfc56c3a7350165007d1176b15e9128fcaf9ab972147840529aed3ae52bc)
Killed
```

## Environment info
- `datasets` version: 2.3.3.dev0
- Platform: Linux-5.15.0-41-generic-x86_64-with-glibc2.35
- Python version: 3.9.12
- PyArrow version: 7.0.0
- Pandas version: 1.3.5
- datasets commit: 4e4222f1b6362c2788aec0dd2cd8cede6dd17b80


Comments:
My bad, `482 * 418 * 50000 * 3 / 1000000 = 30221 MB` ( https://stackoverflow.com/a/42979315 ).

Meanwhile `256 * 256 * 50000 * 3 / 1000000 = 9830 MB`. We are loading the non-cropped images and that is why we take so much RAM.
Refactor Hub tests
Opened by albertvillanova (closed)

This PR refactors `test_upstream_hub` by removing unittests and using the following pytest Hub fixtures:
- `ci_hub_config`
- `set_ci_hub_access_token`: to replace setUp/tearDown
- `temporary_repo` context manager: to replace `try... finally`
- `cleanup_repo`: to delete repo accidentally created if one of the tests fails

This is a preliminary work done to manage unit/integration tests separately.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
load_dataset gives "403" error when using Financial Phrasebank
Opened by rohitvincent (closed)

I tried both codes below to download the financial phrasebank dataset (https://huggingface.co/datasets/financial_phrasebank) with the sentences_allagree subset. However, the code gives a 403 error when executed from multiple machines locally or on the cloud.

```
from datasets import load_dataset, DownloadMode
load_dataset(path='financial_phrasebank',name='sentences_allagree',download_mode=DownloadMode.FORCE_REDOWNLOAD)
```

```
from datasets import load_dataset, DownloadMode
load_dataset(path='financial_phrasebank',name='sentences_allagree')
```

**Error**
ConnectionError: Couldn't reach https://www.researchgate.net/profile/Pekka_Malo/publication/251231364_FinancialPhraseBank-v10/data/0c96051eee4fb1d56e000000/FinancialPhraseBank-v10.zip (error 403)


Comments:
Hi @rohitvincent, thanks for reporting.

Unfortunately I'm not able to reproduce your issue:
```python
In [2]: from datasets import load_dataset, DownloadMode
   ...: load_dataset(path='financial_phrasebank',name='sentences_allagree', download_mode="force_redownload")
Downloading builder script: 6.04kB [00:00, 2.87MB/s]                                                                                                                                                       
Downloading metadata: 13.7kB [00:00, 7.24MB/s]                                                                                                                                                             
Downloading and preparing dataset financial_phrasebank/sentences_allagree (download: 665.91 KiB, generated: 296.26 KiB, post-processed: Unknown size, total: 962.17 KiB) to .../.cache/huggingface/datasets/financial_phrasebank/sentences_allagree/1.0.0/550bde12e6c30e2674da973a55f57edde5181d53f5a5a34c1531c53f93b7e141...
Downloading data: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 682k/682k [00:00<00:00, 7.66MB/s]
Dataset financial_phrasebank downloaded and prepared to .../.cache/huggingface/datasets/financial_phrasebank/sentences_allagree/1.0.0/550bde12e6c30e2674da973a55f57edde5181d53f5a5a34c1531c53f93b7e141. Subsequent calls will reuse this data.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 918.80it/s]
Out[2]: 
DatasetDict({
    train: Dataset({
        features: ['sentence', 'label'],
        num_rows: 2264
    })
})
```

Are you able to access the link? https://www.researchgate.net/profile/Pekka-Malo/publication/251231364_FinancialPhraseBank-v10/data/0c96051eee4fb1d56e000000/FinancialPhraseBank-v10.zip

Yes was able to download from the link manually. But still, get the same error when I use load_dataset.

Fixed once data files are hosted on the Hub:
- #4598
Dataset Viewer issue for TheNoob3131/mosquito-data
Opened by thenerd31 (closed)

### Link

https://huggingface.co/datasets/TheNoob3131/mosquito-data/viewer/TheNoob3131--mosquito-data/test

### Description

Dataset preview not showing with large files. Says 'split cache is empty' even though there are train and test splits.

### Owner

_No response_

Comments:
The preview is working OK:

![Screenshot from 2022-07-21 09-46-09](https://user-images.githubusercontent.com/8515462/180158929-bd8faad4-6392-4fc1-8d9c-df38aa9f8438.png)
Fix broken link to the Hub
Opened by stevhliu (closed)

The Markdown link fails to render if it is in the same line as the `<span>`. This PR implements @mishig25's fix by using `<a href=" ">` instead.

![Screen Shot 2022-07-20 at 3 53 05 PM](https://user-images.githubusercontent.com/59462357/180096412-7fbb33be-abb0-4e54-a52d-201b3b58e0f9.png)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
the_pile datasets URL broken. 
Opened by TrentBrick (closed)

https://github.com/huggingface/datasets/pull/3627 changed the Eleuther AI Pile dataset URL from https://the-eye.eu/ to https://mystic.the-eye.eu/ but the latter is now broken and the former works again. 

Note that when I git clone the repo and use `pip install -e .` and then edit the URL back the codebase doesn't seem to use this edit so the mystic URL is also cached somewhere else that I can't find? 

Comments:
Thanks for reporting, @TrentBrick. We are addressing the change with their data host server.

On the meantime, if you would like to work with your fixed local copy of the_pile script, you should use:
```python
load_dataset("path/to/your/local/the_pile/the_pile.py",...
```
instead of just `load_dataset("the_pile",...`.

The latter downloads a copy of `the_pile.py` from our GitHub, caches it locally (inside `~/.cache/huggingface/modules`) and uses that.

@TrentBrick, I have checked the URLs and both hosts work, the original (https://the-eye.eu/) and the mirror (https://mystic.the-eye.eu/). See e.g.:
- https://mystic.the-eye.eu/public/AI/pile/
- https://mystic.the-eye.eu/public/AI/pile_preliminary_components/

Please, let me know if you still find any issue loading this dataset by using current server URLs.

Great this is working now. Re the download from GitHub... I'm sure thought went into doing this but could it be made more clear maybe here? https://huggingface.co/docs/datasets/installation for example under installing from source? I spent over an hour questioning my sanity as I kept trying to edit this file, uninstall and reinstall the repo, git reset to previous versions of the file etc.

Thanks for the quick reply and help too


Thanks @TrentBrick for the suggestion about improving our docs: we should definitely do this if you find they are not clear enough.

Currently, our docs explain how to load a dataset from a local loading script here: [Load > Local loading script](https://huggingface.co/docs/datasets/loading#local-loading-script)

I've opened an issue here:
- #4732

Feel free to comment on it any additional explanation/suggestion/requirement related to this problem.
Refactor conftest fixtures
Opened by albertvillanova (closed)

Previously, fixture modules `hub_fixtures` and `s3_fixtures`:
- were both at the root test directory
- were imported using `import *`
  - as a side effect, the modules `os` and `pytest` were imported from `s3_fixtures` into `conftest`

This PR:
- puts both fixture modules in a dedicated directory `fixtures`
- renames both to: `fixtures.hub` and `fixtures.s3`
- imports them into `conftest` as plugins, using the `pytest_plugins`: this avoids the `import *`
- additionally creates a new fixture module `fixtures.files` with all file-related fixtures

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Docs: Fix same-page haslinks
Opened by mishig25 (closed)

`href="/docs/datasets/quickstart#audio"` implicitly goes to `href="/docs/datasets/{$LATEST_STABLE_VERSION}/quickstart#audio"`. Therefore, https://huggingface.co/docs/datasets/quickstart#audio #audio hashlink does not work since the new docs were not added to v2.3.2 (LATEST_STABLE_VERSION)

to preserve the version, it should be just `href="#audio"`, which will implicilty go to curren_page + #audio element

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Issue loading TheNoob3131/mosquito-data dataset
Opened by thenerd31 (closed)

![image](https://user-images.githubusercontent.com/53668030/179815591-d75fa7d3-3122-485f-a852-b06a68909066.png)

So my dataset is public in the Huggingface Hub, but when I try to load it using the load_dataset command, it shows that it is downloading the files, but throws a ValueError. When I went to my directory to see if the files were downloaded, the folder was blank.

Here is the error below:
ValueError                                Traceback (most recent call last)
Input In [8], in <cell line: 3>()
      1 from datasets import load_dataset
----> 3 dataset = load_dataset("TheNoob3131/mosquito-data", split="train")

File ~\Anaconda3\lib\site-packages\datasets\load.py:1679, in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, **config_kwargs)
   1676 try_from_hf_gcs = path not in _PACKAGED_DATASETS_MODULES
   1678 # Download and prepare data
-> 1679 builder_instance.download_and_prepare(
   1680     download_config=download_config,
   1681     download_mode=download_mode,
   1682     ignore_verifications=ignore_verifications,
   1683     try_from_hf_gcs=try_from_hf_gcs,
   1684     use_auth_token=use_auth_token,
   1685 )
   1687 # Build dataset for splits
   1688 keep_in_memory = (
   1689     keep_in_memory if keep_in_memory is not None else is_small_dataset(builder_instance.info.dataset_size)
   1690 )

Is the dataset in the wrong format or is there some security permission that I should enable?

Comments:
I am also getting a ValueError: 'Couldn't cast' at the bottom. Is this because of some delimiter issue? My dataset is on the Huggingface Hub. If you could look at it, that would be greatly appreciated.

Hi @thenerd31, thanks for reporting.

Please note that your issue is not caused by the Hugging Face Datasets library, but it has to do with the specific implementation of your dataset on the Hub.

Therefore, I'm transferring this discussion to your own dataset Community tab: https://huggingface.co/datasets/TheNoob3131/mosquito-data/discussions/1
Dataset Viewer issue for LawalAfeez/englishreview-ds-mini
Opened by lawalAfeez820 (closed)

### Link

_No response_

### Description

Unable to view the split data

### Owner

_No response_

Comments:
It's currently working, as far as I understand

https://huggingface.co/datasets/LawalAfeez/englishreview-ds-mini/viewer/LawalAfeez--englishreview-ds-mini/train

<img width="1556" alt="Capture d’écran 2022-07-19 à 09 24 01" src="https://user-images.githubusercontent.com/1676121/179761130-2d7980b9-c0f6-4093-8b1d-f0a3872fef3f.png">

---

What was your issue?
Support "tags" yaml tag
Opened by lhoestq (closed)

Added the "tags" YAML tag, so that users can specify data domain/topics keywords for dataset search

Comments:
_The documentation is not available anymore as the PR was closed or merged._

IMO `DatasetMetadata` shouldn't crash with attributes that it doesn't know, btw

Yea this PR is mostly to have a validation that this field contains a list of strings.

Regarding unknown fields, the tagging app currently returns an error if a field is unknown using the `DatasetMetadata`. We can change that though
Fix POS tags
Opened by lhoestq (closed)

We're now using `part-of-speech` and not `part-of-speech-tagging`, see discussion here: https://github.com/huggingface/datasets/commit/114c09aff2fa1519597b46fbcd5a8e0c0d3ae020#r78794777

Comments:
_The documentation is not available anymore as the PR was closed or merged._

CI failures are about missing content in the dataset cards or bad tags, and this is unrelated to this PR. Merging :)