konrud/llm_course_chapter5_own_dataset_github_issues huggingface dataset:
MultiWOZ Dataset NonMatchingChecksumError

## Describe the bug
The checksums for the downloaded MultiWOZ dataset and source MultiWOZ dataset aren't matching.

## Steps to reproduce the bug
Both of the below dataset versions yield the checksum error:
```python
from datasets import load_dataset
dataset = load_dataset('multi_woz_v22', 'v2.2')
dataset = load_dataset('multi_woz_v22', 'v2.2_active_only')
```

## Expected results
For the above calls to `load_dataset` to work.

## Actual results
NonMatchingChecksumError. Traceback:
> Traceback (most recent call last):
  File "/Users/brady/anaconda3/envs/elysium/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3441, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-15-4e91280e112e>", line 1, in <module>
    dataset = load_dataset('multi_woz_v22', 'v2.2')
  File "/Users/brady/anaconda3/envs/elysium/lib/python3.8/site-packages/datasets/load.py", line 847, in load_dataset
    builder_instance.download_and_prepare(
  File "/Users/brady/anaconda3/envs/elysium/lib/python3.8/site-packages/datasets/builder.py", line 615, in download_and_prepare
    self._download_and_prepare(
  File "/Users/brady/anaconda3/envs/elysium/lib/python3.8/site-packages/datasets/builder.py", line 675, in _download_and_prepare
    verify_checksums(
  File "/Users/brady/anaconda3/envs/elysium/lib/python3.8/site-packages/datasets/utils/info_utils.py", line 40, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://github.com/budzianowski/multiwoz/raw/master/data/MultiWOZ_2.2/dialog_acts.json', 'https://github.com/budzianowski/multiwoz/raw/master/data/MultiWOZ_2.2/test/dialogues_001.json']

## Environment info
- `datasets` version: 1.11.0
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.10
- PyArrow version: 5.0.0


Hi Brady! I met the similar issue, it stuck in the downloading stage instead of download anything, maybe it is broken. After I change the downloading from URLs to one url of the [Multiwoz project](https://github.com/budzianowski/multiwoz/archive/44f0f8479f11721831c5591b839ad78827da197b.zip) and use dirs to get separate files, the problems gone.
Cache problem in the `load_dataset` method for local compressed file(s)

## Describe the bug

Cache problem in the `load_dataset` method: when modifying a compressed file in a local folder `load_dataset` doesn't detect the change and load the previous version.

## Steps to reproduce the bug

To test it directly, I have prepared a [Google Colaboratory notebook](https://colab.research.google.com/drive/11Em_Amoc-aPGhSBIkSHU2AvEh24nVayy?usp=sharing) that shows this behavior. 

For this example, I have created a toy dataset at: https://huggingface.co/datasets/SaulLu/toy_struc_dataset

This dataset is composed of two versions: 
- v1 on commit `a6beb46` which has a single example `{'id': 1, 'value': {'tag': 'a', 'value': 1}}` in file `train.jsonl.gz`
- v2 on commit `e7935f4` (`main` head) which has a single example `{'attr': 1, 'id': 1, 'value': 'a'}` in file `train.jsonl.gz`

With a terminal, we can start to get the v1 version of the dataset
```bash
git lfs install
git clone https://huggingface.co/datasets/SaulLu/toy_struc_dataset
cd toy_struc_dataset
git checkout a6beb46
```
Then we can load it with python and look at the content:
```python
from datasets import load_dataset

path = "/content/toy_struc_dataset"
dataset = load_dataset(path, data_files={"train": "*.jsonl.gz"})
print(dataset["train"][0])
```
Output
```
{'id': 1, 'value': {'tag': 'a', 'value': 1}} # This is the example in v1
```

With a terminal, we can now start to get the v1 version of the dataset
```bash
git checkout main
```
Then we can load it with python and look at the content:
```python
from datasets import load_dataset

path = "/content/toy_struc_dataset"
dataset = load_dataset(path, data_files={"train": "*.jsonl.gz"})
print(dataset["train"][0])
```
Output
```
{'id': 1, 'value': {'tag': 'a', 'value': 1}} # This is the example in v1 (not v2)
```
## Expected results
The last output should have been 
```
{"id":1, "value": "a", "attr": 1} # This is the example in v2
```

## Ideas

As discussed offline with Quentin, if the cache hash was ever sensitive to changes in a compressed file we would probably not have the problem anymore.

This situation leads me to suggest 2 other features:
- to also have an `load_from_cache_file` argument in the "load_dataset" method
- to reorganize the cache so that we can delete the caches related to a dataset (cf issue #ToBeFilledSoon)

And thanks again for this great library :hugs: 

## Environment info
- `datasets` version: 1.12.1
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyArrow version: 3.0.0


The problem is still present. 
One solution would be to add the `download_mode="force_redownload"` argument to load_dataset. 
However, doing so may lead to a `DatasetGenerationError: An error occurred while generating the dataset`. To mitigate, just do:
`rm -r ~/.cache/huggingface/datasets/*`
Update legacy Python image for CI tests in Linux

Instead of legacy, use next-generation convenience images, built from the ground up with CI, efficiency, and determinism in mind. Here are some of the highlights:

- Faster spin-up time - In Docker terminology, these next-gen images will generally have fewer and smaller layers. Using these new images will lead to faster image downloads when a build starts, and a higher likelihood that the image is already cached on the host.

- Improved reliability and stability - The existing legacy convenience images are rebuilt practically every day with potential changes from upstream that we cannot always test fast enough. This leads to frequent breaking changes, which is not the best environment for stable, deterministic builds. Next-gen images will only be rebuilt for security and critical-bugs, leading to more stable and deterministic images.

More info: https://circleci.com/docs/2.0/circleci-images

There is an exception when running `pip install .[tests]`:
```
Processing /home/circleci/datasets
Collecting numpy>=1.17 (from datasets==1.12.2.dev0)
  Downloading https://files.pythonhosted.org/packages/45/b2/6c7545bb7a38754d63048c7696804a0d947328125d81bf12beaa692c3ae3/numpy-1.19.5-cp36-cp36m-manylinux1_x86_64.whl (13.4MB)
    100% |████████████████████████████████| 13.4MB 3.9MB/s  eta 0:00:011

...

Collecting faiss-cpu (from datasets==1.12.2.dev0)
  Downloading https://files.pythonhosted.org/packages/87/91/bf8ea0d42733cbb04f98d3bf27808e4919ceb5ec71102e21119398a97237/faiss-cpu-1.7.1.post2.tar.gz (41kB)
    100% |████████████████████████████████| 51kB 30.9MB/s ta 0:00:01
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "/home/circleci/.pyenv/versions/3.6.14/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "/home/circleci/.pyenv/versions/3.6.14/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/home/circleci/.pyenv/versions/3.6.14/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "/home/circleci/.pyenv/versions/3.6.14/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-1pop4blm/numpy-1.21.2/setup.py", line 34, in <module>
        method can be invoked.
    RuntimeError: Python version >= 3.7 required.
```

Apparently, `numpy-1.21.2` tries to be installed in the temporary directory `/tmp/easy_install-1pop4blm` instead of the downloaded `numpy-1.19.5` (requirement of `datasets`).

This is caused because `pip` downloads the `.tar.gz` (instead of the `.whl`) and tries to build it in a tmp dir.
Run tests in parallel

Run CI tests in parallel to speed up the test suite.

Speed up results:
- Linux: from `7m 30s` to `5m 32s`
- Windows: from `13m 52s` to `11m 10s`


There is a speed up in Windows machines:
- From `13m 52s` to `11m 10s`

In Linux machines, some workers crash with error message:
```
OSError: [Errno 12] Cannot allocate memory
```

There is also a speed up in Linux machines:
- From `7m 30s` to `5m 32s`
Trying to get in touch regarding a security issue

Hey there!

I'd like to report a security issue but cannot find contact instructions on your repository.

If not a hassle, might you kindly add a `SECURITY.md` file with an email, or another contact method? GitHub [recommends](https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository) this best practice to ensure security issues are responsibly disclosed, and it would serve as a simple instruction for security researchers in the future.

Thank you for your consideration, and I look forward to hearing from you!

(cc @huntr-helper)

Hi @JamieSlome,

Thanks for reaching out. Yes, you are right: I'm opening a PR to add the `SECURITY.md` file and a contact method.

In the meantime, please feel free to report the security issue to: feedback@huggingface.co
Dummy labels no longer on by default in `to_tf_dataset`

After more experimentation, I think I have a way to do things that doesn't depend on adding `dummy_labels` - they were quite a hacky solution anyway!

@lhoestq Let me make sure we never need it, and if not then I'll remove it entirely in a follow-up PR.

Thanks ;) it will be less confusing and easier to maintain to not keep unused hacky features
Introduce web and wiki config in triviaqa dataset

The TriviaQA paper suggests that the two subsets (Wikipedia and Web)
should be treated differently. There are also different leaderboards
for the two sets on CodaLab. For that reason, introduce additional
builder configs in the trivia_qa dataset.

I just made the dummy data smaller :)
Once github refreshes the change I think we can merge !

Thank you so much for reviewing and accepting my pull request!! :)

I created these rather large dummy data sets to cover all different cases for the row structure. E.g. in the web configuration, it's possible that a row has evidence from wikipedia ("EntityPages") and the web ("SearchResults"). But it also might happen that either EntityPages or SearchResults is empty. Probably, I will add this thought to the dataset description in the future.

Ok I see ! Yes feel free to mention it in the dataset card, this can be useful.

For the dummy data though we can keep the small ones, as the tests are mainly about testing the parsing from the dataset script rather than the actual content of the dataset.
Protect master branch

After accidental merge commit (91c55355b634d0dc73350a7ddee1a6776dbbdd69) into `datasets` master branch, all commits present in the feature branch were permanently added to `datasets` master branch history, as e.g.:
- 00cc036fea7c7745cfe722360036ed306796a3f2
- 13ae8c98602bbad8197de3b9b425f4c78f582af1
- ...

I propose to protect our master branch, so that we avoid we can accidentally make this kind of mistakes in the future:
- [x] For Pull Requests using GitHub, allow only squash merging, so that only a single commit per Pull Request is merged into the master branch
  - Currently, simple merge commits are already disabled
  - I propose to disable rebase merging as well
- ~~Protect the master branch from direct pushes (to avoid accidentally pushing of merge commits)~~
  - ~~This protection would reject direct pushes to master branch~~
  - ~~If so, for each release (when we need to commit directly to the master branch), we should previously disable the protection and re-enable it again after the release~~
- [x] Protect the master branch only from direct pushing of **merge commits**
  - GitHub offers the possibility to protect the master branch only from merge commits (which are the ones that introduce all the commits from the feature branch into the master branch).
  - No need to disable/re-enable this protection on each release 

This purpose of this Issue is to open a discussion about this problem and to agree in a solution.

Cool, I think we can do both :)

@lhoestq now the 2 are implemented.

Please note that for the the second protection, finally I have chosen to protect the master branch only from **merge commits** (see update comment above), so no need to disable/re-enable the protection on each release (direct commits, different from merge commits, can be pushed to the remote master branch; and eventually reverted without messing up the repo history).
Add  `remove_columns` to `IterableDataset ` 

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is.

```python
from datasets import load_dataset
dataset = load_dataset("c4", 'realnewslike', streaming =True, split='train')
dataset = dataset.remove_columns('url')
```
```
AttributeError: 'IterableDataset' object has no attribute 'remove_columns'
```

**Describe the solution you'd like**

It would be nice to have `.remove_columns()` to match the `Datasets` api. 


**Describe alternatives you've considered**

This can be done with a single call to `.map()`, 

I can try to help add this. 🤗

Hi ! Good idea :)
If you are interested in contributing, feel free to give it a try and open a Pull Request. Also let me know if I can help you with this or if you have questions
Add SEDE dataset

This PR adds the SEDE dataset for the task of realistic Text-to-SQL, following the instructions of how to add a database and a dataset card.

Please see our paper for more details: https://arxiv.org/abs/2106.05006

Thanks @albertvillanova for your great suggestions! I just pushed a new commit with the necessary fixes. For some reason, the test `test_metric_common` failed for `meteor` metric, which doesn't have any connection to this PR, so I'm trying to rebase and see if it helps.

Hi @Hazoom,

You were right: the non-passing test had nothing to do with this PR.

Unfortunately, you did a git rebase (instead of a git merge), which is not recommended once you have already opened a pull request because you mess up your pull request history. You can see that your pull request now contains:
- your commits repeated two times
- and commits which are not yours from the master branch

If you would like to clean your pull request, please make:
```
git reset --hard 587b93a
git fetch upstream master
git merge upstream/master
git push --force origin sede
```

> Hi @Hazoom,
> 
> You were right: the non-passing test had nothing to do with this PR.
> 
> Unfortunately, you did a git rebase (instead of a git merge), which is not recommended once you have already opened a pull request because you mess up your pull request history. You can see that your pull request now contains:
> 
> * your commits repeated two times
> * and commits which are not yours from the master branch
> 
> If you would like to clean your pull request, please make:
> 
> ```
> git reset --hard 587b93a
> git fetch upstream master
> git merge upstream/master
> git push --force origin sede
> ```

Thanks @albertvillanova 

> Nice! Just one final request before approving your pull request:
> 
> As you have updated the "QuerySetId" field data type, the size of the dataset is smaller now. You should regenerate the metadata. Please run:
> 
> ```
> rm datasets/sede/dataset_infos.json
> datasets-cli test datasets/sede --save_infos --all_configs
> ```

@albertvillanova Good catch, just fixed it.
OSCAR unshuffled_original_ko: NonMatchingSplitsSizesError

## Describe the bug

Cannot download OSCAR `unshuffled_original_ko` due to `NonMatchingSplitsSizesError`.

## Steps to reproduce the bug

```python
>>> dataset = datasets.load_dataset('oscar', 'unshuffled_original_ko')
NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=25292102197, num_examples=7345075, dataset_name='oscar'), 'recorded': SplitInfo(name='train', num_bytes=25284578514, num_examples=7344907, dataset_name='oscar')}]
```

## Expected results

Loading is successful.

## Actual results

Loading throws above error.

## Environment info

- `datasets` version: 1.12.1
- Platform: Linux-5.4.0-81-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 5.0.0


I tried `unshuffled_original_da` and it is also not working
Take namespace into account in caching

Loading a dataset "username/dataset_name" hosted by a user on the hub used to cache the dataset only taking into account the dataset name, and ignorign the username. Because of this, if a user later loads "dataset_name" without specifying the username, it would reload the dataset from the cache instead of failing.

I changed the dataset cache and module cache mechanism to include the username in the name of the cache directory that is used:


`~/.cache/huggingface/datasets/username/dataset_name`  for the data
`~/.cache/huggingface/modules/datasets_modules/datasets/username/dataset_name` for the python files


EDIT: actually using three underscores:
`~/.cache/huggingface/datasets/username___dataset_name` for the data
`~/.cache/huggingface/modules/datasets_modules/datasets/username___dataset_name` for the python files

This PR should fix the issue https://github.com/huggingface/datasets/issues/2842

cc @stas00 

We might have collisions if a username and a dataset_name are the same. Maybe instead serialize the dataset name by replacing `/` with some string, eg `__SLASH__`, that will hopefully never appear in a dataset or user name (it's what I did in https://github.com/huggingface/datasets-preview-backend/blob/master/benchmark/scripts/serialize.py. That way, all the datasets are one-level deep directories

IIRC we enforce that no repo id or username can contain `___` (exactly 3 underscores) specifically for this reason, so you can use that string (that we use in other projects)

cc @Pierrci 

> IIRC we enforce that no repo id or username can contain ___ (exactly 3 underscores) specifically for this reason, so you can use that string (that we use in other projects)

out of curiosity: where is it enforced?

> where is it enforced?

Nowhere yet but we should :) feel free to track in internal tracker and/or implement, as this will be useful in the future

Thanks for the trick, I'm doing the change :)
We can use
`~/.cache/huggingface/datasets/username___dataset_name` for the data
`~/.cache/huggingface/modules/datasets_modules/datasets/username___dataset_name` for the python files

Merging, though it will have to be integrated again the refactor at #2986

@lhoestq we changed a bit the naming policy on the Hub, and the substring '--' is now forbidden, which makes it available for serializing the repo names (`namespace/repo` -> `namespace--repo`). See https://github.com/huggingface/moon-landing/pull/1657 and https://github.com/huggingface/huggingface_hub/pull/545
Add Jigsaw unintended Bias

Hi,

Here's a first attempt at this dataset. Would be great if it could be merged relatively quickly as it is needed for Bigscience-related stuff. 

This requires manual download, and I had some trouble generating dummy_data in this setting, so welcoming feedback there.

Note that the tests seem to fail because of a bug in an Exception at the moment, see: https://github.com/huggingface/datasets/pull/2936 for the fix

@lhoestq implemented your changes, I think this might be ready for another look.

Thanks @lhoestq, implemented the changes, let me know if anything else pops up.
to_tf_dataset keeps a reference to the open data somewhere, causing issues on windows

To reproduce:
```python
import datasets as ds
import weakref
import gc

d = ds.load_dataset("mnist", split="train")
ref = weakref.ref(d._data.table)
tfd = d.to_tf_dataset("image", batch_size=1, shuffle=False, label_cols="label")
del tfd, d
gc.collect()
assert ref() is None, "Error: there is at least one reference left"
```

This causes issues because the table holds a reference to an open arrow file that should be closed. So on windows it's not possible to delete or move the arrow file afterwards.

Moreover the CI test of the `to_tf_dataset` method isn't able to clean up the temporary arrow files because of this.

cc @Rocketknight1 

I did some investigation and, as it seems, the bug stems from [this line](https://github.com/huggingface/datasets/blob/8004d7c3e1d74b29c3e5b0d1660331cd26758363/src/datasets/arrow_dataset.py#L325). The lifecycle of the dataset from the linked line is bound to one of the returned `tf.data.Dataset`. So my (hacky) solution involves wrapping the linked dataset with `weakref.proxy` and adding a custom `__del__` to `tf.python.data.ops.dataset_ops.TensorSliceDataset` (this is the type of a dataset that is returned by `tf.data.Dataset.from_tensor_slices`; this works for TF 2.x, but I'm not sure `tf.python.data.ops.dataset_ops` is a valid path for TF 1.x) that deletes the linked dataset, which is assigned to the dataset object as a property. Will open a draft PR soon!

Thanks a lot for investigating !
Replace script_version with revision

As discussed in https://github.com/huggingface/datasets/pull/2718#discussion_r707013278, the parameter name `script_version` is no longer applicable to datasets without loading script (i.e., datasets only with raw data files).

This PR replaces the parameter name `script_version` with `revision`.

This way, we are also aligned with:
- Transformers: `AutoTokenizer.from_pretrained(..., revision=...)`
- Hub: `HfApi.dataset_info(..., revision=...)`, `HfApi.upload_file(..., revision=...)`

I'm also fine with the removal in 1.15
Mutable columns argument breaks set_format

## Describe the bug
If you pass a mutable list to the `columns` argument of `set_format` and then change the list afterwards, the returned columns also change.

## Steps to reproduce the bug
```python
from datasets import load_dataset
dataset = load_dataset("glue", "cola")

column_list = ["idx", "label"]
dataset.set_format("python", columns=column_list)
column_list[1] = "foo"  # Change the list after we call `set_format`
dataset['train'][:4].keys()
```

## Expected results
```python
dict_keys(['idx', 'label'])
```

## Actual results
```python
dict_keys(['idx'])
```

Pushed a fix to my branch #2731
Error when downloading datasets to non-traditional cache directories

## Describe the bug
When the cache directory is linked (soft link) to a directory on a NetApp device, the download fails. 

## Steps to reproduce the bug
```bash
ln -s /path/to/netapp/.cache ~/.cache
```

```python
load_dataset("imdb")
```

## Expected results
Successfully loading IMDB dataset

## Actual results
```
datasets.utils.info_utils.NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=33432835, 
num_examples=25000, dataset_name='imdb'), 'recorded': SplitInfo(name='train', num_bytes=0, num_examples=0,
 dataset_name='imdb')}, {'expected': SplitInfo(name='test', num_bytes=32650697, num_examples=25000, dataset_name='imdb'),
 'recorded': SplitInfo(name='test', num_bytes=659932, num_examples=503, dataset_name='imdb')}, {'expected':
 SplitInfo(name='unsupervised', num_bytes=67106814, num_examples=50000, dataset_name='imdb'), 'recorded':
 SplitInfo(name='unsupervised', num_bytes=0, num_examples=0, dataset_name='imdb')}]
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.1.2
- Platform: Ubuntu 
- Python version: 3.8

## Extra notes
Stranger yet, trying to debug the phenomenon, I found the range of results to vary a lot without clear direction:
 - With `cache_dir="/path/to/netapp/.cache"` the same thing happens.
 - However, when linking `~/netapp/` to `/path/to/netapp` *and* setting `cache_dir="~/netapp/.cache/huggingface/datasets"` - it does work
 - On the other hand, when linking `~/.cache` to `~/netapp/.cache` without using `cache_dir`, it does work anymore.

While I could test it only for a NetApp device, it might have to do with any other mounted FS.

Thanks :)



Same here !
Add tutorial for no-code dataset upload

This PR is for a tutorial for uploading a dataset to the Hub. It relies on the Hub UI elements to upload a dataset, introduces the online tagging tool for creating tags, and the Dataset card template to get a head start on filling it out. The addition of this tutorial should make it easier for beginners to upload a dataset without accessing the terminal or knowing Git.

Cool, love it ! :)

Feel free to add a paragraph saying how to load the dataset:
```python
from datasets import load_dataset

dataset = load_dataset("stevhliu/demo")

# or to separate each csv file into several splits
data_files = {"train": "train.csv", "test": "test.csv"}
dataset = load_dataset("stevhliu/demo", data_files=data_files)
print(dataset["train"][0])
```

Perfect, feel free to mark this PR ready for review :)

cc @albertvillanova do you have any comment ? You can check the tutorial here:
https://47389-250213286-gh.circle-artifacts.com/0/docs/_build/html/no_code_upload.html

Maybe we can just add a list of supported file types:
- csv
- json
- json lines
- text
- parquet

I just added a mention of the login for private datasets. Don't hesitate to edit or comment.

Otherwise I think it's all good, feel free to merge it @stevhliu  if you don't have other changes to make :)
Unwanted progress bars when accessing examples

When accessing examples from a dataset formatted for pytorch, some progress bars appear when accessing examples:
```python
In [1]: import datasets as ds                                        

In [2]: d = ds.Dataset.from_dict({"a": [0, 1, 2]}).with_format("torch")                                                           

In [3]: d[0]                                                         
100%|████████████████████████████████| 1/1 [00:00<00:00, 3172.70it/s]
Out[3]: {'a': tensor(0)}
```

This is because the pytorch formatter calls `map_nested` that uses progress bars

cc @sgugger 

doing a patch release now :)
windows download abnormal

## Describe the bug
The script clearly exists (accessible from the browser), but the script download fails on windows. Then I tried it again and it can be downloaded normally on linux. why??
## Steps to reproduce the bug
```python3.7 + windows
![image](https://user-images.githubusercontent.com/52347799/133436174-4303f847-55d5-434f-a749-08da3bb9b654.png)


# Sample code to reproduce the bug
```

## Expected results
It can be downloaded normally.

## Actual results
it cann't

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:1.11.0
- Platform:windows
- Python version:3.7
- PyArrow version:


Hi ! Is there some kind of proxy that is configured in your browser that gives you access to internet ? If it's the case it could explain why it doesn't work in the code, since the proxy wouldn't be used

It is indeed an agency problem, thank you very, very much

Let me know if you have other questions :)

Closing this issue now
Add OpenAI's pass@k code evaluation metric

This PR introduces the `code_eval` metric which implements [OpenAI's code evaluation harness](https://github.com/openai/human-eval) introduced in the [Codex paper](https://arxiv.org/abs/2107.03374). It is heavily based on the original implementation and just adapts the interface to follow the `predictions`/`references` convention.

The addition of this metric should enable the evaluation against the code evaluation datasets added in #2897 and #2893.

A few open questions:

- The implementation makes heavy use of multiprocessing which this PR does not touch. Is this conflicting with multiprocessing natively integrated in `datasets`?
- This metric executes generated Python code and as such it poses dangers of executing malicious code. OpenAI addresses this issue by 1) commenting the `exec` call in the code so the user has to actively uncomment it and read the warning and 2) suggests using a sandbox environment (gVisor container). Should we add a similar safeguard? E.g. a prompt that needs to be answered when initialising the metric? Or at least a warning message?
- Naming: the implementation sticks to the `predictions`/`references` naming, however, the references are not reference solutions but unittest to test the solution. While reference solutions are also available they are not used. Should the naming be adapted?

> The implementation makes heavy use of multiprocessing which this PR does not touch. Is this conflicting with multiprocessing natively integrated in datasets?

It should work normally, but feel free to test it.
There is some documentation about using metrics in a distributed setup that uses multiprocessing [here](https://huggingface.co/docs/datasets/loading.html?highlight=rank#distributed-setup)
You can test to spawn several processes where each process would load the metric. Then in each process you add some references/predictions to the metric. Finally you call compute() in each process and on process 0 it should return the result on all the references/predictions

Let me know if you have questions or if I can help

Is there a good way to debug the Windows tests? I suspect it is an issue with `multiprocessing`, but I can't see the error messages.

Indeed it has an issue on windows.
In your example it's supposed to output
```python
{'pass@1': 0.5, 'pass@2': 1.0}
```
but it gets
```python
{'pass@1': 0.0, 'pass@2': 0.0}
```

I'm not on my windows machine today so I can't take a look at it. I can dive into it early next week if you want

> I'm not on my windows machine today so I can't take a look at it. I can dive into it early next week if you want

That would be great - unfortunately I have no access to a windows machine at the moment. I am quite sure it is an issue with in exectue.py because of multiprocessing.
Having a dependency defining fsspec entrypoint raises an AttributeError when importing datasets

## Describe the bug
In one of my project, I defined a custom fsspec filesystem with an entrypoint.
My guess is that by doing so, a variable named `spec` is created in the module `fsspec` (created by entering a for loop as there are entrypoints defined, see the loop in question [here](https://github.com/intake/filesystem_spec/blob/0589358d8a029ed6b60d031018f52be2eb721291/fsspec/__init__.py#L55)).
So that `fsspec.spec`, that was previously referring to the `spec` submodule, is now referring to that `spec` variable.
This make the import of datasets failing as it is using that `fsspec.spec`.

## Steps to reproduce the bug
I could reproduce the bug with a dummy poetry project.

Here is the pyproject.toml:
```toml
[tool.poetry]
name = "debug-datasets"
version = "0.1.0"
description = ""
authors = ["Pierre Godard"]

[tool.poetry.dependencies]
python = "^3.8"
datasets = "^1.11.0"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.plugins."fsspec.specs"]
"file2" = "fsspec.implementations.local.LocalFileSystem"
```

The only other file being a `debug_datasets/__init__.py` empty file.

The overall structure of the project is as follows:
```
.
├── pyproject.toml
└── debug_datasets
    └── __init__.py
```

Then, within the project folder run:

```
poetry install
poetry run python
```

And in the python interpreter, try to import `datasets`:

```
import datasets
```

## Expected results
The import should run successfully.

## Actual results

Here is the trace of the error I get:

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/godarpi/.cache/pypoetry/virtualenvs/debug-datasets-JuFzTKL--py3.8/lib/python3.8/site-packages/datasets/__init__.py", line 33, in <module>
    from .arrow_dataset import Dataset, concatenate_datasets
  File "/home/godarpi/.cache/pypoetry/virtualenvs/debug-datasets-JuFzTKL--py3.8/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 48, in <module>
    from .filesystems import extract_path_from_uri, is_remote_filesystem
  File "/home/godarpi/.cache/pypoetry/virtualenvs/debug-datasets-JuFzTKL--py3.8/lib/python3.8/site-packages/datasets/filesystems/__init__.py", line 30, in <module>
    def is_remote_filesystem(fs: fsspec.spec.AbstractFileSystem) -> bool:
AttributeError: 'EntryPoint' object has no attribute 'AbstractFileSystem'
```

## Suggested fix

`datasets/filesystems/__init__.py`, line 30, replace:
```
    def is_remote_filesystem(fs: fsspec.spec.AbstractFileSystem) -> bool:
```
by:
```
    def is_remote_filesystem(fs: fsspec.AbstractFileSystem) -> bool:
```

I will come up with a PR soon if this effectively solves the issue.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform: WSL2 (Ubuntu 20.04.1 LTS)
- Python version: 3.8.5
- PyArrow version: 5.0.0
- `fsspec` version: 2021.8.1


Closed by #2915.
timit_asr dataset only includes one text phrase

## Describe the bug
The dataset 'timit_asr' only includes one text phrase. It only includes the transcription "Would such an act of refusal be useful?" multiple times rather than different phrases.

## Steps to reproduce the bug
Note: I am following the tutorial https://huggingface.co/blog/fine-tune-wav2vec2-english

1. Install the dataset and other packages
```python
!pip install datasets>=1.5.0
!pip install transformers==4.4.0
!pip install soundfile
!pip install jiwer
```
2. Load the dataset
```python
from datasets import load_dataset, load_metric

timit = load_dataset("timit_asr")
```
3. Remove columns that we don't want
```python
timit = timit.remove_columns(["phonetic_detail", "word_detail", "dialect_region", "id", "sentence_type", "speaker_id"])
```
4. Write a short function to display some random samples of the dataset.
```python
from datasets import ClassLabel
import random
import pandas as pd
from IPython.display import display, HTML

def show_random_elements(dataset, num_examples=10):
    assert num_examples <= len(dataset), "Can't pick more elements than there are in the dataset."
    picks = []
    for _ in range(num_examples):
        pick = random.randint(0, len(dataset)-1)
        while pick in picks:
            pick = random.randint(0, len(dataset)-1)
        picks.append(pick)
    
    df = pd.DataFrame(dataset[picks])
    display(HTML(df.to_html()))

show_random_elements(timit["train"].remove_columns(["file"]))
```

## Expected results
10 random different transcription phrases.

## Actual results
10 of the same transcription phrase "Would such an act of refusal be useful?"

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.4.1
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.5
- PyArrow version: not listed


Hi @margotwagner, 
This bug was fixed in #1995. Upgrading the datasets should work (min v1.8.0 ideally)

Hi @margotwagner,

Yes, as @bhavitvyamalik has commented, this bug was fixed in `datasets` version 1.5.0. You need to update it, as your current version is 1.4.1:
> Environment info
> - `datasets` version: 1.4.1
feat: 🎸 add a function to get a dataset config's split names

Also: pass additional arguments (use_auth_token) to get private configs + info of private datasets on the hub

Questions:

- [x] I'm not sure how the versions work: I changed 1.12.1.dev0 to 1.12.1.dev1, was it correct?
    -> no: reverted
- [x] Should I add a section in https://github.com/huggingface/datasets/blob/master/docs/source/load_hub.rst? (there is no section for get_dataset_infos)
    -> yes: added

> Should I add a section in https://github.com/huggingface/datasets/blob/master/docs/source/load_hub.rst? (there is no section for get_dataset_infos)

Yes totally :) This tutorial should indeed mention this, given how fundamental it is
FORCE_REDOWNLOAD does not work

## Describe the bug
With GenerateMode.FORCE_REDOWNLOAD, the documentation says 
    +------------------------------------+-----------+---------+
    |                                    | Downloads | Dataset |
    +====================================+===========+=========+
    | `REUSE_DATASET_IF_EXISTS` (default)| Reuse     | Reuse   |
    +------------------------------------+-----------+---------+
    | `REUSE_CACHE_IF_EXISTS`            | Reuse     | Fresh   |
    +------------------------------------+-----------+---------+
    | `FORCE_REDOWNLOAD`                 | Fresh     | Fresh   |
    +------------------------------------+-----------+---------+

However, the old dataset is loaded even when FORCE_REDOWNLOAD is chosen.

## Steps to reproduce the bug
```python

import pandas as pd
from datasets import load_dataset, GenerateMode
pd.DataFrame(range(5), columns=['numbers']).to_csv('/tmp/test.tsv.gz', index=False)
ee = load_dataset('csv', data_files=['/tmp/test.tsv.gz'], delimiter='\t', split='train', download_mode=GenerateMode.FORCE_REDOWNLOAD)
print(ee)
pd.DataFrame(range(10), columns=['numerals']).to_csv('/tmp/test.tsv.gz', index=False)
ee = load_dataset('csv', data_files=['/tmp/test.tsv.gz'], delimiter='\t', split='train', download_mode=GenerateMode.FORCE_REDOWNLOAD)
print(ee)

```

## Expected results
Dataset({
    features: ['numbers'],
    num_rows: 5
})
Dataset({
    features: ['numerals'],
    num_rows: 10
})

## Actual results
Dataset({
    features: ['numbers'],
    num_rows: 5
})
Dataset({
    features: ['numbers'],
    num_rows: 5
})


## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.8.0
- Platform: Linux-4.14.181-108.257.amzn1.x86_64-x86_64-with-glibc2.10
- Python version: 3.7.10
- PyArrow version: 3.0.0


Hi ! Thanks for reporting. The error seems to happen only if you use compressed files.

The second dataset is prepared in another dataset cache directory than the first - which is normal, since the source file is different. However, it doesn't uncompress the new data file because it finds the old uncompressed data in the extraction cache directory.

If we fix the extraction cache mechanism to uncompress a local file if it changed then it should fix the issue.
Currently the extraction cache mechanism only takes into account the path of the compressed file, which is an issue.

Facing the same issue, is there any way to overtake this issue until it will be fixed? 

You can clear your extraction cache in the meantime (by default at `~/.cache/huggingface/datasets/downloads/extracted`)
Add WIT Dataset

## Adding a Dataset
- **Name:** *WIT*
- **Description:** *Wikipedia-based Image Text Dataset*
- **Paper:** *[WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning
](https://arxiv.org/abs/2103.01913)*
- **Data:** *https://github.com/google-research-datasets/wit*
- **Motivation:**  (excerpt from their Github README.md)

> - The largest multimodal dataset (publicly available at the time of this writing) by the number of image-text examples.
> - A massively multilingual dataset (first of its kind) with coverage for over 100+ languages.
> - A collection of diverse set of concepts and real world entities.
> - Brings forth challenging real-world test sets.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


@hassiahk is working on it  #2810 

WikiMedia is now hosting the pixel values directly which should make it a lot easier!
The files can be found here:
https://techblog.wikimedia.org/2021/09/09/the-wikipedia-image-caption-matching-challenge-and-a-huge-release-of-image-data-for-research/
https://analytics.wikimedia.org/published/datasets/one-off/caption_competition/training/image_pixels/

> @hassiahk is working on it #2810

Thank you @bhavitvyamalik! Added this issue so we could track progress 😄 . Just linked the PR as well for visibility. 

Hey folks, we are now hosting the merged pixel values + embeddings + metadata ourselves. I gave it a try - [nateraw/wit](https://huggingface.co/datasets/nateraw/wit)

**⚠️ - Make sure you add `streaming=True` unless you're prepared to download 400GB of data!**

```python
from datasets import load_dataset

ds = load_dataset('nateraw/wit', streaming=True)
example = next(iter(ds))
```

```python
>>> example = next(iter(ds['train']))
>>> example.keys()
dict_keys(['b64_bytes', 'original_width', 'image_url', 'wit_features', 'original_height', 'metadata_url', 'mime_type', 'caption_attribution_description', 'embedding'])
>>> example['wit_features'].keys()
dict_keys(['hierarchical_section_title', 'language', 'attribution_passes_lang_id', 'context_section_description', 'is_main_image', 'page_title', 'caption_title_and_reference_description', 'caption_alt_text_description', 'caption_reference_description', 'page_url', 'context_page_description', 'section_title', 'page_changed_recently'])
```

Hi! `datasets` now hosts two versions of the WIT dataset:
* [`google/wit`](https://huggingface.co/datasets/google/wit): Google's version with the image URLs
* [`wikimedia/wit_base`](https://huggingface.co/datasets/wikimedia/wit_base): Wikimedia's version with the images + ResNet embeddings, but with less data than Google's version

Does this dataset work with `data_files` parameter. I tried and it doesn't. it attempts to download ~600 GB data even when i specify single file
Dataset

## Adding a Dataset
- **Name:** *name of the dataset*
- **Description:** *short description of the dataset (or link to social media or blog post)*
- **Paper:** *link to the dataset paper if available*
- **Data:** *link to the Github repository or current dataset location*
- **Motivation:** *what are some good reasons to have this dataset*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).
Hug emoji

## Adding a Dataset
- **Name:** *name of the dataset*
- **Description:** *short description of the dataset (or link to social media or blog post)*
- **Paper:** *link to the dataset paper if available*
- **Data:** *link to the Github repository or current dataset location*
- **Motivation:** *what are some good reasons to have this dataset*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).
Add OpenAI's HumanEval dataset

This PR adds OpenAI's [HumanEval](https://github.com/openai/human-eval) dataset. The dataset consists of 164 handcrafted programming problems with solutions and unittests to verify solution. This dataset is useful to evaluate code generation models.

I just fixed the class name, and added `[More Information Needed]` in empty sections in case people want to complete the dataset card :)
add multi-proc in `to_csv`

This PR extends the multi-proc method used in #2747 for`to_json` to `to_csv` as well. 

Results on my machine post benchmarking on `ascent_kb` dataset (giving ~45% improvement when compared to num_proc = 1):
```
Time taken on 1 num_proc, 10000 batch_size  674.2055702209473
Time taken on 4 num_proc, 10000 batch_size  425.6553490161896

Time taken on 1 num_proc, 50000 batch_size  623.5897650718689
Time taken on 4 num_proc, 50000 batch_size  380.0402421951294

Time taken on 4 num_proc, 100000 batch_size  361.7168130874634
```
This is a WIP as writing tests is pending for this PR. 

I'm also exploring [this](https://arrow.apache.org/docs/python/csv.html#incremental-writing) approach for which I'm using `pyarrow-5.0.0`.


I think you can just add a test `test_dataset_to_csv_multiproc` in `tests/io/test_csv.py` and we'll be good

Hi @lhoestq, 
I've added `test_dataset_to_csv` apart from `test_dataset_to_csv_multiproc` as no test was there to check generated CSV file when `num_proc=1`. Please let me know if anything is also required!
add mbpp dataset

This PR adds the mbpp dataset introduced by Google [here](https://github.com/google-research/google-research/tree/master/mbpp) as mentioned in #2816.

The dataset contain two versions: a full and a sanitized one. They have a slightly different schema and it is current state the loading preserves the original schema. An open question is whether to harmonize the two schemas when loading the dataset or to preserve the original one. Since not all fields are overlapping the schema will not be exactly the same.

I think it's fine to have the original schema
Error when encoding a dataset with None objects with a Sequence feature

There is an error when encoding a dataset with None objects with a Sequence feature

To reproduce:
```python
from datasets import Dataset, Features, Value, Sequence
data = {"a": [[0], None]}
features = Features({"a": Sequence(Value("int32"))})
dataset = Dataset.from_dict(data, features=features)
```
raises

```python
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-24-40add67f8751> in <module>
      2 data = {"a": [[0], None]}
      3 features = Features({"a": Sequence(Value("int32"))})
----> 4 dataset = Dataset.from_dict(data, features=features)
[...]
~/datasets/features.py in encode_nested_example(schema, obj)
    888         if isinstance(obj, str):  # don't interpret a string as a list
    889             raise ValueError("Got a string but expected a list instead: '{}'".format(obj))
--> 890         return [encode_nested_example(schema.feature, o) for o in obj]
    891     # Object with special encoding:
    892     # ClassLabel will convert from string to int, TranslationVariableLanguages does some checks

TypeError: 'NoneType' object is not iterable
```

Instead, if should run without error, as if the `features` were not passed

This has been fixed by https://github.com/huggingface/datasets/pull/2900
We're doing a new release 1.12 today to make the fix available :)
0x290B112ED1280537B24Ee6C268a004994a16e6CE

## Adding a Dataset
- **Name:** *name of the dataset*
- **Description:** *short description of the dataset (or link to social media or blog post)*
- **Paper:** *link to the dataset paper if available*
- **Data:** *link to the Github repository or current dataset location*
- **Motivation:** *what are some good reasons to have this dataset*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).
Coc

## Adding a Dataset
- **Name:** *name of the dataset*
- **Description:** *short description of the dataset (or link to social media or blog post)*
- **Paper:** *link to the dataset paper if available*
- **Data:** *link to the Github repository or current dataset location*
- **Motivation:** *what are some good reasons to have this dataset*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).