sunny2309/githug-issues huggingface dataset:
LexGLUE: A Benchmark Dataset for Legal Language Understanding in English.

Inspired by the recent widespread use of the GLUE multi-task benchmark NLP dataset (Wang et al., 2018), the subsequent more difficult SuperGLUE (Wang et al., 2019), other previous multi-task NLP benchmarks (Conneau and Kiela, 2018; McCann et al., 2018), and similar initiatives in other domains (Peng et al., 2019), we introduce the Legal General Language Understanding Evaluation (LexGLUE) benchmark, a benchmark dataset to evaluate the performance of NLP methods in legal tasks. LexGLUE is based on seven existing legal NLP datasets, selected using criteria largely from SuperGLUE.

As in GLUE and SuperGLUE (Wang et al., 2019b,a), one of our goals is to push towards generic (or ‘foundation’) models that can cope with multiple NLP tasks, in our case legal NLP tasks possibly with limited task-specific fine-tuning. Another goal is to provide a convenient and informative entry point for NLP researchers and practitioners wishing to explore or develop methods for legalNLP. Having these goals in mind, the datasets we include in LexGLUE and the tasks they address have been simplified in several ways to make it easier for newcomers and generic models to address all tasks.

LexGLUE benchmark is accompanied by experimental infrastructure that relies on Hugging Face Transformers library and resides at: https://github.com/coastalcph/lex-glue.
Remove a reference to the open Arrow file when deleting a TF dataset created with to_tf_dataset

This [comment](https://github.com/huggingface/datasets/issues/2934#issuecomment-922970919) explains the issue. This PR fixes that with a `weakref` callback, and additionally:
* renames `TensorflowDatasetMixIn` to `TensorflowDatasetMixin` for consistency
* correctly indents `TensorflowDatasetMixin`'s docstring
* replaces `tf.data.AUTOTUNE` with `tf.data.experimental.AUTOTUNE` (we support TF>=2.2 according to the [setup.py](https://github.com/huggingface/datasets/blob/fc46bba66ba4f432cc10501c16a677112e13984c/setup.py#L188) and `AUTOTUNE` has been moved to the experimental part of `tf.data` in 1.X if I'm not mistaken)

Fixes #2934
Fix json loader when conversion not implemented

Sometimes the arrow json parser fails if the `block_size` is too small and returns an `ArrowNotImplementedError: JSON conversion to struct...` error.

By increasing the block size it makes it work again.

Hopefully it should help with https://github.com/huggingface/datasets/issues/2799

I tried with the file mentioned in the issue and it worked for me
cc @lewtun can you try again from this branch ?
Dataset has incorrect labels

The dataset https://huggingface.co/datasets/turkish_product_reviews has incorrect labels - all reviews are labelled with "1" (positive sentiment). None of the reviews is labelled with "0". See screenshot attached:

![Capture](https://user-images.githubusercontent.com/63367770/135617428-14ce0b27-5208-4e66-a3ee-71542e3257b4.PNG)
IndexError: Invalid key: 14 is out of bounds for size 0

## Describe the bug
A clear and concise description of what the bug is.

Hi. I am trying to implement stochastic weighted averaging optimizer with transformer library as described here https://pytorch.org/blog/pytorch-1.6-now-includes-stochastic-weight-averaging/ , for this I am using a run_clm.py codes which is working fine before adding SWA optimizer, the moment I modify the model with `swa_model = AveragedModel(model)` in this script, I am getting the below error,  since I am NOT touching the dataloader part, I am confused why this is occurring, I very much appreciate your opinion on this @lhoestq 

## Steps to reproduce the bug
```
Traceback (most recent call last):
  File "run_clm.py", line 723, in <module>
    main()
  File "run_clm.py", line 669, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/transformers/trainer.py", line 1258, in train
    for step, inputs in enumerate(epoch_iterator):
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 475, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1530, in __getitem__
    format_kwargs=self._format_kwargs,
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1517, in _getitem
    pa_subtable = query_table(self._data, key, indices=self._indices if self._indices is not None else None)
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/formatting/formatting.py", line 368, in query_table
    _check_valid_index_key(key, size)
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/formatting/formatting.py", line 311, in _check_valid_index_key
    raise IndexError(f"Invalid key: {key} is out of bounds for size {size}")
IndexError: Invalid key: 14 is out of bounds for size 0
```



## Expected results
not getting the index error

## Actual results
Please see the above

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: datasets                  1.12.1 
- Platform: linux
- Python version: 3.7.11 
- PyArrow version:  5.0.0
OpenSLR 25: ASR data for Amharic, Swahili and Wolof

## Adding a Dataset
- **Name:** *SLR25*
- **Description:** *Subset 25 from OpenSLR. Other subsets have been added to https://huggingface.co/datasets/openslr, 25 covers Amharic, Swahili and Wolof data*
- **Paper:** *https://www.openslr.org/25/ has citations for each of the three subsubsets. *
- **Data:** *Currently the three links to the .tar.bz2 files can be found a thttps://www.openslr.org/25/*
- **Motivation:** *Increase ASR data for underrepresented African languages. Also, other subsets of OpenSLR speech recognition have been uploaded, so this would be easy.*


https://github.com/huggingface/datasets/blob/master/datasets/openslr/openslr.py already has been created for various other OpenSLR subsets, this should be relatively straightforward to do.
ValueError when computing f1 metric with average None

## Describe the bug
When I try to compute the f1 score for each class in a multiclass classification problem, I get a ValueError. The same happens with recall and precision. I traced the error to the `.item()` in these scripts, which is probably there for the other averages. E.g. from f1.py:
```python 
return {
            "f1": f1_score(
                references,
                predictions,
                labels=labels,
                pos_label=pos_label,
                average=average,
                sample_weight=sample_weight,
            ).item(),
        }
```
Since the result is an array with more than one item, the `.item()` throws the error. I didn't submit a PR because this might be needed for the other averages, I'm not very familiar with the library

## Steps to reproduce the bug
```python
from datasets import load_metric
metric = load_metric("f1")
metric.add_batch(predictions=[2,34,1,34,1,2,3], references=[23,52,1,3,523,5,8])
metric.compute(average=None)
```

## Expected results
`array([0.66666667, 0.        , 0.        , 0.        , 0.        ,
       0.        , 0.        , 0.        , 0.        ])`

## Actual results
ValueError: can only convert an array of size 1 to a Python scalar

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.9.5
- PyArrow version: 5.0.0
Impossible to load compressed csv

## Describe the bug
It is not possible to load from a compressed csv anymore.

## Steps to reproduce the bug
```python
load_dataset('csv', data_files=['/path/to/csv.bz2'])
```

## Problem and possible solution
This used to work, but the commit that broke it is [this one](https://github.com/huggingface/datasets/commit/ad489d4597381fc2d12c77841642cbeaecf7a2e0#diff-6f60f8d0552b75be8b3bfd09994480fd60dcd4e7eb08d02f721218c3acdd2782).

`pandas` usually gets the compression information from the filename itself (which was previously directly passed). Now, since it gets a file descriptor, it might be good to auto-infer the compression or let the user pass the `compression` kwarg to `load_dataset` (or maybe warn the user if the file ends with a commonly known compression scheme?).

## Environment info
- `datasets` version: 1.10.0 (and over)
- Platform: Linux-5.8.0-45-generic-x86_64-with-glibc2.17
- Python version: 3.8.10
- PyArrow version: 3.0.0
Can't load dataset

I'm trying to load a wikitext dataset

```
from datasets import load_dataset
raw_datasets = load_dataset("wikitext")
```

ValueError: Config name is missing.
Please pick one among the available configs: ['wikitext-103-raw-v1', 'wikitext-2-raw-v1', 'wikitext-103-v1', 'wikitext-2-v1']
Example of usage:
	`load_dataset('wikitext', 'wikitext-103-raw-v1')`.

If I try
```
from datasets import load_dataset
raw_datasets = load_dataset("wikitext-2-v1")
```

FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/1.12.1/datasets/wikitext-2-v1/wikitext-2-v1.py

#### Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic (colab)
- Python version: 3.7.12
- PyArrow version: 3.0.0
OSError: Not enough disk space.

## Describe the bug
I'm trying to download `natural_questions` dataset from the Internet, and I've specified the cache_dir which locates in a mounted disk and has enough disk space. However, even though the space is enough, the disk space checking function still reports the space of root `/` disk having no enough space. 

The file system structure is like below. The root `/` has `115G` disk space available, and the `sda1` is mounted to `/mnt`, which has `1.2T` disk space available:
```
/
/mnt/sda1/path/to/args.dataset_cache_dir 
```

## Steps to reproduce the bug
```python
dataset_config = DownloadConfig(
    cache_dir=os.path.abspath(args.dataset_cache_dir),
    resume_download=True,
)
dataset = load_dataset("natural_questions", download_config=dataset_config)
```

## Expected results

Can download the dataset without an error.

## Actual results

The following error raised:
```
OSError: Not enough disk space. Needed: 134.92 GiB (download: 41.97 GiB, generated: 92.95 GiB, post-processed: Unknown size)
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.9.0
- Platform: Ubuntu 18.04
- Python version: 3.8.10
- PyArrow version:
masakhaner dataset load problem

## Describe the bug
Masakhaner dataset is not loading

## Steps to reproduce the bug
```
from datasets import load_dataset
dataset = load_dataset("masakhaner",'amh')
```

## Expected results
Expected the return of a dataset

## Actual results

```

NonMatchingSplitsSizesError               Traceback (most recent call last)
<ipython-input-3-a6abc1161d4c> in <module>()
      1 from datasets import load_dataset
      2 
----> 3 dataset = load_dataset("masakhaner",'amh')



3 frames
/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py
 in verify_splits(expected_splits, recorded_splits)
     72     ]
     73     if len(bad_splits) > 0:
---> 74         raise NonMatchingSplitsSizesError(str(bad_splits))
     75     logger.info("All the splits matched successfully.")
     76 

NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=639927, num_examples=1751, dataset_name='masakhaner'), 'recorded': SplitInfo(name='train', num_bytes=639911, num_examples=1750, dataset_name='masakhaner')}, {'expected': SplitInfo(name='validation', num_bytes=92768, num_examples=251, dataset_name='masakhaner'), 'recorded': SplitInfo(name='validation', num_bytes=92753, num_examples=250, dataset_name='masakhaner')}, {'expected': SplitInfo(name='test', num_bytes=184286, num_examples=501, dataset_name='masakhaner'), 'recorded': SplitInfo(name='test', num_bytes=184271, num_examples=500, dataset_name='masakhaner')}]
```

## Environment info
Google Colab
medical-dialog error

## Describe the bug
A clear and concise description of what the bug is.
When I attempt to download the huggingface datatset medical_dialog it errors out midway through
## Steps to reproduce the bug
```python
raw_datasets = load_dataset("medical_dialog", "en", split="train", download_mode="force_redownload", data_dir="./Medical-Dialogue-Dataset-English")
```

## Expected results
A clear and concise description of the expected results.
No error
## Actual results

```
3 frames
/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py in verify_splits(expected_splits, recorded_splits)
     72     ]
     73     if len(bad_splits) > 0:
---> 74         raise NonMatchingSplitsSizesError(str(bad_splits))
     75     logger.info("All the splits matched successfully.")
     76 

NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=0, num_examples=0, dataset_name='medical_dialog'), 'recorded': SplitInfo(name='train', num_bytes=295097913, num_examples=229674, dataset_name='medical_dialog')}]
```
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.21.1
- Platform: colab
- Python version: colab 3.7
- PyArrow version: N/A
Invalid download URL of WMT17 `zh-en` data 

## Describe the bug
Partial data (wmt17 zh-en) cannot be downloaded due to an invalid URL.

## Steps to reproduce the bug
```python
from datasets import load_dataset
 dataset = load_dataset('wmt17','zh-en')
```

## Expected results
ConnectionError: Couldn't reach ftp://cwmt-wmt:cwmt-wmt@datasets.nju.edu.cn/parallel/casia2015.zip
MultiWOZ Dataset NonMatchingChecksumError

## Describe the bug
The checksums for the downloaded MultiWOZ dataset and source MultiWOZ dataset aren't matching.

## Steps to reproduce the bug
Both of the below dataset versions yield the checksum error:
```python
from datasets import load_dataset
dataset = load_dataset('multi_woz_v22', 'v2.2')
dataset = load_dataset('multi_woz_v22', 'v2.2_active_only')
```

## Expected results
For the above calls to `load_dataset` to work.

## Actual results
NonMatchingChecksumError. Traceback:
> Traceback (most recent call last):
  File "/Users/brady/anaconda3/envs/elysium/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3441, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-15-4e91280e112e>", line 1, in <module>
    dataset = load_dataset('multi_woz_v22', 'v2.2')
  File "/Users/brady/anaconda3/envs/elysium/lib/python3.8/site-packages/datasets/load.py", line 847, in load_dataset
    builder_instance.download_and_prepare(
  File "/Users/brady/anaconda3/envs/elysium/lib/python3.8/site-packages/datasets/builder.py", line 615, in download_and_prepare
    self._download_and_prepare(
  File "/Users/brady/anaconda3/envs/elysium/lib/python3.8/site-packages/datasets/builder.py", line 675, in _download_and_prepare
    verify_checksums(
  File "/Users/brady/anaconda3/envs/elysium/lib/python3.8/site-packages/datasets/utils/info_utils.py", line 40, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://github.com/budzianowski/multiwoz/raw/master/data/MultiWOZ_2.2/dialog_acts.json', 'https://github.com/budzianowski/multiwoz/raw/master/data/MultiWOZ_2.2/test/dialogues_001.json']

## Environment info
- `datasets` version: 1.11.0
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.10
- PyArrow version: 5.0.0
Cache problem in the `load_dataset` method for local compressed file(s)

## Describe the bug

Cache problem in the `load_dataset` method: when modifying a compressed file in a local folder `load_dataset` doesn't detect the change and load the previous version.

## Steps to reproduce the bug

To test it directly, I have prepared a [Google Colaboratory notebook](https://colab.research.google.com/drive/11Em_Amoc-aPGhSBIkSHU2AvEh24nVayy?usp=sharing) that shows this behavior. 

For this example, I have created a toy dataset at: https://huggingface.co/datasets/SaulLu/toy_struc_dataset

This dataset is composed of two versions: 
- v1 on commit `a6beb46` which has a single example `{'id': 1, 'value': {'tag': 'a', 'value': 1}}` in file `train.jsonl.gz`
- v2 on commit `e7935f4` (`main` head) which has a single example `{'attr': 1, 'id': 1, 'value': 'a'}` in file `train.jsonl.gz`

With a terminal, we can start to get the v1 version of the dataset
```bash
git lfs install
git clone https://huggingface.co/datasets/SaulLu/toy_struc_dataset
cd toy_struc_dataset
git checkout a6beb46
```
Then we can load it with python and look at the content:
```python
from datasets import load_dataset

path = "/content/toy_struc_dataset"
dataset = load_dataset(path, data_files={"train": "*.jsonl.gz"})
print(dataset["train"][0])
```
Output
```
{'id': 1, 'value': {'tag': 'a', 'value': 1}} # This is the example in v1
```

With a terminal, we can now start to get the v1 version of the dataset
```bash
git checkout main
```
Then we can load it with python and look at the content:
```python
from datasets import load_dataset

path = "/content/toy_struc_dataset"
dataset = load_dataset(path, data_files={"train": "*.jsonl.gz"})
print(dataset["train"][0])
```
Output
```
{'id': 1, 'value': {'tag': 'a', 'value': 1}} # This is the example in v1 (not v2)
```
## Expected results
The last output should have been 
```
{"id":1, "value": "a", "attr": 1} # This is the example in v2
```

## Ideas

As discussed offline with Quentin, if the cache hash was ever sensitive to changes in a compressed file we would probably not have the problem anymore.

This situation leads me to suggest 2 other features:
- to also have an `load_from_cache_file` argument in the "load_dataset" method
- to reorganize the cache so that we can delete the caches related to a dataset (cf issue #ToBeFilledSoon)

And thanks again for this great library :hugs: 

## Environment info
- `datasets` version: 1.12.1
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyArrow version: 3.0.0
Update legacy Python image for CI tests in Linux

Instead of legacy, use next-generation convenience images, built from the ground up with CI, efficiency, and determinism in mind. Here are some of the highlights:

- Faster spin-up time - In Docker terminology, these next-gen images will generally have fewer and smaller layers. Using these new images will lead to faster image downloads when a build starts, and a higher likelihood that the image is already cached on the host.

- Improved reliability and stability - The existing legacy convenience images are rebuilt practically every day with potential changes from upstream that we cannot always test fast enough. This leads to frequent breaking changes, which is not the best environment for stable, deterministic builds. Next-gen images will only be rebuilt for security and critical-bugs, leading to more stable and deterministic images.

More info: https://circleci.com/docs/2.0/circleci-images
Trying to get in touch regarding a security issue

Hey there!

I'd like to report a security issue but cannot find contact instructions on your repository.

If not a hassle, might you kindly add a `SECURITY.md` file with an email, or another contact method? GitHub [recommends](https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository) this best practice to ensure security issues are responsibly disclosed, and it would serve as a simple instruction for security researchers in the future.

Thank you for your consideration, and I look forward to hearing from you!

(cc @huntr-helper)
Protect master branch

After accidental merge commit (91c55355b634d0dc73350a7ddee1a6776dbbdd69) into `datasets` master branch, all commits present in the feature branch were permanently added to `datasets` master branch history, as e.g.:
- 00cc036fea7c7745cfe722360036ed306796a3f2
- 13ae8c98602bbad8197de3b9b425f4c78f582af1
- ...

I propose to protect our master branch, so that we avoid we can accidentally make this kind of mistakes in the future:
- [x] For Pull Requests using GitHub, allow only squash merging, so that only a single commit per Pull Request is merged into the master branch
  - Currently, simple merge commits are already disabled
  - I propose to disable rebase merging as well
- ~~Protect the master branch from direct pushes (to avoid accidentally pushing of merge commits)~~
  - ~~This protection would reject direct pushes to master branch~~
  - ~~If so, for each release (when we need to commit directly to the master branch), we should previously disable the protection and re-enable it again after the release~~
- [x] Protect the master branch only from direct pushing of **merge commits**
  - GitHub offers the possibility to protect the master branch only from merge commits (which are the ones that introduce all the commits from the feature branch into the master branch).
  - No need to disable/re-enable this protection on each release 

This purpose of this Issue is to open a discussion about this problem and to agree in a solution.
Add  `remove_columns` to `IterableDataset ` 

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is.

```python
from datasets import load_dataset
dataset = load_dataset("c4", 'realnewslike', streaming =True, split='train')
dataset = dataset.remove_columns('url')
```
```
AttributeError: 'IterableDataset' object has no attribute 'remove_columns'
```

**Describe the solution you'd like**

It would be nice to have `.remove_columns()` to match the `Datasets` api. 


**Describe alternatives you've considered**

This can be done with a single call to `.map()`, 

I can try to help add this. 🤗
OSCAR unshuffled_original_ko: NonMatchingSplitsSizesError

## Describe the bug

Cannot download OSCAR `unshuffled_original_ko` due to `NonMatchingSplitsSizesError`.

## Steps to reproduce the bug

```python
>>> dataset = datasets.load_dataset('oscar', 'unshuffled_original_ko')
NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=25292102197, num_examples=7345075, dataset_name='oscar'), 'recorded': SplitInfo(name='train', num_bytes=25284578514, num_examples=7344907, dataset_name='oscar')}]
```

## Expected results

Loading is successful.

## Actual results

Loading throws above error.

## Environment info

- `datasets` version: 1.12.1
- Platform: Linux-5.4.0-81-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 5.0.0