noamaanMulla-03/datasets-issues huggingface dataset:
BLEURT: Match key names to correspond with filename

In order to properly locate downloaded ckpt files key name needs to match filename. Correcting change introduced in #3235 


Thanks for the suggestion! I think the current checked-in `CHECKPOINT_URLS` is already not working. I believe anyone who tried using the new ckpts (`BLEURT-20-X`) can't unless this fix is in. The zip file from bleurt side unzips to directory name matching the filename (capitalized for new ones). For example without current changes I'd get the following error

```
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-5-f6832fe20f84> in <module>()
      1 predictions = ["hello there", "general kenobi"]
      2 references = ["hello there", "general kenobi"]
----> 3 bleurt = datasets.load_metric("bleurt", "bleurt-20")
      4 results = bleurt.compute(predictions=predictions, references=references)

4 frames
/usr/local/lib/python3.7/dist-packages/bleurt/checkpoint.py in read_bleurt_config(path)
     84   """Reads and checks config file from a BLEURT checkpoint."""
     85   assert tf.io.gfile.exists(path), \
---> 86       "Could not find BLEURT checkpoint {}".format(path)
     87   config_path = os.path.join(path, CONFIG_FILE)
     88   assert tf.io.gfile.exists(config_path), \

AssertionError: Could not find BLEURT checkpoint /root/.cache/huggingface/metrics/bleurt/bleurt-20/downloads/extracted/e34c60f1a05394ecda54e253a10413ca7b5d59f9a23f3cc73258c6b78ffa2f50/bleurt-20
```
inspecting specified path I see that directory name is `BLEURT-20` instead of `bleurt-20`. 
Other solution similar to your suggestion is meddle with `dl_manager.download_and_extract` to unzip to paths with lowering all the paths but I imagine this will affect other parts of the library. 

Indeed, good catch ! Your solution that fixes `CHECKPOINT_URLS ` is simple and works well, thanks :)

Furthermore to avoid breaking changes though we could also keep the support for the lowercase one:
```python
        if self.config_name.lower() in CHECKPOINT_URLS:
            checkpoint_name = self.config_name.lower()
        elif self.config_name.upper() in CHECKPOINT_URLS:
            checkpoint_name = self.config_name.upper()
        else:
            raise KeyError(
                f"{self.config_name} model not found. You should supply the name of a model checkpoint for bleurt in {CHECKPOINT_URLS.keys()}"
            )
```
and then we can use `checkpoint_name` instead of `self.config_name` to download and instantiate the model:
```python
        model_path = dl_manager.download_and_extract(CHECKPOINT_URLS[checkpoint_name])
        self.scorer = score.BleurtScorer(os.path.join(model_path, checkpoint_name))
```

Please let me know if that sounds reasonable to you !

Thanks for the suggestion! I believe your suggestion should work to make keys case insensitive. Changes are committed to the PR now.
Add Natural Adversarial Objects dataset

## Adding a Dataset
- **Name:** Natural Adversarial Objects (NAO)
- **Description:** Natural Adversarial Objects (NAO) is a new dataset to evaluate the robustness of object detection models. NAO contains 7,934 images and 9,943 objects that are unmodified and representative of real-world scenarios, but cause state-of-the-art detection models to misclassify with high confidence.
- **Paper:** https://arxiv.org/abs/2111.04204v1
- **Data:** https://drive.google.com/drive/folders/15P8sOWoJku6SSEiHLEts86ORfytGezi8
- **Motivation:** interesting object detection dataset useful for miscclassifications

cc @NielsRogge 

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Labels: name: dataset request

name: vision
Add ACAV100M Dataset

## Adding a Dataset
- **Name:** *ACAV100M*
- **Description:** *contains 100 million videos with high audio-visual correspondence, ideal for self-supervised video representation learning.*
- **Paper:** *https://arxiv.org/abs/2101.10803*
- **Data:** *https://github.com/sangho-vision/acav100m*
- **Motivation:** *The largest dataset (to date) for audio-visual learning.*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Labels: name: dataset request

name: vision
Wound segmentation datasets

## Adding a Dataset
- **Name:** Wound segmentation datasets
- **Description:** annotated wound image dataset 
- **Paper:** https://www.nature.com/articles/s41598-020-78799-w
- **Data:** https://github.com/uwm-bigdata/wound-segmentation
- **Motivation:** Interesting simple image dataset, useful for segmentation, with visibility due to http://www.miccai.org/special-interest-groups/challenges/ and https://fusc.grand-challenge.org/

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Labels: name: dataset request

name: vision
Add Hateful Memes Dataset

## Add Hateful Memes Dataset
- **Name:** Hateful Memes
- **Description:** [https://ai.facebook.com/blog/hateful-memes-challenge-and-data-set]( https://ai.facebook.com/blog/hateful-memes-challenge-and-data-set)
- **Paper:** [https://arxiv.org/pdf/2005.04790.pdf](https://arxiv.org/pdf/2005.04790.pdf)
- **Data:** [This link](https://drivendata-competition-fb-hateful-memes-data.s3.amazonaws.com/XjiOc5ycDBRRNwbhRlgH.zip?AWSAccessKeyId=AKIARVBOBDCY4MWEDJKS&Signature=DaUuGgZWUgDHzEPPbyJ2PhSJ56Q%3D&Expires=1612816874)
- **Motivation:** Including multi-modal datasets to 🤗 datasets.

I will be adding this dataset. It requires the user to sign an agreement on DrivenData. So, it will be used with a manual download.

The issue with this dataset is that the images are of different sizes. The image datasets added so far (CIFAR-10 and MNIST) have a uniform shape throughout.
So something like 
```python
 datasets.Array2D(shape=(28, 28), dtype="uint8")
```
won't work for the images. How would I add image features then? I checked `datasets/features.py` but couldn't figure out the appropriate class for this. I'm assuming I would want to avoid re-sizing at all since we want the user to be able to access the original images.

Also, in case I want to load only a subset of the data, since the actual data is around 8.8GB, how would that be possible?

Thanks,
Gunjan

Labels: name: dataset request

name: vision

I am not sure, but would `datasets.Sequence(datasets.Sequence(datasets.Sequence(datasets.Value("int")))` work?

Also, I found the information for loading only subsets of the data [here](https://github.com/huggingface/datasets/blob/master/docs/source/splits.rst).

Hi @lhoestq,

Request you to check this once.

Thanks,
Gunjan

Hi @gchhablani since Array2D doesn't support images of different sizes, I would suggest to store in the dataset the paths to the image file instead of the image data. This has the advantage of not decompressing the data (images are often compressed using jpeg, png etc.). Users can still apply `.map` to load the images if they want to. Though it would en up being Sequences features.

In the future we'll add support for ragged tensors for this case and update the relevant dataset with this feature.
Add Nuswide dataset

## Adding a Dataset
- **Name:** *NUSWIDE*
- **Description:** *[A Real-World Web Image Dataset from National University of Singapore](https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/NUS-WIDE.html)*
- **Paper:** *[here](https://lms.comp.nus.edu.sg/wp-content/uploads/2019/research/nuswide/nuswide-civr2009.pdf)*
- **Data:** *[here](https://github.com/wenting-zhao/nuswide)*
- **Motivation:** *This dataset is a benchmark in the Text Retrieval task.*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Labels: name: dataset request

name: vision
Add Hypersim dataset

## Adding a Dataset
- **Name:** Hypersim
- **Description:** photorealistic synthetic dataset for holistic indoor scene understanding
- **Paper:** *link to the dataset paper if available*
- **Data:** https://github.com/apple/ml-hypersim

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Labels: name: dataset request

name: vision
Add Microsoft Building Footprints dataset

## Adding a Dataset
- **Name:** Microsoft Building Footprints
- **Description:** With the goal to increase the coverage of building footprint data available as open data for OpenStreetMap and humanitarian efforts, we have released millions of building footprints as open data available to download free of charge.
- **Paper:** *link to the dataset paper if available*
- **Data:** https://www.microsoft.com/en-us/maps/building-footprints
- **Motivation:** this can be a useful dataset for researchers working on climate change adaptation, urban studies, geography, etc.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Reported by: @sashavor

Labels: name: dataset request

name: vision

Motivation: this can be a useful dataset for researchers working on climate change adaptation, urban studies, geography, etc. I'll see if I can figure out how to add it!
nested sequence feature won't encode example if the first item of the outside sequence is an empty list

## Describe the bug
As the title, nested sequence feature won't encode example if the first item of the outside sequence is an empty list.

## Steps to reproduce the bug
```python
from datasets import Features, Sequence, ClassLabel
features = Features({
    'x': Sequence(Sequence(ClassLabel(names=['a', 'b']))),
})
print(features.encode_batch({
    'x': [
        [['a'], ['b']],
        [[], ['b']],
    ]
}))
```

## Expected results
print `{'x': [[[0], [1]], [[], ['1']]]}`

## Actual results
print `{'x': [[[0], [1]], [[], ['b']]]}`

## Environment info
- `datasets` version: 1.15.1
- Platform: Linux-5.13.0-21-generic-x86_64-with-glibc2.34
- Python version: 3.9.7
- PyArrow version: 6.0.0

## Additional information
I think the issue stems from [here](https://github.com/huggingface/datasets/blob/8555197a3fe826e98bd0206c2d031c4488c53c5c/src/datasets/features/features.py#L847-L848).



Labels: name: bug

knock knock

Hi, thanks for reporting! I've linked a PR that should fix the issue.

I've checked the PR and it looks great, thanks a lot!
ZIP format inference does not work when files located in a dir inside the archive

## Describe the bug
When a zipped file contains archived files within a directory, the function `infer_module_for_data_files_in_archives` does not work.

It only works for files located in the root directory of the ZIP file.

## Steps to reproduce the bug
```python
infer_module_for_data_files_in_archives(["path/to/zip/file.zip"], False)
```

Labels: name: bug
New: Americas NLI dataset

This PR adds the [Americas NLI](https://arxiv.org/abs/2104.08726) dataset, extension of XNLI to 10 low-resource indigenous languages spoken in the Americas: Ashaninka, Aymara, Bribri, Guarani, Nahuatl, Otomi, Quechua, Raramuri, Shipibo-Konibo, and Wixarika.

One odd thing (not sure) is that I had to set

`datasets-cli dummy_data ./datasets/americas_nli/ --auto_generate --n_lines 7500`

`n_lines` very large to successfully generate the dummy files for all the subsets. Happy to get some guidance here.

Otherwise, I hope everything is in order :)

e: missed a step, onto fixing the tests
e2: there you go -- hope it's ok to have added more languages with their ISO codes to `languages.json`, need those tests to pass :laughing:
NonMatchingChecksumError for the CLUE:cluewsc2020, chid, c3 and tnews

Hi, it seems like there are updates in cluewsc2020, chid, c3 and tnews, since i could not load them due to the checksum error.


Seems like the issue still exists,:
`Downloading and preparing dataset clue/chid (download: 127.15 MiB, generated: 259.71 MiB, post-processed: Unknown size, total: 386.86 MiB) to /mnt/cache/tanhaochen/.cache/huggingface/datasets/clue/chid/1.0.0/e55b490cb7809dcd8db31b9a87119f2e2ec87cdc060da8a9ac070b070ca3e379...
Traceback (most recent call last):
  File "/mnt/cache/tanhaochen/PromptCLUE/test_datasets.py", line 3, in <module>
    cluewsc2020 = datasets.load_dataset("clue","chid")
  File "/mnt/cache/tanhaochen/dependencies/datasets/src/datasets/load.py", line 1667, in load_dataset
    builder_instance.download_and_prepare(
  File "/mnt/cache/tanhaochen/dependencies/datasets/src/datasets/builder.py", line 593, in download_and_prepare
    self._download_and_prepare(
  File "/mnt/cache/tanhaochen/dependencies/datasets/src/datasets/builder.py", line 663, in _download_and_prepare
    verify_checksums(
  File "/mnt/cache/tanhaochen/dependencies/datasets/src/datasets/utils/info_utils.py", line 40, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://storage.googleapis.com/cluebenchmark/tasks/chid_public.zip']
`

Hi,

the fix hasn't been merged yet (it should be merged early next week).
to_tf_dataset() refactor

This is the promised cleanup to `to_tf_dataset()` now that the course is out of the way! The main changes are:

- A collator is always required (there was way too much hackiness making things like labels work without it)
- Lots of cleanup and a lot of code moved to `_get_output_signature`
- Should now handle it gracefully when the data collator adds unexpected columns


Also, please don't merge yet - I need to make sure all the code samples and notebooks have a collate_fn specified, since we're removing the ability for this method to work without one!

Hi @lhoestq @mariosasko, the other PRs this was depending on in Transformers and huggingface/notebooks are now merged, so this is ready to go. Do you want to take one more look at it, or are you happy at this point?

The documentation for the method is fine, it doesn't need to be changed, but the tutorial notebook definitely looks a little out of date. Let me see what I can do!

@lhoestq I rewrote the last bit of the notebook - let me know what you think!

Cool thank you ! It's much nicer that what we had :)

I also spotted other things I'd like to update in the notebook (especially the beginning) but it can be fixed later
`GeneratorBasedBuilder` does not support `None` values

## Describe the bug
`GeneratorBasedBuilder` does not support `None` values.

## Steps to reproduce the bug
See [this repository](https://github.com/pavel-lexyr/huggingface-datasets-bug-reproduction) for minimal reproduction.

## Expected results
Dataset is initialized with a `None` value in the `value` column.

## Actual results
```
Traceback (most recent call last):
  File "main.py", line 3, in <module>
    datasets.load_dataset("./bad-data")
  File ".../datasets/load.py", line 1632, in load_dataset
    builder_instance.download_and_prepare(
  File ".../datasets/builder.py", line 607, in download_and_prepare
    self._download_and_prepare(
  File ".../datasets/builder.py", line 697, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File ".../datasets/builder.py", line 1103, in _prepare_split
    example = self.info.features.encode_example(record)
  File ".../datasets/features/features.py", line 1033, in encode_example
    return encode_nested_example(self, example)
  File ".../datasets/features/features.py", line 808, in encode_nested_example
    return {
  File ".../datasets/features/features.py", line 809, in <dictcomp>
    k: encode_nested_example(sub_schema, sub_obj) for k, (sub_schema, sub_obj) in utils.zip_dict(schema, obj)
  File ".../datasets/features/features.py", line 855, in encode_nested_example
    return schema.encode_example(obj)
  File ".../datasets/features/features.py", line 299, in encode_example
    return float(value)
TypeError: float() argument must be a string or a number, not 'NoneType'
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.15.1
- Platform: Linux-5.4.0-81-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 6.0.0

Labels: name: bug

Hi,

thanks for reporting and providing a minimal reproducible example. 

This line of the PR I've linked in our discussion on the Forum will add support for `None` values:
https://github.com/huggingface/datasets/blob/a53de01842aac65c66a49b2439e18fa93ff73ceb/src/datasets/features/features.py#L835

I expect that PR to be merged soon.
Common Voice Belarusian Dataset

## Adding a Dataset
- **Name:** *Common Voice Belarusian Dataset*
- **Description:** *[commonvoice.mozilla.org/be](https://commonvoice.mozilla.org/be)*
- **Data:** *[commonvoice.mozilla.org/be/datasets](https://commonvoice.mozilla.org/be/datasets)*
- **Motivation:** *It has more than 7GB of data, so it will be great to have it in this package so anyone can try to train something for Belarusian language.*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Labels: name: dataset request

name: speech
Adapt image datasets

This PR:
* adapts the ImageClassification template to use the new Image feature
* adapts the following datasets to use the new Image feature:
  * beans (+ fixes streaming)
  * cast_vs_dogs (+ fixes streaming)
  * cifar10
  * cifar100
  * fashion_mnist
  * mnist
  * head_qa

cc @nateraw


This PR can be merged after #3163 is merged (this PR is pretty big because I was working on the forked branch).

@lhoestq @albertvillanova Could you please take a look at the changes in `src/datasets/utils/streaming_download_manager.py`? These changes were required to support streaming of the `cats_vs_dogs` and the `beans` datasets.

The CI failures are due to the missing fields in the README files.

and thanks for adding support for Path.name and Path.parent for streaming :)
Add Speech Commands dataset

## Adding a Dataset
- **Name:** Speech commands
- **Description:** A Dataset for Limited-Vocabulary Speech Recognition
- **Paper:** https://arxiv.org/abs/1804.03209
- **Data:** https://www.tensorflow.org/datasets/catalog/speech_commands,  Available:
http://download.tensorflow.org/data/speech_commands_v0.02.tar.gz
- **Motivation:** Nice dataset for audio classification training

cc @anton-l 

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Labels: name: dataset request

name: speech

#self-assign
`None` converted to `"None"` when loading a dataset

## Describe the bug
When loading a dataset `None` values of the type `NoneType` are converted to `'None'` of the type `str`.

## Steps to reproduce the bug
```python
from datasets import load_dataset
qasper = load_dataset("qasper", split="train", download_mode="reuse_cache_if_exists")
print(qasper[60]["full_text"]["section_name"])
```

When installing version 1.1.40, the output is
`[None, 'Introduction', 'Benchmark Datasets', ...]`

When installing from the master branch, the output is
`['None', 'Introduction', 'Benchmark Datasets', ...]`

Notice how the first element was changed from `NoneType` to `str`.

## Expected results
`None` should stay as is.

## Actual results
`None` is converted to a string.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: master
- Platform: Linux-4.4.0-19041-Microsoft-x86_64-with-glibc2.17
- Python version: 3.8.10
- PyArrow version: 4.0.1


Labels: name: bug

Hi @eladsegal, thanks for reporting.

@mariosasko I saw you are already working on this, but maybe my comment will be useful to you.

All values are casted to their corresponding feature type (including `None` values). For example if the feature type is `Value("bool")`, `None` is casted to `False`.

It is true that strings were an exception, but this was recently fixed by @lhoestq (see #3158).

Thanks for reporting.

This is actually a breaking change that I think can cause issues when users preprocess their data. String columns used to be nullable. Maybe we can correct https://github.com/huggingface/datasets/pull/3158 to keep the None values and avoid this breaking change ?

EDIT: the other types (bool, int, etc) can also become nullable IMO

So what would be the best way to handle a feature that can have a null value in some of the instances? So far I used `None`.
Using the empty string won't be a good option, as it can be an actual value in the data and is not the same as not having a value at all.

Hi @eladsegal,

Use `None`. As @albertvillanova correctly pointed out, this change in conversion was introduced (by mistake) in #3158. To avoid it, install the earlier revision with:
```
pip install git+https://github.com/huggingface/datasets.git@8107844ec0e7add005db0585c772ee20adc01a5e
```

I'm making all the feature types nullable as we speak, and the fix will be merged probably early next week.

Hi @mariosasko, is there an estimation as to when this issue will be fixed?

https://github.com/huggingface/datasets/pull/3195 fixed it, we'll do a new release soon :)

For now feel free to install `datasets` from the master branch

Thanks, but unfortunately looks like it isn't fixed yet 😢 
[notebook for 1.14.0](https://colab.research.google.com/drive/1SV3sFXPJMWSQgbm4pr9Y1Q8OJ4JYKcDo?usp=sharing)
[notebook for master](https://colab.research.google.com/drive/145wDpuO74MmsuI0SVLcI1IswG6aHpyhi?usp=sharing)

Oh, sorry. I deleted the fix by accident when I was resolving a merge conflict. Let me fix this real quick.

Thank you, it works! 🎊
Getting configs names takes too long


## Steps to reproduce the bug
```python
from datasets import get_dataset_config_names
get_dataset_config_names("allenai/c4")
```

## Expected results

I would expect to get the answer quickly, at least in less than 10s

## Actual results

It takes about 45s on my environment

## Environment info

- `datasets` version: 1.16.1
- Platform: Linux-5.11.0-1022-aws-x86_64-with-glibc2.31
- Python version: 3.9.6
- PyArrow version: 4.0.1

Labels: name: bug

maybe related to https://github.com/huggingface/datasets/issues/2859


It looks like it's currently calling `HfFileSystem.ls()` ~8 times at the root and for each subdirectory:
- ""
- "en.noblocklist"
- "en.noclean"
- "en"
- "multilingual"
- "realnewslike"

Currently `ls` is slow because it iterates on all the files inside the repository.

An easy optimization would be to cache the result of each call to `ls`.
We can also optimize `ls` by using a tree structure per directory instead of a list of all the files.


ok