noamaanMulla-03/datasets-issues huggingface dataset:
added semeval18_emotion_classification dataset

I added the data set of SemEval 2018 Task 1 (Subtask 5) for emotion detection in three languages.

```
datasets-cli test datasets/semeval18_emotion_classification/ --save_infos --all_configs

RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_real_dataset_semeval18_emotion_classification
```
Both commands ran successfully.

I couldn't create the dummy data (the files are tsvs but have .txt ending, maybe that's the problem?) and therefore the test on the dummy data fails, maybe someone can help here.

I also formatted the code:
```
black --line-length 119 --target-version py36 datasets/semeval18_emotion_classification/
isort datasets/semeval18_emotion_classification/
flake8 datasets/semeval18_emotion_classification/
```
That's the publication for reference:

Mohammad, S., Bravo-Marquez, F., Salameh, M., & Kiritchenko, S. (2018). SemEval-2018 task 1: Affect in tweets. Proceedings of the 12th International Workshop on Semantic Evaluation, 1–17. https://doi.org/10.18653/v1/S18-1001


For training the multilabel classifier, I would combine the labels into a list, for example for the English dataset:

```
dfpre=pd.read_csv(path+"2018-E-c-En-train.txt",sep="\t")
dfpre['list'] = dfpre[dfpre.columns[2:]].values.tolist()
df = dfpre[['Tweet', 'list']].copy()
df.rename(columns={'list': 'labels'}, inplace=True)
```

Hi @maxpel , have you had a chance to take my comments into account ?

Let me know if you have questions or if I can help :)

Hi @lhoestq ! I did take your comments into account, changed the naming and tried to add dummy data (manually). I am not sure if the dummy data is correct, maybe you can take a look at that.
The model card is still missing as I am currently very busy.

Thanks ! The dummy data looks all good, good job :)

The CI error can be fixed by merging `master` into your branch
```bash
git fetch upstream
git merge upstream/master
```

Hi! I just added the model card and I did the merge you showed above. Should I then add and commit again? The CI error is still there right now.

@lhoestq Unfortunately, I discovered a problem with the test data sets on the competion page (train and dev is fine). They still contain NONE labels for each of the emotions, for example for English: http://saifmohammad.com/WebDocs/AIT-2018/AIT2018-DATA/AIT2018-TEST-DATA/semeval2018englishtestfiles/2018-E-c-En-test.zip
Luckily, a zip file with all data of the competition contains the correct labels also for the test set:
http://saifmohammad.com/WebDocs/AIT-2018/AIT2018-DATA/SemEval2018-Task1-all-data.zip
What's the best way to correct this?

Hi ! I think we can edit the sem_eval_2018_task_1.py file to use this URL instead, and maybe update the `os.path.join` calls to the new paths to the text data in the new ZIP file. Would you like to try to make this work ?
OpenSLR/83 is empty

## Describe the bug
As the summary says, openslr / SLR83 / train is empty.

The dataset returned after loading indicates there are **zero** rows. The correct number should be **17877**.
## Steps to reproduce the bug
```python
import datasets


datasets.load_dataset('openslr', 'SLR83')
```

## Expected results
```
DatasetDict({
    train: Dataset({
        features: ['path', 'audio', 'sentence'],
        num_rows: 17877
    })
})
```
## Actual results
```
DatasetDict({
    train: Dataset({
        features: ['path', 'audio', 'sentence'],
        num_rows: 0
    })
})
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.14.1.dev0 (master HEAD)
- Platform: Ubuntu 20.04
- Python version: 3.7.10
- PyArrow version: 3.0.0


Labels: name: bug

Hi @tyrius02, thanks for reporting. I see you self-assigned this issue: are you working on this?

@albertvillanova Yes. Figured I introduced the broken config, I should fix it too.

I've got it working, but I'm struggling with one of the tests. I've started a PR so I/we can work through it.

Looks like the tests all passed on the PR.
Make inspect.get_dataset_config_names always return a non-empty list

Make all named configs cases, so that no special unnamed config case needs to be handled differently.

Fix #3135.


This PR is already working (although not very beautiful; see below): the idea was to have the `DatasetModule.builder_kwargs` accessible from the `builder_cls`, so that this can generate the default builder config (at the class level, without requiring the builder to be instantiated).

I have a plan for a follow-up refactoring (same functionality, better implementation, much nicer), but I think we could already merge this, so that @severo can test it in the datasets previewer and report any potential issues.

Yes @lhoestq you are completely right. Indeed I was exclusively using `builder_cls.kwargs` to get the community dataset `name` (nothing else): "lhoestq___demo1"

See et: https://github.com/huggingface/datasets/pull/3159/files#diff-f933ce41f71c6c0d1ce658e27de62cbe0b45d777e9e68056dd012ac3eb9324f7R413-R415

In your example, the `name` I was getting from `builder_cls.kwargs` was:
```python
{"name": "lhoestq___demo1",...}
```

I'm going to refactor all the approach... as I only need the name for this specific case ;)

I think this makes more sense now, @lhoestq @severo 😅 

It works well, thanks!
Dataset viewer issues

I was looking through the dataset viewer and I like it a lot. Version numbers, citation information, everything's there! I've spotted a few issues/bugs though:

- the URL is still under `nlp`, perhaps an alias for `datasets` can be made
- when I remove a **feature** (and the feature list is empty), I get an error. This is probably expected, but perhaps a better error message can be shown to the user

```bash
IndexError: list index out of range
Traceback:
File "/home/sasha/streamlit/lib/streamlit/ScriptRunner.py", line 322, in _run_script
    exec(code, module.__dict__)
File "/home/sasha/nlp-viewer/run.py", line 316, in <module>
    st.table(style)
File "/home/sasha/streamlit/lib/streamlit/DeltaGenerator.py", line 122, in wrapped_method
    return dg._enqueue_new_element_delta(marshall_element, delta_type, last_index)
File "/home/sasha/streamlit/lib/streamlit/DeltaGenerator.py", line 367, in _enqueue_new_element_delta
    rv = marshall_element(msg.delta.new_element)
File "/home/sasha/streamlit/lib/streamlit/DeltaGenerator.py", line 120, in marshall_element
    return method(dg, element, *args, **kwargs)
File "/home/sasha/streamlit/lib/streamlit/DeltaGenerator.py", line 2944, in table
    data_frame_proto.marshall_data_frame(data, element.table)
File "/home/sasha/streamlit/lib/streamlit/elements/data_frame_proto.py", line 54, in marshall_data_frame
    _marshall_styles(proto_df.style, df, styler)
File "/home/sasha/streamlit/lib/streamlit/elements/data_frame_proto.py", line 73, in _marshall_styles
    translated_style = styler._translate()
File "/home/sasha/.local/share/virtualenvs/lib-ogGKnCK_/lib/python3.7/site-packages/pandas/io/formats/style.py", line 351, in _translate
    * (len(clabels[0]) - len(hidden_columns))
```

- there seems to be **an encoding issue** in the default view, the dataset examples are shown as raw monospace text, without a decent encoding. That makes it hard to read for languages that use a lot of special characters. Take for instance the [cs-en WMT19 set](https://huggingface.co/nlp/viewer/?dataset=wmt19&config=cs-en). This problem goes away when you enable "List view", because then some syntax highlighteris used, and the special characters are coded correctly.


Labels: name: nlp-viewer

Thanks for reporting !
cc @srush for the empty feature list issue and the encoding issue
cc @julien-c maybe we can update the url and just have a redirection from the old url to the new one ?

Ok, I redirected on our side to a new url. ⚠️ @srush: if you update the Streamlit config too to `/datasets/viewer`, let me know because I'll need to change our nginx config at the same time

9

‏⠀‏‏‏⠀‏‏‏⠀ ‏⠀ 

‏⠀‏‏‏⠀‏‏‏⠀ ‏⠀
Multiprocessing filter/map (tests) not working on Windows

While running the tests, I found that the multiprocessing examples fail on Windows, or rather they do not complete: they cause a deadlock. I haven't dug deep into it, but they do not seem to work as-is. I currently have no time to tests this in detail but at least the tests seem not to run correctly (deadlocking).

## Steps to reproduce the bug
```shell
pytest tests/test_arrow_dataset.py -k "test_filter_multiprocessing"
pytest tests/test_arrow_dataset.py -k "test_map_multiprocessing"
```

## Expected results
The functionality to work on all platforms.

## Actual results
Deadlock.

## Environment info
- `datasets` version: 1.14.1.dev0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.9.2, also tested with 3.7.9
- PyArrow version: 4.0.1



Labels: name: bug
Don't memoize strings when hashing since two identical strings may have different python ids

When hashing an object that has several times the same string, the hashing could return a different hash if the identical strings share the same python `id()` or not.

Here is an example code that shows how the issue can affect the caching:
```python
import json
import pyarrow as pa
from datasets.features import Features
from datasets.fingerprint import Hasher

schema = pa.schema([pa.field("some_string", pa.string()), pa.field("another_string", pa.string())])
features_from_schema = Features.from_arrow_schema(schema)
Hasher.hash(features_from_schema) # dffa9dca9a73fd8c

features_dict = json.loads('{"some_string": {"dtype": "string", "id": null, "_type": "Value"}, "another_string": {"dtype": "string", "id": null, "_type": "Value"}}')
features_from_json = Features.from_dict(features_dict)
Hasher.hash(features_from_json) # 3812e76b15e6420e

features_from_schema == features_from_json # True
```

This is because in `features_dict`, some strings like "dtype" are repeated but don't share the same id, contrary to the ones in `features_from_schema`.

I fixed that by disabling memoization for strings.

This could be optimized in the future by implementing a smarter memoization with a special handling for strings.


This change slows down the hash computation a little bit but from my tests it doesn't look too impactful. So I think it's fine to merge this.
Faiss _is_ available on Windows

In the setup file, I find the following:

https://github.com/huggingface/datasets/blob/87c71b9c29a40958973004910f97e4892559dfed/setup.py#L171

However, FAISS does install perfectly fine on Windows on my system. You can also confirm this on the [PyPi page](https://pypi.org/project/faiss-cpu/#files), where Windows wheels are available. Maybe this was true for older versions? For current versions, this can be removed I think.

(This isn't really a bug but didn't know how else to tag.)

If you agree I can do a quick PR and remove that line.


Sure, feel free to open a PR.
Re-add faiss to windows testing suite

In recent versions, `faiss-cpu` seems to be available for Windows as well. See the [PyPi page](https://pypi.org/project/faiss-cpu/#files) to confirm. We can therefore included it for Windows in the setup file.

At first tests didn't pass due to problems with permissions as caused by `NamedTemporaryFile` on Windows. This built-in library is notoriously poor in playing nice on Windows. The required change isn't pretty, but it works. First set `delete=False` to not automatically try to delete the file on `exit`. Then, manually delete the file with `unlink`. It's weird, I know, but it works.

```python
with tempfile.NamedTemporaryFile(delete=False) as tmp_file:
    # do stuff
os.unlink(tmp_file.name)
```

closes #3150
combination of shuffle and filter results in a bug

## Describe the bug
Hi,
I would like to shuffle a dataset, then filter it based on each existing label. however, the combination of `filter`, `shuffle` seems to results in a bug. In the minimal example below, as you see in the filtered results, the filtered labels are not unique, meaning filter has not worked. Any suggestions as a temporary fix is appreciated @lhoestq.

Thanks.
 Best regards
Rabeeh 

## Steps to reproduce the bug
```python
import numpy as np
import datasets 

datasets = datasets.load_dataset('super_glue', 'rte', script_version="master")
shuffled_data = datasets["train"].shuffle(seed=42)
for label in range(2):
    print("label ", label)
    data = shuffled_data.filter(lambda example: int(example['label']) == label)
    print("length ", len(data), np.unique(data['label']))
```

## Expected results
Filtering per label, should only return the data with that specific label.

## Actual results
As you can see, filtered data per label, has still two labels of [0, 1]
```
label  0
length  1249 [0 1]
label  1
length  1241 [0 1]
```
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1 
- Platform: linux 
- Python version: 3.7.11 
- PyArrow version: 5.0.0 


Labels: name: bug

I cannot reproduce this on master and pyarrow==4.0.1.


Hi ! There was a regression in `datasets` 1.12 that introduced this bug. It has been fixed in #3019 in 1.13

Can you try to update `datasets` and try again ?

Thanks a lot, fixes with 1.13
fix label mapping

Fixing label mapping for hlgd.
0 correponds to same event and 1 corresponds to different event
<img width="642" alt="Capture d’écran 2021-10-29 à 10 39 58 AM" src="https://user-images.githubusercontent.com/16107619/139454810-1f225e3d-ad48-44a8-b8b1-9205c9533839.png">
<img width="638" alt="Capture d’écran 2021-10-29 à 10 40 09 AM" src="https://user-images.githubusercontent.com/16107619/139454813-93066a3c-7d33-4f56-b133-2f1a7661e438.png">
nt


heck, test failings. moving to draft. will come back to this later today hopefully

Thanks for fixing this :)
I just updated the dataset_infos.json and added the missing `pretty_name` tag to the dataset card

thank you @lhoestq! running around as always it felt through as a lower priority...
Segmentation fault when loading datasets from file

## Describe the bug
Custom dataset loading sometimes segfaults and kills the process if chunks contain a variety of features/

## Steps to reproduce the bug

Download an example file:
```
wget https://gist.githubusercontent.com/TevenLeScao/11e2184394b3fa47d693de2550942c6b/raw/4232704d08fbfcaf93e5b51def9e5051507651ad/tiny_kelm.jsonl
```
Then in Python:
```
import datasets
tiny_kelm = datasets.load_dataset("json", data_files="tiny_kelm.jsonl", chunksize=100000)
```

## Expected results
a `tiny_kelm` functional dataset

## Actual results
☠️ `Segmentation fault (core dumped)` ☠️

## Environment info
- `datasets` version: 1.14.0
- Platform: Linux-5.11.0-38-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 5.0.0

Labels: name: bug

Hi ! I created an issue on Arrow's JIRA after making a minimum reproducible example

https://issues.apache.org/jira/browse/ARROW-14439

```python
import io

import pyarrow.json as paj

batch = b'{"a": [], "b": 1}\n{"b": 1}'
block_size = 12

paj.read_json(
    io.BytesIO(batch), read_options=paj.ReadOptions(block_size=block_size)
)
```

I don't see a way to workaround this properly now without hurting the performance of the JSON loader significantly though

The issue has been fixed in pyarrow 6.0.0, please update pyarrow :)

The issue was due to missing fields in the JSON data of type list. Now it's working fine and missing list fields are replaced with empty lists
More efficient nested features encoding

Nested encoding of features wastes a lot of time on operations which are effectively doing nothing when lists are used.
For example, if in the input we have a list of integers, `encoded_nested_example` will iterate over it and apply `encoded_nested_example` on every element even though it just return the int as is.

A similar issue is handled at an earlier stage when casting pytorch/tensorflow/pandas objects to python lists/numpy arrays:
https://github.com/huggingface/datasets/blob/c98c23c4260edadab00f997d1a5d66b7f2e93ce9/src/datasets/features/features.py#L149-L156
https://github.com/huggingface/datasets/blob/c98c23c4260edadab00f997d1a5d66b7f2e93ce9/src/datasets/features/features.py#L212-L228

In this pull request I suggest to use the same approach in `encoded_nested_example`.
In my setup there was a major speedup with this change: loading the data was at least x4 faster. 


@lhoestq @albertvillanova @mariosasko
Can you please check this out?

Thanks, done!
[save_to_disk] Using `select()` followed by `save_to_disk` saves complete dataset making it hard to create dummy dataset

## Describe the bug

When only keeping a dummy size of a dataset (say the first 100 samples), and then saving it  to disk to upload it in the following to the hub for easy demo/use - not just the small dataset is saved but the whole dataset with an indices file. The problem with this is that the dataset is still very big.

## Steps to reproduce the bug

E.g. run the following:

```python
from datasets import load_dataset, save_to_disk

nlp = load_dataset("glue", "mnli", split="train")
nlp.save_to_disk("full")

nlp = nlp.select(range(100))
nlp.save_to_disk("dummy")
```

Now one can see that both `"dummy"` and `"full"` have the same size. This shouldn't be the case IMO.

## Expected results

IMO `"dummy"` should be much smaller so that one can easily play around with the dataset on the hub.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->

- `datasets` version: 1.12.2.dev0
- Platform: Linux-5.11.0-34-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyArrow version: 5.0.0



Labels: name: bug

Hi,

the `save_to_disk` docstring explains that `flatten_indices` has to be called on a dataset before saving it to save only the shard/slice of the dataset.

That works! Thansk!

Might be worth doing that automatically actually in case the `save_to_disk` is called on a dataset that has an indices mapping :-)

I agree with @patrickvonplaten: this issue is reported recurrently, so better if we implement the `.flatten_indices()` automatically?

That would be great indeed - I don't really see a use case where one would not like to call `.flatten_indices()` before calling `save_to_disk`

+1 on this !
RONEC v2

Hi, as we've recently finished with the new RONEC (Romanian Named Entity Corpus), we'd like to update the dataset here as well. It's actually essential as links to V1 are no longer valid. 

In reality we'd like to replace completely v1, as v2 is a full re-annotation of v1 with additional data (up to 2x size vs v1).

I've run the make style and all the dummy and real data test, and they passed.

I hope it's okay to merge the new RONEC v2 in the datasets. 

Thanks!


@lhoestq Thanks for the review. I totally understand what you are saying. Normally, I would definitely agree with you, but in this particular case, the quality of v1 is poor, and the dataset itself is small (at the time we created v1 it was the only RO NER dataset, and its size was limited by the available resources). 

This is why we worked to build a larger one, with much better inter-annotator agreement. Fact is, models trained on v1 will be of very low quality and I would not recommend to anybody to use/do that. That's why I'd strongly suggest we replace v1 with v2, and kindof make v1 vanish :)  

What do you think? If you insist on having v1 accessible, I'll add the required code. Thanks!



Ok I see ! I think it's fine then, no need to re-add V1
Add docs for `to_tf_dataset`

This PR adds some documentation for new features released in v1.13.0, with the main addition being `to_tf_dataset`:

- Show how to use `to_tf_dataset` in the tutorial, and move `set_format(type='tensorflow'...)` to the Process section (let me know if I'm missing anything @Rocketknight1 😅).
- Add an example for loading dataset from multiple zipped CSV files to the Load section.
- Add an example for removing columns for an `IterableDataset`.
- Add graphic for visualizing streaming.

Labels: name: documentation

This looks great, thank you!

Thanks !

For some reason the new GIF is 6MB, which is a bit heavy for an image on a website. The previous one was around 200KB though which is perfect. For a good experience we usually expect images to be less than 500KB - otherwise for users with poor connection it takes too long to load. Could you try to reduce its size ? Than I think we can merge :)
CI error: Another metric with the same name already exists in Keras 2.7.0

## Describe the bug
Release of TensorFlow 2.7.0 contains an incompatibility with Keras. See:
- keras-team/keras#15579

This breaks our CI test suite: https://app.circleci.com/pipelines/github/huggingface/datasets/8493/workflows/055c7ae2-43bc-49b4-9f11-8fc71f35a25c/jobs/52363


Labels: name: bug
Fix disable_nullable default value to False

Currently the `disable_nullable` parameter is not consistent across all dataset transforms. For example it is `False` in `map` but `True` in `flatten_indices`.

This creates unexpected behaviors like this
```python
from datasets import Dataset, concatenate_datasets

d1 = Dataset.from_dict({"a": [0, 1, 2, 3]})
d2 = d1.filter(lambda x: x["a"] < 2).flatten_indices()
d1.data.schema == d2.data.schema  # False
```
This can cause issues when concatenating datasets for example.

For consistency I set `disable_nullable` to `False` in `flatten_indices` and I fixed some docstrings

cc @SBrandeis
Updated: DaNE - updated URL for download

It seems that DaNLP has updated their download URLs and it therefore also needs to be updated in here...


Actually it looks like the old URL is still working, and it's also the one that is mentioned in https://github.com/alexandrainst/danlp/blob/master/docs/docs/datasets.md

What makes you think we should use the new URL ?

@lhoestq Sorry! I might have jumped to conclusions a bit too fast here... 

I was working in Google Colab and got an error that it was unable to use the URL. I then forked the project, updated the URL, ran it locally and it worked. I therefore assumed that my URL update fixed the issue, however, I see now that it might rather be a Google Colab issue... 

Still - this seems to be the official URL for downloading the dataset, and I think that it will be most beneficial to use. :-) 

It looks like they're using these new urls for their new datasets. Maybe let's change to the new URL in case the old one stops working at one point. Thanks
Pin version exclusion for tensorflow incompatible with keras

Once `tensorflow` version 2.6.2 is released:
- https://github.com/tensorflow/tensorflow/commit/c1867f3bfdd1042f694df7a9870be51ba80543cb
- https://pypi.org/project/tensorflow/2.6.2/

with the patch:
- tensorflow/tensorflow#52927

we can remove the temporary fix we introduced in:
- #3208

Fix #3209.
Better error msg if `len(predictions)` doesn't match `len(references)` in metrics

Improve the error message in `Metric.add_batch` if `len(predictions)` doesn't match `len(references)`.

cc: @BramVanroy (feel free to test this code on your examples and review this PR)


Can't test this now but it may be a good improvement indeed.

I added a function, but it only works with the `list` type. For arrays/tensors, we delegate formatting to the frameworks.
Use huggingface_hub.HfApi to list datasets/metrics

Delete `datasets.inspect.HfApi` and use `huggingface_hub.HfApi` instead.

WIP until https://github.com/huggingface/huggingface_hub/pull/429 is merged, then wait for the new release of  `huggingface_hub`, update the `huggingface_hub` version in `setup.py` and merge this PR.

cc: @lhoestq