SergeiGKS/huggingface-dataset-issues huggingface dataset:
Issue: AutoTokenizer hash value got change after datasets.map

Comment:
Thanks a lot for the explanation !
I think if we set these 2 dicts at initialization time it would be amazing already

Shall we open an issue in `transformers` to ask for these dictionaries to be set when the tokenizer is instantiated ?

> Edit: Another option would be to override the default hash function, but I don't know if there's a sound implementation that could fit.

In `datasets` we can easily have custom hashing for objects of the other HF libraries if we want. For example we ignore the cache some tokenizers have. However in this specific case it touches parameters that may change the behavior of the tokenizer itself. I'm not sure the logic that determines how a tokenizer behaves should be in `datasets`
Issue: AutoTokenizer hash value got change after datasets.map

Comment:
A hack we could have in the `datasets` lib would be to call the tokenizer before hashing it in order to set all its parameters correctly - but it sounds a lot like a hack and I'm not sure this can work in the long run
Issue: AutoTokenizer hash value got change after datasets.map

Comment:
Fully agree with everything you said. 

I think the best course of action is creating an issue in `transformers`. I can start the work on this.
I think the code changes are fairly simple. Making a sound test + not breaking other stuff might be different :D
Issue: AutoTokenizer hash value got change after datasets.map

Comment:
It should be noted that this problem also occurs in other AutoClasses, such as AutoFeatureExtractor, so I don't think handling it in Datasets is a long-term practice either.
Issue: AutoTokenizer hash value got change after datasets.map

Comment:
> I think the best course of action is creating an issue in `transformers`. I can start the work on this.

@Narsil Hi, I reopen this issue in `transformers` https://github.com/huggingface/transformers/issues/14931
Issue: AutoTokenizer hash value got change after datasets.map

Comment:
Here is @Narsil comment from https://github.com/huggingface/transformers/issues/14931#issuecomment-1074981569
> # TL;DR
> Call the function once on a dummy example beforehand will fix it.
> 
> ```python
> tokenizer("Some", "test", truncation=True)
> ```
> 
> # Long answer
> If I remember the last status, it's hard doing anything, since the call itself
> 
> ```python
> tokenizer(example["sentence1"], example["sentence2"], truncation=True)
> ```
> 
> will modify the tokenizer. It's the `truncation=True` that modifies the tokenizer to put it into truncation mode if you will. Calling the tokenizer once with that argument would fix the cache.
> 
> Finding a fix that :
> 
> * Doesn't imply a huge chunk of work on `tokenizers` (with potential loss of performance, and breaking backward compatibility)
> * Doesn't imply `datasets` running a first pass of the loop
> * Doesn't imply `datasets` looking at the map function itself
> * Uses a sound `hash` for this object in `datasets`.
> 
> is IIRC impossible for this use case.
> 
> I can explain a bit more why the first option is not desirable.
> 
> In order to "fix" this for tokenizers, we would need to make `tokenizer(..)` purely without side effects. This means that the "options" of tokenization (like `truncation` and `padding` at least) would have
Issue: [TypeError: Couldn't cast array of type] Cannot load dataset in v1.18

Comment:
Hi @lewtun!
 
This one was tricky to debug. Initially, I tought there is a bug in the recently-added (by @lhoestq ) `cast_array_to_feature` function because `git bisect` points to the https://github.com/huggingface/datasets/commit/6ca96c707502e0689f9b58d94f46d871fa5a3c9c commit. Then, I noticed that the feature tpye of the `dialogue` field is `list`, which explains why you didn't get an error in earlier versions. Is there a specific reason why you use `list` instead of `Sequence` in the script? Maybe to avoid turning list of dicts to dicts of lists as it's done by `Sequence` for compatibility with TFDS or for performance reasons? If the field was `Sequence`, you would get an error in `encode_nested_example` because **the scripts yields some additional (nested) columns which are not specified in the `features` dictionary**. Previously, these additional columns would've been ignored by PyArrow (1), but now we have a check for them (2).
(1) See PyArrow behavior:
```
>>> pa.array([{"a": 2, "b": 3}], type=pa.struct({"a": pa.int32()})) # pyarrow ignores the extra column
-- is_valid: all not null
-- child 0 type: int32
  [
    2
  ]
 ```

(2) Check:
https://github.com/huggingface/datasets/blob/4c417d52def6e20359ca16c6723e0a2855e5c3fd/src/datasets/table.py#L1059

The fix is very simple: just add the missing columns to the _EMPTY_BELIEF_STATE list:
```python
_EMPTY_BELIEF_STATE.extend(['通用-产品类别', '火车-舱位档次', '通用-系列', '通用-价格区间', '通用-品牌'])
```
Issue: [TypeError: Couldn't cast array of type] Cannot load dataset in v1.18

Comment:
Hey @mariosasko, thank you so much for figuring this one out - it certainly looks like a tricky bug 😱 ! I don't think there's a specific reason to use `list` instead of `Sequence` with the script, but I'll let the dataset creators know to see if your suggestion is acceptable.

Thank you again!
Issue: [TypeError: Couldn't cast array of type] Cannot load dataset in v1.18

Comment:
Thanks, this was indeed the fix! Would it make sense to produce a more informative error message in such cases? 

The issue can be closed.
Issue: Dataset.shuffle(seed=None) gives fixed row permutation

Comment:
I'm not sure if this is expected behavior.

Am I supposed to work with a copy of the dataset, i.e. `shuffled_dataset = data.shuffle(seed=None)`?

```diff
import datasets

# Some toy example
data = datasets.Dataset.from_dict(
    {"feature": [1, 2, 3, 4, 5], "label": ["a", "b", "c", "d", "e"]}
)

+shuffled_data = data.shuffle(seed=None)

# Doesn't work as expected
print("Shuffle dataset")
for _ in range(3):
+    shuffled_data = shuffled_data.shuffle(seed=None)
+    print(shuffled_data[:])
-    print(data.shuffle(seed=None)[:])

# This seems to work with pandas
print("\nShuffle via pandas")
for _ in range(3):
    df = data.to_pandas().sample(frac=1.0)
    print(datasets.Dataset.from_pandas(df, preserve_index=False)[:])

```

or provide a `generator` instead?

```diff
import datasets
+from numpy.random import default_rng

# Some toy example
data = datasets.Dataset.from_dict(
    {"feature": [1, 2, 3, 4, 5], "label": ["a", "b", "c", "d", "e"]}
)

+rng = default_rng()

# Doesn't work as expected
print("Shuffle dataset")
for _ in range(3):
+   print(data.shuffle(generator=rng)[:])
-   print(data.shuffle(seed=None)[:])

# This seems to work with pandas
print("\nShuffle via pandas")
for _ in range(3):
    df = data.to_pandas().sample(frac=1.0)
    print(datasets.Dataset.from_pandas(df, preserve_index=False)[:])

```
Issue: Dataset.shuffle(seed=None) gives fixed row permutation

Comment:
Hi! Thanks for reporting! Yes, this is not expected behavior. I've opened a PR with the fix.
Issue: Adding CC-100: Monolingual Datasets from Web Crawl Data (Datasets links are invalid)

Comment:
Hi @AnzorGozalishvili,

Maybe their site was temporarily down, but it seems to work fine now.

Could you please try again and confirm if the problem persists?
Issue: Adding CC-100: Monolingual Datasets from Web Crawl Data (Datasets links are invalid)

Comment:
Hi @albertvillanova 
I checked and it works. 
It seems that it was really temporarily down.
Thanks!
Issue: Add a metadata field for when source data was produced 

Comment:
A question to the datasets maintainers: is there a policy about how the set of allowed metadata fields is maintained and expanded?

Metadata are very important, but defining the standard is always a struggle between allowing exhaustivity without being too complex. Archivists have Dublin Core, open data has https://frictionlessdata.io/, geo has ISO 19139 and INSPIRE, etc. and it's always a mess! I'm not sure we want to dig too much into it, but I'm curious to know if there has been some work on the metadata standard.
Issue: Add a metadata field for when source data was produced 

Comment:
> Metadata are very important, but defining the standard is always a struggle between allowing exhaustivity without being too complex. Archivists have Dublin Core, open data has [frictionlessdata.io](https://frictionlessdata.io/), geo has ISO 19139 and INSPIRE, etc. and it's always a mess! I'm not sure we want to dig too much into it, but I'm curious to know if there has been some work on the metadata standard.


I thought this is a potential issue with adding this field since it might be hard to define what is general enough to be useful for most data vs what becomes very domain-specific. Potentially adding one extra field leads to more and more fields in the future. 

Another issue is that there are some metadata standards around data i.e. [datacite](https://schema.datacite.org/meta/kernel-4.4/), but not many aimed explicitly at ML data afaik. Some of the discussions around metadata for ML are also more focused on versioning/managing data in production environments. My thinking is that here, some reference to the time of production would also often be tracked/relevant, i.e. for triggering model training, so having this information available in the hub would also help address this use case.
Issue: Add a metadata field for when source data was produced 

Comment:
Also related: the [Data Catalog Vocabulary - DCAT](https://www.w3.org/TR/vocab-dcat/) standard will be discussed in a new Working Group at the W3C: https://www.w3.org/2022/06/dx-wg-charter.html
Issue: Consider adding `ipywidgets` as a dependency.

Comment:
Hi! We use `tqdm` to display progress bars, so I suggest you open this issue in their repo.
Issue: Consider adding `ipywidgets` as a dependency.

Comment:
It depends on how you use `tqdm`, no? 

Doesn't this library import via; 

```
from tqdm.notebook import tqdm
```
Issue: Consider adding `ipywidgets` as a dependency.

Comment:
Hi! Sorry for the late reply. We import `tqdm` as `from tqdm.auto import tqdm`, which should be equal to `from tqdm.notebook import tqdm` in Jupyter.
Issue: Consider adding `ipywidgets` as a dependency.

Comment:
Any objection if I make a PR that checks if the widgets library is installed beforehand?
Issue: TIMIT Dataset not working with GPU

Comment:
Hi ! I think you should avoid calling `timit_train['audio']`. Indeed by doing so you're **loading all the audio column in memory**. This is problematic in your case because the TIMIT dataset is huge.

If you want to access the audio data of some samples, you should do this instead `timit_train[:10]["train"]` for example.

Other than that, I'm not sure why you get a `TypeError: string indices must be integers`, do you have a code snippet that reproduces the issue that you can share here ?
Issue: TIMIT Dataset not working with GPU

Comment:
I get the same error when I try to do `timit_train[0]` or really any indexing into the whole thing. 

Really, that IS the code snippet that reproduces the issue. If you index into other fields like 'file' or whatever, it works. As soon as one of the fields you're looking into is 'audio', you get that issue. It's a weird issue and I suspect it's Sagemaker/environment related, maybe the mix of libraries and dependencies are not good. 


Example code snippet with issue. 
```python
from datasets import load_dataset

timit_train = load_dataset('timit_asr', split='train')
print(timit_train[0])
```
Issue: TIMIT Dataset not working with GPU

Comment:
Ok I see ! From the error you got, it looks like the `value` encoded in the arrow file of the TIMIT dataset you loaded is a string instead of a dictionary with keys "path" and "bytes" but we don't support this since 1.18

Can you try regenerating the dataset with `load_dataset('timit_asr', download_mode="force_redownload")` please ? I think it should fix the issue.
Issue: Dataset BnL Historical Newspapers does not work in streaming mode

Comment:
@albertvillanova let me know if there is anything I can do to help with this. I had a quick look at the code again and though I could try the following changes:
- use `download` instead of `download_and_extract`
https://github.com/huggingface/datasets/blob/d3d339fb86d378f4cb3c5d1de423315c07a466c6/datasets/bnl_newspapers/bnl_newspapers.py#L136
- swith to using `iter_archive` to loop through downloaded data to replace
https://github.com/huggingface/datasets/blob/d3d339fb86d378f4cb3c5d1de423315c07a466c6/datasets/bnl_newspapers/bnl_newspapers.py#L159

Let me know if it's useful for me to try and make those changes.
Issue: Dataset BnL Historical Newspapers does not work in streaming mode

Comment:
Thanks @davanstrien.

I have already been working on it so that it can be used in the BigScience workshop.

I agree that the `rglob()` is not efficient in this case.

I tried different solutions without success:
- `iter_archive` cannot be used in this case because it does not support ZIP files yet

Finally I have used `iter_files()`.
Issue: Dataset BnL Historical Newspapers does not work in streaming mode

Comment:
I see this is fixed now 🙂.  I also picked up a few other tips from your redactors so hopefully my next attempts will support streaming from the start.
Issue: Files not updating in dataset viewer

Comment:
Yes. The jobs queue is full right now, following an upgrade... Back to normality in the next hours hopefully. I'll look at your datasets to be sure the dataset viewer works as expected on them.
Issue: Add support for continuous metrics (RMSE, MAE)

Comment:
Hey @ck37 

You can always use a custom metric as explained [in this guide from HF](https://huggingface.co/docs/datasets/master/loading_metrics.html#using-a-custom-metric-script).

If this issue needs to be contributed to (for enhancing the metric API) I think [this link](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.mean_absolute_error.html) would be helpful for the `MAE` metric.
Issue: Add support for continuous metrics (RMSE, MAE)

Comment:
You can use a local metric script just by providing its path instead of the usual shortcut name
Issue: audio column not saved correctly after resampling

Comment:
Hi ! We just released a new version of `datasets` that should fix this.

I tested resampling and using save/load_from_disk afterwards and it seems to be fixed now
Issue: audio column not saved correctly after resampling

Comment:
Hi @lhoestq, 

Just tested the latest datasets version, and confirming that this is fixed for me. 

Thanks!
Issue: Readme info not being parsed to show on Dataset card page

Comment:
i suspect a markdown parsing error, @severo do you want to take a quick look at it when you have some time?
Issue: Readme info not being parsed to show on Dataset card page

Comment:
# Problem
The issue seems to coming from the front matter of the README
```---
annotations_creators:
- no-annotation
language_creators:
- machine-generated
languages:
- 'ca'
- 'de'
licenses:
- cc-by-4.0
multilinguality:
- translation
pretty_name: Catalan-German aligned corpora to train NMT systems.
size_categories:
- "1M<n<10M"            
source_datasets:
- extended|tilde_model
task_categories:
- machine-translation
task_ids:
- machine-translation
---
``` 
# Solution
The fix is to correctly style the README as explained [here](https://huggingface.co/docs/datasets/v1.12.0/dataset_card.html). I have also correctly parsed the font matter as shown below:
```
---
annotations_creators: []
language_creators: [machine-generated]
languages: ['ca', 'de']
licenses: []
multilinguality:
- multilingual
pretty_name: 'Catalan-German aligned corpora to train NMT systems.'
size_categories: 
- 1M<n<10M
source_datasets: ['extended|tilde_model']
task_categories: ['machine-translation']
task_ids: ['machine-translation']
---
```
You can find the README for a sample dataset [here](https://huggingface.co/datasets/ritwikraha/Test)
Issue: Readme info not being parsed to show on Dataset card page

Comment:
Thank you. It finally worked implementing your changes and leaving a white line between title and text in the description.
Issue: ERROR: File "setup.py" or "setup.cfg" not found. Directory cannot be installed in editable mode: /content

Comment:
Hi! The `cd` command in Jupyer/Colab needs to start with `%`, so this should work:
```
!git clone https://github.com/huggingface/datasets.git
%cd datasets
!pip install -e ".[streaming]"
```
Issue: Loss of cast `Image` feature on certain dataset method

Comment:
Hi! Thanks for reporting! The issue with `cast_column` should be fixed by #3575 and after we merge that PR I'll start working on the `push_to_hub` support for the `Image`/`Audio` feature.
Issue: Loss of cast `Image` feature on certain dataset method

Comment:
> Hi! Thanks for reporting! The issue with `cast_column` should be fixed by #3575 and after we merge that PR I'll start working on the `push_to_hub` support for the `Image`/`Audio` feature.

Thanks, I'll keep an eye out for #3575 getting merged. I managed to use `push_to_hub` sucesfully with images when they were loaded via `map` - something like `ds.map(lambda example: {"img": load_image_function(example['fname']})`, this only pushed  the images to the hub if the `load_image_function` return a PIL Image without the filename attribute though. I guess this might often be the prefered behaviour though.
Issue: Loss of cast `Image` feature on certain dataset method

Comment:
Hi ! We merged the PR and did a release of `datasets` that includes the changes. Can you try updating `datasets` and try again ?
Issue: Loss of cast `Image` feature on certain dataset method

Comment:
> Hi ! We merged the PR and did a release of `datasets` that includes the changes. Can you try updating `datasets` and try again ?

Thanks for checking. There is no longer an error when calling `select` but it appears the cast value isn't preserved. Before `select`

```python
dataset.features
{'url': Image(id=None)}
```

after select:
```
{'url': Value(dtype='string', id=None)}
```

Updated Colab example [here](https://colab.research.google.com/gist/davanstrien/4e88f55a3675c279b5c2f64299ae5c6f/potential_casting_bug.ipynb)
Issue: Loss of cast `Image` feature on certain dataset method

Comment:
Hmmm, if I re-run your google colab I'm getting the right type at the end:
```
sample.features
# {'url': Image(id=None)}
```
Issue: Loss of cast `Image` feature on certain dataset method

Comment:
Appolgies - I've just run again and also got this output. I have also sucesfully used the `push_to_hub` method. I think this is fixed now so will close this issue.
Issue: Add The Medical Segmentation Decathlon Dataset

Comment:
Hello! I have recently been involved with a medical image segmentation project myself and was going through the `The Medical Segmentation Decathlon Dataset` as well. 
I haven't yet had experience adding datasets to this repository yet but would love to get started. Should I take this issue?
If yes, I've got two questions -
1. There are 10 different datasets available, so are all datasets to be added in a single PR, or one at a time? 
2. Since it's a competition, masks for the test-set are not available. How is that to be tackled? Sorry if it's a silly question, I have recently started exploring `datasets`.
Issue: Add The Medical Segmentation Decathlon Dataset

Comment:
Hi! Sure, feel free to take this issue. You can self-assign the issue by commenting `#self-assign`.

To answer your questions:
1. It makes the most sense to add each one as a separate config, so one dataset script with 10 configs in a single PR.
2. Just set masks in the test set to `None`.

Note that the images/masks in this dataset are in NIfTI format, which our `Image` feature currently doesn't support, so I think it's best to yield the paths to the images/masks in the script and add a preprocessing section to the card where we explain how to load/process the images/masks with `nibabel` (I can help with that).
Issue: Add The Medical Segmentation Decathlon Dataset

Comment:
> Note that the images/masks in this dataset are in NIfTI format, which our `Image` feature currently doesn't support, so I think it's best to yield the paths to the images/masks in the script and add a preprocessing section to the card where we explain how to load/process the images/masks with `nibabel` (I can help with that).

Gotcha, thanks. Will start working on the issue and let you know in case of any doubt.
Issue: Add The Medical Segmentation Decathlon Dataset

Comment:
This is great! There is a first model on the HUb that uses this dataset! https://huggingface.co/MONAI/example_spleen_segmentation
Issue: conll 2003 dataset source url is no longer valid

Comment:
Thanks for reporting !

I pushed a temporary fix on `master` that uses an URL from a previous commit to access the dataset for now, until we have a better solution
Issue: conll 2003 dataset source url is no longer valid

Comment:
I changed the URL again to use another host, the fix is available on `master` and we'll probably do a new release of `datasets` tomorrow.

In the meantime, feel free to do `load_dataset(..., revision="master")` to use the fixed script
Issue: conll 2003 dataset source url is no longer valid

Comment:
We just released a new version of `datasets` with a working URL. Feel free to update `datasets` and try again :)
Issue: conll 2003 dataset source url is no longer valid

Comment:
@lhoestq Thank you very much for your answer! 

It works this way, but for my research I need datasets==1.6.3 or closest to it because otherwise the other package would not work as it is built on this version.
Do you have any other suggestion? I would really appreciate it. Maybe which version of the datasets is without hard-coded link but closest to 1.6.3