Make decoding of Audio and Image feature optional
State: closed | Author: mariosasko (COLLABORATOR)

Add the `decode` argument (`True` by default) to the `Audio` and the `Image` feature to make it possible to toggle on/off decoding of these features.

Even though we've discussed that on Slack, I'm not removing the `_storage_dtype` argument of the Audio feature in this PR to avoid breaking the Audio feature tests.

Comments: Closing this PR for now due to https://github.com/huggingface/datasets/issues/3145#issuecomment-993664104.

Okay, after some more thinking, I'm re-opening this PR for three reasons:
* This feature will allow us to remove the `image_file_path`/`audio_file_path` columns in our vision/audio datasets. Currently, it makes sense to keep those columns because it's not obvious how to access the underlying path information of the Image/Audio feature. However, if the user is not aware and does `dataset[0]["image_file_path"]` on our vision/audio datasets, this will be costly because the image/audio file data has to be decoded first (stored in `dataset[0]["image"]`/`dataset[0]["audio"]`)
* In CV, we often work with the so-called "half life" datasets (RedCaps, WIT, ...) that only provide image URLs and not actual image data, and some of these image URLs even go down. Our solution to this problem is to give a note on how to efficiently download the image data using `map` in the datasets cards of such datasets. This feature will remove the need for a separate `image_url` column of type `Value("string")` in such datasets. Instead, we will be able to use the `image` column of type `Image()`  (the image feature knows how to decode image URLs using `xopen`), disable decoding and use `requests.get` for download, which I expect to be faster than `xopen`.
* This feature should help us in implementing `push_to_hub` for the Image/Audio where we transfer actual image/audio data and not paths

> This feature will allow us to remove the image_file_path/audio_file_path columns in our vision/audio datasets. Currently, it makes sense to keep those columns because it's not obvious how to access the underlying path information of the Image/Audio feature. However, if the user is not aware and does dataset[0]["image_file_path"] on our vision/audio datasets, this will be costly because the image/audio file data has to be decoded first (stored in dataset[0]["image"]/dataset[0]["audio"])

That makes sense !

> Instead, we will be able to use the image column of type Image() (the image feature knows how to decode image URLs using xopen), disable decoding and use requests.get for download, which I expect to be faster than xopen.

I feel like it's a bit convoluted compared to having the `image_url` column as string, and say to users to `map` using `requests.get` with `image_url`.

Moreover I'm not 100% sure that we should have `Image` features with both local paths and URLs, since this behavior is a bit hidden the users and they don't give the same performance at all.

> This feature should help us in implementing push_to_hub for the Image/Audio where we transfer actual image/audio data and not paths

Cool !

Thanks, @lhoestq.

> I feel like it's a bit convoluted compared to having the image_url column as string, and say to users to map using requests.get with image_url.

Yes, that makes sense. 

>Moreover I'm not 100% sure that we should have Image features with both local paths and URLs, since this behavior is a bit hidden the users and they don't give the same performance at all.

Do you mean we should remove support for URLs in the Image feature? Because this is what we get for free by adding support for streaming (by using `xopen` instead of `open`) and this is also what the Audio feature does.

> Do you mean we should remove support for URLs in the Image feature? Because this is what we get for free by adding support for streaming (by using xopen instead of open) and this is also what the Audio feature does.

I think it might not be ideal to have URLs in an `Image` type column for a dataset in **non-streaming** mode, since you'd expect to have everything locally. But for a streaming dataset it must use `xopen` indeed

Yes, I agree. Let's have the `image_url` columns as `Value("string")` + a note with the map function to download images for local datasets and a note with `cast_column` (which is requested in https://github.com/huggingface/datasets/issues/3369) for streamed datasets (`ds.cast_column("image_url", Image())`).

I fixed the merge conflicts and small bugs in nested decoding introduced by #3575. Additionally, I addressed https://github.com/huggingface/datasets/issues/3473 by adding the `_iter` method to `Dataset` (inspired by the `_getitem` method). For `iter(dset)` I set `_iter(dset, decoded=True)` to enable decoding and for `map` `_iter(dset, decoded=False)` to make it lazy.