GitHub issue #4199: Cache miss during reload for datasets using image fetch utilities through map 
State: closed | Created: 2022-04-22T07:47:08Z

## Describe the bug

It looks like that result of `.map` operation dataset are missing the cache when you reload the script and always run from scratch. In same interpretor session, they are able to find the cache and reload it. But, when you exit the interpretor and reload it, the downloading starts from scratch.

## Steps to reproduce the bug

Using the example provided in `red_caps` dataset.
```python
from concurrent.futures import ThreadPoolExecutor
from functools import partial
import io
import urllib

import PIL.Image

import datasets
from datasets import load_dataset
from datasets.utils.file_utils import get_datasets_user_agent


def fetch_single_image(image_url, timeout=None, retries=0):
    for _ in range(retries + 1):
        try:
            request = urllib.request.Request(
                image_url,
                data=None,
                headers={"user-agent": get_datasets_user_agent()},
            )
            with urllib.request.urlopen(request, timeout=timeout) as req:
                image = PIL.Image.open(io.BytesIO(req.read()))
            break
        except Exception:
            image = None
    return image


def fetch_images(batch, num_threads, timeout=None, retries=0):
    fetch_single_image_with_args = partial(fetch_single_image, timeout=timeout, retries=retries)
    with ThreadPoolExecutor(max_workers=num_threads) as executor:
        batch["image"] = list(executor.map(lambda image_urls: [fetch_single_image_with_args(image_url) for image_url in image_urls], batch["image_url"]))
    return batch


def process_image_urls(batch):
    processed_batch_image_urls = []
    for image_url in batch["image_url"]:
        processed_example_image_urls = []
        image_url_splits = re.findall(r"http\S+", image_url)
        for image_url_split in image_url_splits:
            if "imgur" in image_url_split and "," in image_url_split:
                for image_url_part in image_url_split.split(","):
                    if not image_url_part:
                        continue
                    image_url_part = image_url_part.strip()
                    root, ext = os.path.splitext(image_url_part)
                    if not root.startswith("http"):
                      root = "http://i.imgur.com/" + root
                    root = root.split("#")[0]
                    if not ext:
                      ext = ".jpg"
                    ext = re.split(r"[?%]", ext)[0]
                    image_url_part = root + ext
                    processed_example_image_urls.append(image_url_part)
            else:
                processed_example_image_urls.append(image_url_split)
        processed_batch_image_urls.append(processed_example_image_urls)
    batch["image_url"] = processed_batch_image_urls
    return batch


dset = load_dataset("red_caps", "jellyfish")
dset = dset.map(process_image_urls, batched=True, num_proc=4)
features = dset["train"].features.copy()
features["image"] = datasets.Sequence(datasets.Image())
num_threads = 5
dset = dset.map(fetch_images, batched=True, batch_size=50, features=features, fn_kwargs={"num_threads": num_threads})
```

Run this in an interpretor or as a script twice and see that the cache is missed the second time.

## Expected results
At reload there should not be any cache miss

## Actual results
Every time script is run, cache is missed and dataset is built from scratch.

## Environment info
- `datasets` version: 2.1.1.dev0
- Platform: Linux-4.19.0-20-cloud-amd64-x86_64-with-glibc2.10
- Python version: 3.8.13
- PyArrow version: 7.0.0
- Pandas version: 1.4.1