Dataset Repo Paths to Locally Stored Images Not Being Appended to Image Path
Opened by dennys246 (closed)

### Describe the bug

I’m not sure if this is a bug or a feature and I just don’t fully understand how dataset loading is to work, but it appears there may be a bug with how locally stored Image() are being accessed. I’ve uploaded a new dataset to hugging face (rmdig/rocky_mountain_snowpack) but I’ve come into a ton of trouble trying to have the images handled properly (at least in the way I’d expect them to be handled). 

I find that I cannot use relative paths for loading images remotely from the Hugging Face repo or from a local repository. Any time I do it always simply appends my current working directory to the dataset. As a result to use the datasets library with my dataset I have to change my working directory to the dataset folder or abandon the dataset object structure, which I cannot imagine you intended. As a result I have to use URL’s since an absolute path on my system obviously wouldn’t work for others. The URL works ok, but despite me having it locally downloaded, it appears to be redownloading the dataset every time I train my snowGAN model on it (and often times I’m coming into HTTPS errors for over requesting the data).

Or maybe image relative paths aren't intended to be loaded directly through your datasets library as images and should be kept as strings for the user to handle? If so I feel like you’re missing out on some pretty seamless functionality

### Steps to reproduce the bug

1. Download a local copy of the dataset (rmdig/rocky_mountain_snowpack) through git or whatever you prefer.
2. Alter the README.md YAML for file_path (the relative path to each image) to be type Image instead of type string

`
---
dataset_info:
  features:
    - name: image
      dtype: Image
    - name: file_path
      dtype: Image
`

3. Initialize the dataset locally, make sure your working directory is not the dataset directory root 

`dataset = datasets.load_dataset(‘path/to/local/rocky_mountain_snowpack/‘)`
 
4. Call to one of the samples and you’ll get an error that the image was not found in current/working/directory/preprocessed/cores/image_1.png. Showing that it’s simply looking in the current working directory + relative path

`
>>> dataset['train'][0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/dennyschaedig/miniconda3/lib/python3.12/site-packages/datasets/arrow_dataset.py", line 2859, in __getitem__
    return self._getitem(key)
           ^^^^^^^^^^^^^^^^^^
  File "/Users/dennyschaedig/miniconda3/lib/python3.12/site-packages/datasets/arrow_dataset.py", line 2841, in _getitem
    formatted_output = format_table(
                       ^^^^^^^^^^^^^
  File "/Users/dennyschaedig/miniconda3/lib/python3.12/site-packages/datasets/formatting/formatting.py", line 657, in format_table
    return formatter(pa_table, query_type=query_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dennyschaedig/miniconda3/lib/python3.12/site-packages/datasets/formatting/formatting.py", line 410, in __call__
    return self.format_row(pa_table)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dennyschaedig/miniconda3/lib/python3.12/site-packages/datasets/formatting/formatting.py", line 459, in format_row
    row = self.python_features_decoder.decode_row(row)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dennyschaedig/miniconda3/lib/python3.12/site-packages/datasets/formatting/formatting.py", line 223, in decode_row
    return self.features.decode_example(row, token_per_repo_id=self.token_per_repo_id) if self.features else row
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dennyschaedig/miniconda3/lib/python3.12/site-packages/datasets/features/features.py", line 2093, in decode_example
    column_name: decode_nested_example(feature, value, token_per_repo_id=token_per_repo_id)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dennyschaedig/miniconda3/lib/python3.12/site-packages/datasets/features/features.py", line 1405, in decode_nested_example
    return schema.decode_example(obj, token_per_repo_id=token_per_repo_id) if obj is not None else None
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dennyschaedig/miniconda3/lib/python3.12/site-packages/datasets/features/image.py", line 171, in decode_example
    image = PIL.Image.open(path)
            ^^^^^^^^^^^^^^^^^^^^
  File "/Users/dennyschaedig/miniconda3/lib/python3.12/site-packages/PIL/Image.py", line 3277, in open
    fp = builtins.open(filename, "rb")
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/Users/dennyschaedig/Datasets/preprocessed/cores/image_1.png'
`

### Expected behavior

I expect the datasets and Image() to load the locally hosted data using path/to/local/rocky_mountain_snowpack/ (that I pass in with my datasets.load_dataset() or the you all handle on the backend) call + relative path. 

Instead it appears to load from my current working directory + relative path.

### Environment info

Tested on…

Windows 11, Ubuntu Linux 22.04 and Mac Sequoia 15.5 Silicone M2

datasets version 4.0.0

Python 3.12 and 3.13

Comments:
This is the download issues I come into, about ever other time it fails...
<img width="1719" height="1226" alt="Image" src="https://github.com/user-attachments/assets/2e5b4b3e-7c13-4bad-a77c-34b47a932831" />

I’m guessing this is just a feature so I’m going to close this thread. I also altered my loading scheme to start on the first index of a particular modality within the dataset (index ~390) and this issue went away with client error from too many requests. Due to how the dataset is sorted in HF, there are gaps in my dataset between modalities (~500) that this issue should theoretically also occur on but it does not. It seems after initially downloading the first image in a dataset the connection becomes approved on HF end and long lapses in checking entries in a dataset, without actually loading the full sample, are enabled. 

TL;DR Local handling doesn’t appear to be possible with images in the datasets library. Load the first image you need right away through storing it’s index and calling to it. Don’t iterate long sequences of HF repo’s looking for a condition to be met without first loading in a sample.