Repository: scikit-image/scikit-image

Issue:
io.imread not closing file?
### Description:

I just noticed we are seeing this on Windows (tests are passing, but this warning is being raised:
```
io/tests/test_io.py::test_imread_file_url
  C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\contextlib.py:120: ResourceWarning: unclosed file <_io.BufferedReader name='C:\\Users\\VssAdministrator\\AppData\\Local\\scikit-image\\scikit-image\\Cache\\main\\data\\camera.png'>
    next(self.gen)
  Enable tracemalloc to get traceback where the object was allocated.
```

We were seeing a similar issue on Linux and Mac when using `expected_warnings`. I suspect something is wrong with io.imread. Here is the test:
```
def test_imread_file_url():
    # tweak data path so that file URI works on both unix and windows.
    data_path = str(fetch('data/camera.png'))
    data_path = data_path.replace(os.path.sep, '/')
    image_url = f'file:///{data_path}'
    image = io.imread(image_url)
    assert image.shape == (512, 512)
```

### Way to reproduce:

_No response_

### Version information:

_No response_


Discussion:
I noticed this on #6934, but the warnings are appearing on main to:
https://dev.azure.com/scikit-image/scikit-image/_build/results?buildId=9689&view=logs&j=c266ee3e-f298-5c55-b78e-d135f25941c2&t=76d8336e-2ec7-54e7-5531-ad7a9096482b&l=8088
Also letting @FirefoxMetzger know.
That's a strange, but interesting find. I built skimage from master just now and I can reproduce the warning. (I'm on Windows.) 

Interestingly ImageIO calls `file.close()` twice on the `_io.BufferedReader` object during this test. First via pillow when the pillow plugin cleans up after itself (line 123 inside `imageio/plugins/pillow.py`), and then again later by our `Request` object when it cleans up after itself (line 570 in `imageio/core/Request.py`). A following `assert _io.BufferedReader.closed` doesn't raise (so the buffered reader object, too, thinks it has been closed).

I also stepped through how we open the file and instantiate the plugin and our `Request` object - in case we re-open a file somewhere and lose the reference. However, we open it exactly once (in `imageio/core/request.py:492`) and then just keep reusing that reference. Pillow, too, reuses the reference at least up until it passes the file to it's internal PNG reader, so no leaked pointers there neither.

When I do the suggested thing and trace the malloc via `python -X tracemalloc -m pytest .\skimage\io\tests\test_io.py::test_imread_file_url` it traces the malloc into python's internal contextlib:

```
skimage/io/tests/test_io.py::test_imread_file_url
  C:\Users\Sebastian\AppData\Local\Programs\Python\Python311\Lib\contextlib.py:144: ResourceWarning: unclosed file <_io.BufferedReader name='C:\\Users\\Sebastian\\AppData\\Local\\scikit-image\\scikit-image\\Cache\\main\\data\\..\\data\\camera.png'>
    next(self.gen)

  Object allocated at:
    File "C:\Users\Sebastian\AppData\Local\Programs\Python\Python311\Lib\contextlib.py", line 144
      next(self.gen)
```

Which also isn't particularly helpful.

---

Unfortunately, I will have to put this down for today but if - in the mean time - somebody has an idea of where the problem originates I'd be happy to check if ImageIO can do something to fix it.
Thank you for the detailed investigation, @FirefoxMetzger! Quite a head-scratcher.
I found the culprit in `skimage.io.util.py:file_or_url_context` ☠️ 

https://github.com/scikit-image/scikit-image/blob/42f134a63735fda4903ea8f6ff0023b66bdd31d0/skimage/io/util.py#L26-L41

`skimage.io.imread` uses `file_or_url_context(fname)` which passes a (temporary) copy of the file to ImageIO. ImageIO dutifully closes this copy, but can't do anything about the original file because it never sees that reference.  Instead, skimage leaks the handle since it opens the original file via `urllib.request.urlopen` but never closes it.

Adding a context manager for `urlopen` like so:

```python
with urllib.request.urlopen(resource_name) as u:
    f.write(u.read())
```

fixes the problem and makes the warning disappear. (Proving once again that it is way too easy to forget about calling `resource.close()` and that my zealousness regarding resource context managers is justified ⛪)
🙈 Oh, jeez, that's embarrassing. Thank you for your careful sleuthing!

Resolving patch:
diff --git a/skimage/io/collection.py b/skimage/io/collection.py
--- a/skimage/io/collection.py
+++ b/skimage/io/collection.py
@@ -111,6 +111,8 @@ class ImageCollection:
     ----------------
     load_func : callable
         ``imread`` by default. See notes below.
+    **load_func_kwargs : dict
+        Any other keyword arguments are passed to `load_func`.
 
     Attributes
     ----------
@@ -124,27 +126,27 @@ class ImageCollection:
     Note that files are always returned in alphanumerical order. Also note
     that slicing returns a new ImageCollection, *not* a view into the data.
 
-    ImageCollection can be modified to load images from an arbitrary
-    source by specifying a combination of `load_pattern` and
-    `load_func`.  For an ImageCollection ``ic``, ``ic[5]`` uses
-    ``load_func(load_pattern[5])`` to load the image.
+    ImageCollection image loading can be customized through
+    `load_func`. For an ImageCollection ``ic``, ``ic[5]`` calls
+    ``load_func(load_pattern[5])`` to load that image.
 
-    Imagine, for example, an ImageCollection that loads every third
-    frame from a video file::
+    For example, here is an ImageCollection that, for each video provided,
+    loads every second frame::
 
-      video_file = 'no_time_for_that_tiny.gif'
+      import imageio.v3 as iio3
+      import itertools
 
       def vidread_step(f, step):
-          vid = imageio.get_reader(f)
-          seq = [v for v in vid.iter_data()]
-          return seq[::step]
+          vid = iio3.imiter(f)
+          return list(itertools.islice(vid, None, None, step)
 
-      ic = ImageCollection(video_file, load_func=vidread_step, step=3)
+      video_file = 'no_time_for_that_tiny.gif'
+      ic = ImageCollection(video_file, load_func=vidread_step, step=2)
 
-      ic  # is an ImageCollection object of length 1 because there is 1 file
+      ic  # is an ImageCollection object of length 1 because 1 video is provided
 
-      x = ic[0]  # calls vidread_step(video_file, step=3)
-      x[5]  # is the sixth element of a list of length 8 (24 / 3)
+      x = ic[0]
+      x[5]  # the 10th frame of the first video
 
     Alternatively, if `load_func` is provided and `load_pattern` is a
     sequence, an `ImageCollection` of corresponding length will be created,
@@ -154,12 +156,13 @@ def vidread_step(f, step):
     files (or strings at all). For example, to create an `ImageCollection`
     containing 500 images from a video::
 
-      class vidread_random:
+      class FrameReader:
           def __init__ (self, f):
-              self.vid = imageio.get_reader(f)
-          def __call__ (self, frameno):
-              return self.vid.get_data(frameno)
-      ic = ImageCollection(range(500), load_func=vidread_random('movie.mp4'))
+              self.f = f
+          def __call__ (self, index):
+              return iio3.imread(self.f, index=index)
+
+      ic = ImageCollection(range(500), load_func=FrameReader('movie.mp4'))
 
       ic  # is an ImageCollection object of length 500
 
@@ -172,9 +175,11 @@ def imread_convert(f):
 
     Examples
     --------
-    >>> import imageio
+    >>> import imageio.v3 as iio3
     >>> import skimage.io as io
-    >>> from skimage import data_dir
+
+    # Where your images are located
+    >>> data_dir = os.path.join(os.path.dirname(__file__), '../data')
 
     >>> coll = io.ImageCollection(data_dir + '/chess*.png')
     >>> len(coll)
@@ -182,18 +187,20 @@ def imread_convert(f):
     >>> coll[0].shape
     (200, 200)
 
-    >>> image_col = io.ImageCollection(['/tmp/work/*.png', '/tmp/other/*.jpg'])
+    >>> image_col = io.ImageCollection([f'{data_dir}/*.png', '{data_dir}/*.jpg'])
 
-    >>> class multiread:
+    >>> class MultiReader:
     ...     def __init__ (self, f):
-    ...         self.vid = imageio.get_reader(f)
-    ...     def __call__ (self, frameno):
-    ...         return self.vid.get_data(frameno)
+    ...         self.f = f
+    ...     def __call__ (self, index):
+    ...         return iio3.imread(self.f, index=index)
     ...
     >>> filename = data_dir + '/no_time_for_that_tiny.gif'
-    >>> image_col = io.ImageCollection(range(24), load_func=multiread(filename))
+    >>> ic = io.ImageCollection(range(24), load_func=MultiReader(filename))
     >>> len(image_col)
-    24
+    23
+    >>> isinstance(ic[0], np.ndarray)
+    True
     """
     def __init__(self, load_pattern, conserve_memory=True, load_func=None,
                  **load_func_kwargs):
@@ -448,7 +455,8 @@ class MultiImage(ImageCollection):
 
     Examples
     --------
-    >>> from skimage import data_dir
+    # Where your images are located
+    >>> data_dir = os.path.join(os.path.dirname(__file__), '../data')
 
     >>> multipage_tiff = data_dir + '/multipage.tif'
     >>> multi_img = MultiImage(multipage_tiff)
diff --git a/skimage/io/util.py b/skimage/io/util.py
--- a/skimage/io/util.py
+++ b/skimage/io/util.py
@@ -25,8 +25,8 @@ def file_or_url_context(resource_name):
         _, ext = os.path.splitext(url_components.path)
         try:
             with tempfile.NamedTemporaryFile(delete=False, suffix=ext) as f:
-                u = urllib.request.urlopen(resource_name)
-                f.write(u.read())
+                with urllib.request.urlopen(resource_name) as u:
+                    f.write(u.read())
             # f must be closed before yielding
             yield f.name
         except (URLError, HTTPError):


Test patch:
diff --git a/skimage/io/tests/test_collection.py b/skimage/io/tests/test_collection.py
--- a/skimage/io/tests/test_collection.py
+++ b/skimage/io/tests/test_collection.py
@@ -1,7 +1,8 @@
 import os
+import itertools
 
 import numpy as np
-import imageio
+import imageio.v3 as iio3
 from skimage import data_dir
 from skimage.io.collection import ImageCollection, MultiImage, alphanumeric_key
 from skimage.io import reset_plugins
@@ -32,6 +33,7 @@ def test_string_sort():
     sorted_filenames = sorted(filenames, key=alphanumeric_key)
     assert_equal(expected_filenames, sorted_filenames)
 
+
 def test_imagecollection_input():
     """Test function for ImageCollection. The new behavior (implemented
     in 0.16) allows the `pattern` argument to accept a list of strings
@@ -44,29 +46,33 @@ def test_imagecollection_input():
     # Ensure that these images are part of the legacy datasets
     # this means they will always be available in the user's install
     # regardless of the availability of pooch
-    pattern = [os.path.join(data_dir, pic)
-               for pic in ['coffee.png',
-                           'chessboard_GRAY.png',
-                           'rocket.jpg']]
+    pics = [
+        fetch('data/coffee.png'),
+        fetch('data/chessboard_GRAY.png'),
+        fetch('data/rocket.jpg')
+    ]
+    pattern = [os.path.join(data_dir, pic) for pic in pics]
     images = ImageCollection(pattern)
     assert len(images) == 3
 
 
 class TestImageCollection():
-    pattern = [os.path.join(data_dir, pic)
-               for pic in ['brick.png', 'color.png']]
-
-    pattern_matched = [os.path.join(data_dir, pic)
-                       for pic in ['brick.png', 'moon.png']]
+    pics = [
+        fetch('data/brick.png'),
+        fetch('data/color.png'),
+        fetch('data/moon.png')
+    ]
+    pattern = pics[:2]
+    pattern_same_shape = pics[::2]
 
     def setup_method(self):
         reset_plugins()
         # Generic image collection with images of different shapes.
         self.images = ImageCollection(self.pattern)
         # Image collection with images having shapes that match.
-        self.images_matched = ImageCollection(self.pattern_matched)
+        self.images_matched = ImageCollection(self.pattern_same_shape)
         # Same images as a collection of frames
-        self.frames_matched = MultiImage(self.pattern_matched)
+        self.frames_matched = MultiImage(self.pattern_same_shape)
 
     def test_len(self):
         assert len(self.images) == 2
@@ -107,24 +113,22 @@ def set_files(f):
     def test_custom_load_func_sequence(self):
         filename = fetch('data/no_time_for_that_tiny.gif')
 
-        def reader(frameno):
-            vid = imageio.get_reader(filename)
-            return vid.get_data(frameno)
+        def reader(index):
+            return iio3.imread(filename, index=index)
 
         ic = ImageCollection(range(24), load_func=reader)
         # the length of ic should be that of the given load_pattern sequence
         assert len(ic) == 24
         # GIF file has frames of size 25x14 with 4 channels (RGBA)
-        assert ic[0].shape == (25, 14, 4)
+        assert ic[0].shape == (25, 14, 3)
 
     @pytest.mark.skipif(not has_pooch, reason="needs pooch to download data")
     def test_custom_load_func_w_kwarg(self):
         load_pattern = fetch('data/no_time_for_that_tiny.gif')
 
         def load_fn(f, step):
-            vid = imageio.get_reader(f)
-            seq = [v for v in vid.iter_data()]
-            return seq[::step]
+            vid = iio3.imiter(f)
+            return list(itertools.islice(vid, None, None, step))
 
         ic = ImageCollection(load_pattern, load_func=load_fn, step=3)
         # Each file should map to one image (array).
diff --git a/skimage/io/tests/test_io.py b/skimage/io/tests/test_io.py
--- a/skimage/io/tests/test_io.py
+++ b/skimage/io/tests/test_io.py
@@ -62,10 +62,10 @@ def test_imread_pathlib_tiff():
     """Tests reading from Path object (issue gh-5545)."""
 
     # read via fetch
-    expected = io.imread(fetch('data/multipage.tif'))
+    fname = fetch('data/multipage.tif')
+    expected = io.imread(fname)
 
     # read by passing in a pathlib.Path object
-    fname = os.path.join(data_dir, 'multipage.tif')
     path = pathlib.Path(fname)
     img = io.imread(path)