Repository: scikit-image/scikit-image

Issue:
Inconsistent behaviour of ImageCollection
# Problem
Calling `ImageCollection` with a string file pattern vs. a list of filenames produces different results when using a custom `load_func` to read files not supported by pillow. 

I ran into this while trying to use `open_rasterio` from [xarray](https://github.com/pydata/xarray)  as a load_func for `ImageCollection`, though this will undoubtedly come up with any files not readable with `Image.open` from PIL.

## Examples

This works:
```python
files = glob('*.dat')
ims = ImageCollection(files, load_func=open_rasterio)
ims[0] # prints the returned xarray
```

This does not:
```python
ims = ImageCollection('*.dat', load_func=open_rasterio)
ims[0] #  `IndexError: There are only 0 images in the collection`
```
## Cause

The reason is that with the pattern, `_find_images` sets `_numframes` to 0, since PIL cannot open any of the files (even though the load_func has no problems opening them):
https://github.com/scikit-image/scikit-image/blob/8c79a2c96f6e20bcfe4a3e6e4ea812ca74899b80/skimage/io/collection.py#L159-L169

https://github.com/scikit-image/scikit-image/blob/8c79a2c96f6e20bcfe4a3e6e4ea812ca74899b80/skimage/io/collection.py#L197-L221

This in turn results in an error later when `_numframes` is checked.

Inconsistent behaviour of ImageCollection
# Problem
Calling `ImageCollection` with a string file pattern vs. a list of filenames produces different results when using a custom `load_func` to read files not supported by pillow. 

I ran into this while trying to use `open_rasterio` from [xarray](https://github.com/pydata/xarray)  as a load_func for `ImageCollection`, though this will undoubtedly come up with any files not readable with `Image.open` from PIL.

## Examples

This works:
```python
files = glob('*.dat')
ims = ImageCollection(files, load_func=open_rasterio)
ims[0] # prints the returned xarray
```

This does not:
```python
ims = ImageCollection('*.dat', load_func=open_rasterio)
ims[0] #  `IndexError: There are only 0 images in the collection`
```
## Cause

The reason is that with the pattern, `_find_images` sets `_numframes` to 0, since PIL cannot open any of the files (even though the load_func has no problems opening them):
https://github.com/scikit-image/scikit-image/blob/8c79a2c96f6e20bcfe4a3e6e4ea812ca74899b80/skimage/io/collection.py#L159-L169

https://github.com/scikit-image/scikit-image/blob/8c79a2c96f6e20bcfe4a3e6e4ea812ca74899b80/skimage/io/collection.py#L197-L221

This in turn results in an error later when `_numframes` is checked.



Discussion:
This isn't exactly a bug, since the documentation specifies that the file pattern needs to be a list in this case:

```
ImageCollection can be modified to load images from an arbitrary
source by specifying a combination of `load_pattern` and
`load_func`.  For an ImageCollection ``ic``, ``ic[5]`` uses
``load_func(file_pattern[5])`` to load the image.
```

That said, I can see how that would be a useful feature to have, so feel free to add this in a PR.
@stefanv I was quite confused by the documentation here. While that bit implicitly implies that a list is desired when specifying load_func, the latter example with `imread_convert` uses the normal pattern.

Another source of confusion is the fact that the `_files` attribute is not emptied by `_find_images` when it finds no images. This also means that whenever  any file matched by the pattern is not loaded by PIL, the `_files` attribute goes out of sync with the images in the collection. I doubt that this is intended functionality?

I'm willing to take a shot at fixing this, but I'm not clear on what the best solution is. It seems the possible options are to
 1. Always use `_find_images` to try and count the frames, which will fully disable loading any files not supported by PIL.
 2. Always set `_num_frames` to `len(_files)`, which makes the former parameter redundant, and breaks functionality for multi-image tiffs, etc.
 3. Somehow use the given `load_func` in place of PIL in `_find_images` to read the number of frames, which is probably not possibly without more specifications placed on `load_func`
 4. Keep the current behaviour and document it explicitly (and possibly fix the `_files` desync issue)

Any suggestions / comments?
@maaleske You're right; that is very confusing!

1. We do not want to disable loading files not supported by PIL.
2. I think it is reasonable to set _num_frames to len(_files) when a load func is provided.
3. It's unclear to me how that would look in practice.
4. An option, but maybe (2) is more desirable?
This isn't exactly a bug, since the documentation specifies that the file pattern needs to be a list in this case:

```
ImageCollection can be modified to load images from an arbitrary
source by specifying a combination of `load_pattern` and
`load_func`.  For an ImageCollection ``ic``, ``ic[5]`` uses
``load_func(file_pattern[5])`` to load the image.
```

That said, I can see how that would be a useful feature to have, so feel free to add this in a PR.
@stefanv I was quite confused by the documentation here. While that bit implicitly implies that a list is desired when specifying load_func, the latter example with `imread_convert` uses the normal pattern.

Another source of confusion is the fact that the `_files` attribute is not emptied by `_find_images` when it finds no images. This also means that whenever  any file matched by the pattern is not loaded by PIL, the `_files` attribute goes out of sync with the images in the collection. I doubt that this is intended functionality?

I'm willing to take a shot at fixing this, but I'm not clear on what the best solution is. It seems the possible options are to
 1. Always use `_find_images` to try and count the frames, which will fully disable loading any files not supported by PIL.
 2. Always set `_num_frames` to `len(_files)`, which makes the former parameter redundant, and breaks functionality for multi-image tiffs, etc.
 3. Somehow use the given `load_func` in place of PIL in `_find_images` to read the number of frames, which is probably not possibly without more specifications placed on `load_func`
 4. Keep the current behaviour and document it explicitly (and possibly fix the `_files` desync issue)

Any suggestions / comments?
@maaleske You're right; that is very confusing!

1. We do not want to disable loading files not supported by PIL.
2. I think it is reasonable to set _num_frames to len(_files) when a load func is provided.
3. It's unclear to me how that would look in practice.
4. An option, but maybe (2) is more desirable?

Resolving patch:
diff --git a/skimage/io/collection.py b/skimage/io/collection.py
--- a/skimage/io/collection.py
+++ b/skimage/io/collection.py
@@ -138,23 +138,24 @@ class ImageCollection(object):
     ImageCollection can be modified to load images from an arbitrary
     source by specifying a combination of `load_pattern` and
     `load_func`.  For an ImageCollection ``ic``, ``ic[5]`` uses
-    ``load_func(file_pattern[5])`` to load the image.
+    ``load_func(load_pattern[5])`` to load the image.
 
-    Imagine, for example, an ImageCollection that loads every tenth
+    Imagine, for example, an ImageCollection that loads every third
     frame from a video file::
 
-      class AVILoader:
-          video_file = 'myvideo.avi'
+      video_file = 'no_time_for_that_tiny.gif'
 
-          def __call__(self, frame):
-              return video_read(self.video_file, frame)
+      def vidread_step(f, step):
+          vid = imageio.get_reader(f)
+          seq = [v for v in vid.iter_data()]
+          return seq[::step]
 
-      avi_load = AVILoader()
+      ic = ImageCollection(video_file, load_func=vidread_step, step=3)
 
-      frames = range(0, 1000, 10) # 0, 10, 20, ...
-      ic = ImageCollection(frames, load_func=avi_load)
+      ic  # is an ImageCollection object of length 1 because there is 1 file
 
-      x = ic[5] # calls avi_load(frames[5]) or equivalently avi_load(50)
+      x = ic[0]  # calls vidread_step(video_file, step=3)
+      x[5]  # is the sixth element of a list of length 8 (24 / 3)
 
     Another use of ``load_func`` would be to convert all images to ``uint8``::
 
@@ -163,10 +164,6 @@ def imread_convert(f):
 
       ic = ImageCollection('/tmp/*.png', load_func=imread_convert)
 
-    For files with multiple images, the images will be flattened into a list
-    and added to the list of available images.  In this case, ``load_func``
-    should accept the keyword argument ``img_num``.
-
     Examples
     --------
     >>> import skimage.io as io
@@ -189,15 +186,22 @@ def __init__(self, load_pattern, conserve_memory=True, load_func=None,
                 load_pattern = load_pattern.split(os.pathsep)
             for pattern in load_pattern:
                 self._files.extend(glob(pattern))
-            self._files = sorted(self._files, key=alphanumeric_key)
-            self._numframes = self._find_images()
         elif isinstance(load_pattern, str):
             self._files.extend(glob(load_pattern))
-            self._files = sorted(self._files, key=alphanumeric_key)
-            self._numframes = self._find_images()
         else:
             raise TypeError('Invalid pattern as input.')
 
+        self._files = sorted(self._files, key=alphanumeric_key)
+
+        if load_func is None:
+            from ._io import imread
+            self.load_func = imread
+            self._numframes = self._find_images()
+        else:
+            self.load_func = load_func
+            self._numframes = len(self._files)
+            self._frame_index = None
+
         if conserve_memory:
             memory_slots = 1
         else:
@@ -206,12 +210,6 @@ def __init__(self, load_pattern, conserve_memory=True, load_func=None,
         self._conserve_memory = conserve_memory
         self._cached = None
 
-        if load_func is None:
-            from ._io import imread
-            self.load_func = imread
-        else:
-            self.load_func = load_func
-
         self.load_func_kwargs = load_func_kwargs
         self.data = np.empty(memory_slots, dtype=object)
 
@@ -391,7 +389,7 @@ def imread_collection(load_pattern, conserve_memory=True):
         load_pattern : str or list
             Pattern glob or filenames to load. The path can be absolute or
             relative.  Multiple patterns should be separated by a colon,
-            e.g. '/tmp/work/*.png:/tmp/other/*.jpg'.  Also see
+            e.g. ``/tmp/work/*.png:/tmp/other/*.jpg``.  Also see
             implementation notes below.
         conserve_memory : bool, optional
             If True, never keep more than one in memory at a specific
@@ -404,16 +402,23 @@ def imread_collection(load_pattern, conserve_memory=True):
 
 
 class MultiImage(ImageCollection):
-    """A class containing a single multi-frame image.
+
+    """A class containing all frames from multi-frame images.
 
     Parameters
     ----------
-    filename : str
-        The complete path to the image file.
+    load_pattern : str or list of str
+        Pattern glob or filenames to load. The path can be absolute or
+        relative.
     conserve_memory : bool, optional
         Whether to conserve memory by only caching a single frame. Default is
         True.
 
+    Other parameters
+    ----------------
+    load_func : callable
+        ``imread`` by default.  See notes below.
+
     Notes
     -----
     If ``conserve_memory=True`` the memory footprint can be reduced, however


Test patch:
diff --git a/skimage/io/tests/test_collection.py b/skimage/io/tests/test_collection.py
--- a/skimage/io/tests/test_collection.py
+++ b/skimage/io/tests/test_collection.py
@@ -1,8 +1,9 @@
 import os
 
 import numpy as np
+import imageio
 from skimage import data_dir
-from skimage.io.collection import ImageCollection, alphanumeric_key
+from skimage.io.collection import ImageCollection, MultiImage, alphanumeric_key
 from skimage.io import reset_plugins
 
 from skimage._shared import testing
@@ -18,11 +19,10 @@ def test_string_split():
 def test_string_sort():
     filenames = ['f9.10.png', 'f9.9.png', 'f10.10.png', 'f10.9.png',
                  'e9.png', 'e10.png', 'em.png']
-    sorted_filenames = ['e9.png', 'e10.png', 'em.png', 'f9.9.png',
-                        'f9.10.png', 'f10.9.png', 'f10.10.png']
+    expected_filenames = ['e9.png', 'e10.png', 'em.png', 'f9.9.png',
+                          'f9.10.png', 'f10.9.png', 'f10.10.png']
     sorted_filenames = sorted(filenames, key=alphanumeric_key)
-    assert_equal(sorted_filenames, sorted_filenames)
-
+    assert_equal(expected_filenames, sorted_filenames)
 
 def test_imagecollection_input():
     """Test function for ImageCollection. The new behavior (implemented
@@ -57,6 +57,8 @@ def setUp(self):
         self.images = ImageCollection(self.pattern)
         # Image collection with images having shapes that match.
         self.images_matched = ImageCollection(self.pattern_matched)
+        # Same images as a collection of frames
+        self.frames_matched = MultiImage(self.pattern_matched)
 
     def test_len(self):
         assert len(self.images) == 2
@@ -93,6 +95,20 @@ def set_files(f):
         with testing.raises(AttributeError):
             set_files('newfiles')
 
+    def test_custom_load_func_w_kwarg(self):
+        load_pattern = os.path.join(data_dir, 'no_time_for_that_tiny.gif')
+
+        def load_fn(f, step):
+            vid = imageio.get_reader(f)
+            seq = [v for v in vid.iter_data()]
+            return seq[::step]
+
+        ic = ImageCollection(load_pattern, load_func=load_fn, step=3)
+        # Each file should map to one image (array).
+        assert len(ic) == 1
+        # GIF file has 24 frames, so 24 / 3 equals 8.
+        assert len(ic[0]) == 8
+
     def test_custom_load_func(self):
 
         def load_fn(x):
@@ -109,3 +125,7 @@ def test_concatenate(self):
     def test_concatenate_mismatched_image_shapes(self):
         with testing.raises(ValueError):
             self.images.concatenate()
+
+    def test_multiimage_imagecollection(self):
+        assert_equal(self.images_matched[0], self.frames_matched[0])
+        assert_equal(self.images_matched[1], self.frames_matched[1])
diff --git a/skimage/io/tests/test_multi_image.py b/skimage/io/tests/test_multi_image.py
--- a/skimage/io/tests/test_multi_image.py
+++ b/skimage/io/tests/test_multi_image.py
@@ -2,7 +2,7 @@
 
 import numpy as np
 from skimage.io import use_plugin, reset_plugins
-from skimage.io.collection import MultiImage, ImageCollection
+from skimage.io.collection import MultiImage
 
 from skimage._shared import testing
 from skimage._shared.testing import assert_equal, assert_allclose
@@ -20,39 +20,34 @@ def imgs():
             MultiImage(paths[0], conserve_memory=False),
             MultiImage(paths[1]),
             MultiImage(paths[1], conserve_memory=False),
-            ImageCollection(paths[0]),
-            ImageCollection(paths[1], conserve_memory=False),
-            ImageCollection(os.pathsep.join(paths))]
+            MultiImage(os.pathsep.join(paths))]
     yield imgs
 
     reset_plugins()
 
 def test_shapes(imgs):
-    img = imgs[-1]
-    imgs = img[:]
-    assert imgs[0].shape == imgs[1].shape
-    assert imgs[0].shape == (10, 10, 3)
+    imgs = imgs[-1]
+    assert imgs[0][0].shape == imgs[0][1].shape
+    assert imgs[0][0].shape == (10, 10, 3)
 
 def test_len(imgs):
-    assert len(imgs[0]) == len(imgs[1]) == 2
-    assert len(imgs[2]) == len(imgs[3]) == 24
-    assert len(imgs[4]) == 2
-    assert len(imgs[5]) == 24
-    assert len(imgs[6]) == 26, len(imgs[6])
+    assert len(imgs[0][0]) == len(imgs[1][0]) == 2
+    assert len(imgs[2][0]) == len(imgs[3][0]) == 24
+    assert len(imgs[-1]) == 2, len(imgs[-1])
 
 def test_slicing(imgs):
     img = imgs[-1]
-    assert type(img[:]) is ImageCollection
-    assert len(img[:]) == 26, len(img[:])
-    assert len(img[:1]) == 1
-    assert len(img[1:]) == 25
+    assert type(img[:]) is MultiImage
+    assert len(img[0][:]) + len(img[1][:]) == 26, len(img[:])
+    assert len(img[0][:1]) == 1
+    assert len(img[1][1:]) == 23
     assert_allclose(img[0], img[:1][0])
     assert_allclose(img[1], img[1:][0])
     assert_allclose(img[-1], img[::-1][0])
     assert_allclose(img[0], img[::-1][-1])
 
 def test_getitem(imgs):
-    for img in imgs:
+    for img in imgs[0]:
         num = len(img)
 
         for i in range(-num, num):
@@ -69,7 +64,7 @@ def test_getitem(imgs):
 
 def test_files_property(imgs):
     for img in imgs:
-        if isinstance(img, ImageCollection):
+        if isinstance(img, MultiImage):
             continue
 
         assert isinstance(img.filename, str)