Repository: scikit-image/scikit-image

Issue:
downscale_local_mean with non-integer factors? 
I'm interested in a version of `skimage.transform.downscale_local_mean()` which handles non-evenly dividing factors without padding the image. I want each pixel in the result to be an weighted average of overlapping pixels in the source, with weights equal to the area of overlap.

To be more explicit, this transformation is linear along equal axis. For example, from size 5 to size 2, it can be written as the matrix:
```
[[0.4, 0.4, 0.2, 0.0, 0.0],
 [0.0, 0.0, 0.2, 0.4, 0.4]]
```

This is actually what I expected `skimage.transform.resize()` to do, before I noticed the docstring warning about anti-aliasing.

Would this be a welcome addition to scikit-image in some form? I wrote a version that works for my own purposes, but my guess is that it's too slow for skimage, since I use a dense matrix multiply along each axis with matrix of weights.
WIP: resize with local mean by non-integer size
**Not for serious review**

I'm making this PR mostly to clarify that it is OK to reuse this code under skimage's license, e.g.,
inside https://github.com/scikit-image/scikit-image/pull/4201



Discussion:
Hi, thanks for reaching out. Even if your implementation is slow, this would be a useful option. I would integrate it into the existing `downscale_local_mean` function as another option. Does your implementation generalize to N-D?
> Does your implementation generalize to N-D?

Yes, it works in N dimensions. It's written as a sequence of matrix multiplications, one along each axis.
Sounds great, looking forward to a pull request with the feature!
This has actually been requested before, so thanks for contributing!
Do you have any guidance on the appropriate API here?

This breaks at least two aspects of the API for `downscale_local_mean`:
- `factors` don't need to be integers.
- `cval` doesn't make sense.

Also, we can just as easily do upscaling with the local mean value as well downscaling, e.g., resizing `[0, 1]` to size 5 would result in `[0, 0, 0.5, 1, 1]`.

My initial thought was to make `cval=None` an indicator to do resizing with fractional overlap. But then there's the separate issue of non-integer factors. An API that allows non-integer factors only if cval is None seems very awkward. I don't like the value dependent semantics. I suppose we could probably support `cval` when padding by a fractional factors, but the implementation is not obvious to me and that's not a feature I'm excited to write.

My tentative proposal would be a new function, perhaps `resize_local_mean(image, output_shape)`.
A new function resize_local_mean (maybe rather rescale_local_mean to match the semantics of the existing resize/rescale functions) sounds like a good proposal to me. Potentially, we want to deprecate the old function and merge it into this new function. For this we would probably have to introduce some parameter, eg mode=“cut|pad” and the cval parameter for the pad mode. What do you think? Let’s hear some feedback on that from other people as well. 

Whenever the input is exactly divisible by integer factors, we could internally also fall back to the old implementation, assuming it was significantly faster?


I think the old behavior is somewhat broken.  We should definitely transition to @shoyer's implementation eventually.

Suggested route: set `cval=None` for the next two releases, and warn if it is not set to zero explicitly.  Then, replace that function with the one proposed.

For now, let's include the function in `skimage.future`, with an entry in TODO.txt to move it over after the deprecation.

See also http://scikit-image.org/docs/dev/contribute.html#deprecation-cycle
@shoyer Would you mind sharing your current implementation? I'm having trouble deriving an analytical expression for the weights of each voxel.
See https://gist.github.com/shoyer/c0f1ddf409667650a076c058f9a17276 for the version I wrote for my needs.
@shoyer Thanks a lot! 


Resolving patch:
diff --git a/benchmarks/benchmark_transform_warp.py b/benchmarks/benchmark_transform_warp.py
--- a/benchmarks/benchmark_transform_warp.py
+++ b/benchmarks/benchmark_transform_warp.py
@@ -1,5 +1,5 @@
 import numpy as np
-from skimage.transform import SimilarityTransform, warp
+from skimage.transform import SimilarityTransform, warp, resize_local_mean
 import warnings
 import functools
 import inspect
@@ -52,3 +52,24 @@ def time_to_float64(self, dtype_in, N, order):
         transformations."""
         result = warp(self.image, self.tform, order=self.order,
                       preserve_range=True)
+
+
+class ResizeLocalMeanSuite:
+    params = ([np.uint8, np.uint16, np.float32, np.float64],
+              [128, 512, 1024, 2048],
+              [128, 512, 1024],
+              [2, 3],
+              [2, 3],
+              [True, False]
+              )
+    param_names = ['dtype', 'shape_in', 'shape_out', 'ndim_in',
+                   'ndim_out', 'grid_mode']
+
+    def setup(self, dtype, shape_in, shape_out, ndim_in, ndim_out, grid_mode):
+        self.shape_in = ndim_in * (shape_in, )
+        self.image = np.zeros(self.shape_in, dtype=dtype)
+        self.shape_out = ndim_out * (shape_out, )
+        self.grid_mode = grid_mode
+
+    def time_resize_local_mean(self):
+        resize_local_mean(self.image, self.shape_out, self.grid_mode)
diff --git a/skimage/transform/__init__.py b/skimage/transform/__init__.py
--- a/skimage/transform/__init__.py
+++ b/skimage/transform/__init__.py
@@ -12,7 +12,8 @@
                          EssentialMatrixTransform, PolynomialTransform,
                          PiecewiseAffineTransform)
 from ._warps import (swirl, resize, rotate, rescale,
-                     downscale_local_mean, warp, warp_coords, warp_polar)
+                     downscale_local_mean, warp, warp_coords, warp_polar,
+                     resize_local_mean)
 from .pyramids import (pyramid_reduce, pyramid_expand,
                        pyramid_gaussian, pyramid_laplacian)
 
@@ -46,6 +47,7 @@
            'PiecewiseAffineTransform',
            'swirl',
            'resize',
+           'resize_local_mean',
            'rotate',
            'rescale',
            'downscale_local_mean',
diff --git a/skimage/transform/_warps.py b/skimage/transform/_warps.py
--- a/skimage/transform/_warps.py
+++ b/skimage/transform/_warps.py
@@ -10,9 +10,9 @@
 
 from .._shared import utils
 from .._shared.utils import (get_bound_method_class, safe_as_int, warn,
-                             convert_to_float, _supported_float_type,
-                             _to_ndimage_mode,
-                             _validate_interpolation_order)
+                             convert_to_float, _to_ndimage_mode,
+                             _validate_interpolation_order,
+                             channel_as_last_axis)
 
 HOMOGRAPHY_TRANSFORMS = (
     SimilarityTransform,
@@ -21,6 +21,53 @@
 )
 
 
+def _preprocess_resize_output_shape(image, output_shape):
+    """Validate resize output shape according to input image.
+
+    Parameters
+    ----------
+    image: ndarray
+        Image to be resized.
+    output_shape: tuple or ndarray
+        Size of the generated output image `(rows, cols[, ...][, dim])`. If
+        `dim` is not provided, the number of channels is preserved.
+
+    Returns
+    -------
+    image: ndarray
+        The input image, but with additional singleton dimensions appended in
+        the case where ``len(output_shape) > input.ndim``.
+    output_shape: tuple
+        The output image converted to tuple.
+
+    Raises
+    ------
+    ValueError:
+        If output_shape length is smaller than the image number of
+        dimensions
+
+    Notes
+    -----
+    The input image is reshaped if its number of dimensions is not
+    equal to output_shape_length.
+
+    """
+    output_ndim = len(output_shape)
+    input_shape = image.shape
+    if output_ndim > image.ndim:
+        # append dimensions to input_shape
+        input_shape += (1, ) * (output_ndim - image.ndim)
+        image = np.reshape(image, input_shape)
+    elif output_ndim == image.ndim - 1:
+        # multichannel case: append shape of last axis
+        output_shape = output_shape + (image.shape[-1], )
+    elif output_ndim < image.ndim:
+        raise ValueError("output_shape length cannot be smaller than the "
+                         "image number of dimensions")
+
+    return image, output_shape
+
+
 def resize(image, output_shape, order=None, mode='reflect', cval=0, clip=True,
            preserve_range=False, anti_aliasing=None, anti_aliasing_sigma=None):
     """Resize image to match a certain size.
@@ -93,19 +140,9 @@ def resize(image, output_shape, order=None, mode='reflect', cval=0, clip=True,
     (100, 100)
 
     """
-    output_shape = tuple(output_shape)
-    output_ndim = len(output_shape)
+
+    image, output_shape = _preprocess_resize_output_shape(image, output_shape)
     input_shape = image.shape
-    if output_ndim > image.ndim:
-        # append dimensions to input_shape
-        input_shape = input_shape + (1, ) * (output_ndim - image.ndim)
-        image = np.reshape(image, input_shape)
-    elif output_ndim == image.ndim - 1:
-        # multichannel case: append shape of last axis
-        output_shape = output_shape + (image.shape[-1], )
-    elif output_ndim < image.ndim - 1:
-        raise ValueError("len(output_shape) cannot be smaller than the image "
-                         "dimensions")
 
     if image.dtype == np.float16:
         image = image.astype(np.float32)
@@ -1127,3 +1164,167 @@ def warp_polar(image, center=None, *, radius=None, output_shape=None,
                   output_shape=output_shape, **kwargs)
 
     return warped
+
+
+def _local_mean_weights(old_size, new_size, grid_mode, dtype):
+    """Create a 2D weight matrix for resizing with the local mean.
+
+    Parameters
+    ----------
+    old_size: int
+        Old size.
+    new_size: int
+        New size.
+    grid_mode : bool
+        Whether to use grid data model of pixel/voxel model for
+        average weights computation.
+    dtype: dtype
+        Output array data type.
+
+    Returns
+    -------
+    weights: (new_size, old_size) array
+        Rows sum to 1.
+
+    """
+    if grid_mode:
+        old_breaks = np.linspace(0, old_size, num=old_size + 1, dtype=dtype)
+        new_breaks = np.linspace(0, old_size, num=new_size + 1, dtype=dtype)
+    else:
+        old, new = old_size - 1, new_size - 1
+        old_breaks = np.pad(np.linspace(0.5, old - 0.5, old, dtype=dtype),
+                            1, 'constant', constant_values=(0, old))
+        if new == 0:
+            val = np.inf
+        else:
+            val = 0.5 * old / new
+        new_breaks = np.pad(np.linspace(val, old - val, new, dtype=dtype),
+                            1, 'constant', constant_values=(0, old))
+
+    upper = np.minimum(new_breaks[1:, np.newaxis], old_breaks[np.newaxis, 1:])
+    lower = np.maximum(new_breaks[:-1, np.newaxis],
+                       old_breaks[np.newaxis, :-1])
+
+    weights = np.maximum(upper - lower, 0)
+    weights /= weights.sum(axis=1, keepdims=True)
+
+    return weights
+
+
+def resize_local_mean(image, output_shape, grid_mode=True,
+                      preserve_range=False, *, channel_axis=None):
+    """Resize an array with the local mean / bilinear scaling.
+
+    Parameters
+    ----------
+    image : ndarray
+        Input image. If this is a multichannel image, the axis corresponding
+        to channels should be specified using `channel_axis`
+    output_shape : tuple or ndarray
+        Size of the generated output image. When `channel_axis` is not None,
+        the `channel_axis` should either be omitted from `output_shape` or the
+        ``output_shape[channel_axis]`` must match
+        ``image.shape[channel_axis]``. If the length of `output_shape` exceeds
+        image.ndim, additional singleton dimensions will be appended to the
+        input ``image`` as needed.
+    grid_mode : bool, optional
+        Defines ``image`` pixels position: if True, pixels are assumed to be at
+        grid intersections, otherwise at cell centers. As a consequence,
+        for example, a 1d signal of length 5 is considered to have length 4
+        when `grid_mode` is False, but length 5 when `grid_mode` is True. See
+        the following visual illustration:
+
+        .. code-block:: text
+
+                | pixel 1 | pixel 2 | pixel 3 | pixel 4 | pixel 5 |
+                     |<-------------------------------------->|
+                                        vs.
+                |<----------------------------------------------->|
+
+        The starting point of the arrow in the diagram above corresponds to
+        coordinate location 0 in each mode.
+    preserve_range : bool, optional
+        Whether to keep the original range of values. Otherwise, the input
+        image is converted according to the conventions of `img_as_float`.
+        Also see
+        https://scikit-image.org/docs/dev/user_guide/data_types.html
+
+    Returns
+    -------
+    resized : ndarray
+        Resized version of the input.
+
+    See Also
+    --------
+    resize, downscale_local_mean
+
+    Notes
+    -----
+    This method is sometimes referred to as "area-based" interpolation or
+    "pixel mixing" interpolation [1]_. When `grid_mode` is True, it is
+    equivalent to using OpenCV's resize with `INTER_AREA` interpolation mode.
+    It is commonly used for image downsizing. If the downsizing factors are
+    integers, then `downscale_local_mean` should be preferred instead.
+
+    References
+    ----------
+    .. [1] http://entropymine.com/imageworsener/pixelmixing/
+
+    Examples
+    --------
+    >>> from skimage import data
+    >>> from skimage.transform import resize_local_mean
+    >>> image = data.camera()
+    >>> resize_local_mean(image, (100, 100)).shape
+    (100, 100)
+
+    """
+    if channel_axis is not None:
+        if channel_axis < -image.ndim or channel_axis >= image.ndim:
+            raise ValueError("invalid channel_axis")
+
+        # move channels to last position
+        image = np.moveaxis(image, channel_axis, -1)
+        nc = image.shape[-1]
+
+        output_ndim = len(output_shape)
+        if output_ndim == image.ndim - 1:
+            # insert channels dimension at the end
+            output_shape = output_shape + (nc,)
+        elif output_ndim == image.ndim:
+            if output_shape[channel_axis] != nc:
+                raise ValueError(
+                    "Cannot reshape along the channel_axis. Use "
+                    "channel_axis=None to reshape along all axes."
+                )
+            # move channels to last position in output_shape
+            channel_axis = channel_axis % image.ndim
+            output_shape = (
+                output_shape[:channel_axis] + output_shape[channel_axis:] +
+                (nc,)
+            )
+        else:
+            raise ValueError(
+                "len(output_shape) must be image.ndim or (image.ndim - 1) "
+                "when a channel_axis is specified."
+            )
+        resized = image
+    else:
+        resized, output_shape = _preprocess_resize_output_shape(image,
+                                                                output_shape)
+    resized = convert_to_float(resized, preserve_range)
+    dtype = resized.dtype
+
+    for axis, (old_size, new_size) in enumerate(zip(image.shape,
+                                                    output_shape)):
+        if old_size == new_size:
+            continue
+        weights = _local_mean_weights(old_size, new_size, grid_mode, dtype)
+        product = np.tensordot(resized, weights, [[axis], [-1]])
+        resized = np.moveaxis(product, -1, axis)
+
+    if channel_axis is not None:
+        # restore channels to original axis
+        resized = np.moveaxis(resized, -1, channel_axis)
+
+    return resized


Test patch:
diff --git a/skimage/transform/tests/test_warps.py b/skimage/transform/tests/test_warps.py
--- a/skimage/transform/tests/test_warps.py
+++ b/skimage/transform/tests/test_warps.py
@@ -15,7 +15,8 @@
                                       _log_polar_mapping, warp,
                                       warp_coords, rotate, resize,
                                       rescale, warp_polar, swirl,
-                                      downscale_local_mean)
+                                      downscale_local_mean,
+                                      resize_local_mean)
 from skimage.transform._geometric import (AffineTransform,
                                           ProjectiveTransform,
                                           SimilarityTransform)
@@ -715,7 +716,7 @@ def test_bool_img_resize():
     assert_array_equal(res, expected)
 
 
-def test_boll_array_warnings():
+def test_bool_array_warnings():
     img = np.zeros((10, 10), dtype=bool)
 
     with expected_warnings(['Input image dtype is bool']):
@@ -762,3 +763,129 @@ def test_nonzero_order_warp_dtype(dtype, order):
     assert rotate(img, 45, order=order).dtype == float_dtype
     assert warp_polar(img, order=order).dtype == float_dtype
     assert swirl(img, order=order).dtype == float_dtype
+
+
+def test_resize_local_mean2d():
+    x = np.zeros((5, 5), dtype=np.double)
+    x[1, 1] = 1
+    resized = resize_local_mean(x, (10, 10))
+    ref = np.zeros((10, 10))
+    ref[2:4, 2:4] = 1
+    assert_array_almost_equal(resized, ref)
+
+
+@pytest.mark.parametrize('channel_axis', [0, 1, 2, -1, -2, -3])
+def test_resize_local_mean3d_keep(channel_axis):
+    # keep 3rd dimension
+    nch = 3
+    x = np.zeros((5, 5, nch), dtype=np.double)
+    x[1, 1, :] = 1
+    # move channels to expected dimension
+    x = np.moveaxis(x, -1, channel_axis)
+    resized = resize_local_mean(x, (10, 10), channel_axis=channel_axis)
+    # move channels back to last axis to match the reference image
+    resized = np.moveaxis(resized, channel_axis, -1)
+    with pytest.raises(ValueError):
+        # output_shape too short
+        resize_local_mean(x, (10, ))
+    ref = np.zeros((10, 10, nch))
+    ref[2:4, 2:4, :] = 1
+    assert_array_almost_equal(resized, ref)
+
+    channel_axis = channel_axis % x.ndim
+    spatial_shape = (10, 10)
+    out_shape = (
+        spatial_shape[:channel_axis] + (nch,) + spatial_shape[channel_axis:]
+    )
+    resized = resize_local_mean(x, out_shape)
+    # move channels back to last axis to match the reference image
+    resized = np.moveaxis(resized, channel_axis, -1)
+    assert_array_almost_equal(resized, ref)
+
+
+def test_resize_local_mean3d_resize():
+    # resize 3rd dimension
+    x = np.zeros((5, 5, 3), dtype=np.double)
+    x[1, 1, :] = 1
+    resized = resize_local_mean(x, (10, 10, 1))
+    ref = np.zeros((10, 10, 1))
+    ref[2:4, 2:4] = 1
+    assert_array_almost_equal(resized, ref)
+
+    # can't resize along specified channel axis
+    with pytest.raises(ValueError):
+        resize_local_mean(x, (10, 10, 1), channel_axis=-1)
+
+
+def test_resize_local_mean3d_2din_3dout():
+    # 3D output with 2D input
+    x = np.zeros((5, 5), dtype=np.double)
+    x[1, 1] = 1
+    resized = resize_local_mean(x, (10, 10, 1))
+    ref = np.zeros((10, 10, 1))
+    ref[2:4, 2:4] = 1
+    assert_array_almost_equal(resized, ref)
+
+
+def test_resize_local_mean2d_4d():
+    # resize with extra output dimensions
+    x = np.zeros((5, 5), dtype=np.double)
+    x[1, 1] = 1
+    out_shape = (10, 10, 1, 1)
+    resized = resize_local_mean(x, out_shape)
+    ref = np.zeros(out_shape)
+    ref[2:4, 2:4, ...] = 1
+    assert_array_almost_equal(resized, ref)
+
+
+@pytest.mark.parametrize("dim", range(1, 6))
+def test_resize_local_mean_nd(dim):
+    shape = 2 + np.arange(dim) * 2
+    x = np.ones(shape)
+    out_shape = (np.asarray(shape) * 1.5).astype(int)
+    resized = resize_local_mean(x, out_shape)
+    expected_shape = 1.5 * shape
+    assert_array_equal(resized.shape, expected_shape)
+    assert_array_equal(resized, 1)
+
+
+def test_resize_local_mean3d():
+    x = np.zeros((5, 5, 2), dtype=np.double)
+    x[1, 1, 0] = 0
+    x[1, 1, 1] = 1
+    resized = resize_local_mean(x, (10, 10, 1))
+    ref = np.zeros((10, 10, 1))
+    ref[2:4, 2:4, :] = 0.5
+    assert_array_almost_equal(resized, ref)
+    resized = resize_local_mean(x, (10, 10, 1), grid_mode=False)
+    ref[1, 1, :] = 0.0703125
+    ref[2, 2, :] = 0.5
+    ref[3, 3, :] = 0.3828125
+    ref[1, 2, :] = ref[2, 1, :] = 0.1875
+    ref[1, 3, :] = ref[3, 1, :] = 0.1640625
+    ref[2, 3, :] = ref[3, 2, :] = 0.4375
+    assert_array_almost_equal(resized, ref)
+
+
+def test_resize_local_mean_dtype():
+    x = np.zeros((5, 5))
+    x_f32 = x.astype(np.float32)
+    x_u8 = x.astype(np.uint8)
+    x_b = x.astype(bool)
+
+    assert resize_local_mean(x, (10, 10),
+                             preserve_range=False).dtype == x.dtype
+    assert resize_local_mean(x, (10, 10),
+                             preserve_range=True).dtype == x.dtype
+    assert resize_local_mean(x_u8, (10, 10),
+                             preserve_range=False).dtype == np.double
+    assert resize_local_mean(x_u8, (10, 10),
+                             preserve_range=True).dtype == np.double
+    assert resize_local_mean(x_b, (10, 10),
+                             preserve_range=False).dtype == np.double
+    assert resize_local_mean(x_b, (10, 10),
+                             preserve_range=True).dtype == np.double
+    assert resize_local_mean(x_f32, (10, 10),
+                             preserve_range=False).dtype == x_f32.dtype
+    assert resize_local_mean(x_f32, (10, 10),
+                             preserve_range=True).dtype == x_f32.dtype