Repository: scikit-image/scikit-image

Issue:
interpolation: mode='constant' edge behaviour
The interpolation functions within **skimage/_shared/interpolation.pxd** treat the `mode='constant'` case  differently than the `NI_GeometricTransform` function defined in **scipy/ndimage/src/ni_interpolation.c** that gets called by functions such as `scipy.ndimage.map_coordinates`.

Specifically, any coordinates outside the range of the image will be assigned a value exactly equal to `cval` in the ndimage implementation.  For the skimage case, however, the values are interpolated between cval and the edge pixels.  I am not sure this is necessarily a bug, but rather an inconsistency with scipy and I was not sure if this is intended behaviour or not.

It is fairly simple to get the scipy-like behaviour with slight modifications.  e.g. in `bilinear_interpolation`, the following can be placed before the bilinear interpolation:

``` python
if (minr < 0) or (maxr >= rows) or (minc < 0) or (maxc >= cols):
    return cval
else:
    ...
```

If you agree that this is a bug, I can make a pull request with solutions similar to the above



Discussion:
That does not sound right to me.  @ahojnnes?

BTW Gregory, where are you located?  @ahojnnes will be here at Berkeley
next week for a sprint.

@stefanv: I am in Cincinnati

@stefanv, @ahojnnes
Here are a couple of concrete examples.  

In the first example below, when order=0, 1, or 3 `rescale` will eventually call the `_warp_fast` function from `skimage.transform._warps_cy.pyx` which in turn calls functions from `_shared/interpolation.pyx`
The order 2, 4, 5 cases, call `scipy.ndimage.map_coordinates` instead.

``` python
import numpy as np
from skimage.transform import rescale
import matplotlib.pyplot as plt
f, ax = plt.subplots(1, 7)
img = np.ones((7, 7))
img[2:5, 2:5] = 0.0
scale_factor = 2.0
ax[0].imshow(img)
# ax[0].set_axis_off()
ax[0].set_title('original')
for order in range(7):
    ax[order+1].imshow(rescale(img, scale_factor, order=order, mode='constant',
                               preserve_range=False),
                       vmin=0, vmax=1)
    # ax[order+1].set_axis_off()
    ax[order+1].set_title('order={}'.format(order))
```

![interp_example2](https://cloud.githubusercontent.com/assets/6528957/7710633/31047ec0-fe33-11e4-9fd9-864e33deb0ee.png)
Here the first and last coordinates end up outside the original range and are set to cval for the orders that call scipy, but are interpolated using a pixel value of cval in the skimage cases.

Also, strangely, if the image is all ones, the interpolation is no longer apparent at the edges for order = 1 or 3.  See below:

``` python
f, ax = plt.subplots(1, 6)
img = np.ones((4, 4))
img[3:5, 3:5] = 0.0
scale_factor = 2.0
for order in range(6):
    ax[order].imshow(rescale(img, scale_factor, order=order, mode='constant'),
                     vmin=0, vmax=1)
    ax[order].set_axis_off()
    ax[order].set_title('order={}'.format(order))
```

![interp_example](https://cloud.githubusercontent.com/assets/6528957/7709526/aba392ea-fe2b-11e4-9e3a-7dc8ac0791dd.png)

In this case, the coordinates that are getting passed to `map_coordinates` along each axis are:
[-0.25,  0.25,  0.75,  1.25,  1.75,  2.25,  2.75,  3.25]
 -0.25 and 3.25 are outside the original range of [0, 1, 2, 3], and get mapped to cval=0

@rgommers @jaimefrio As reported here, there is an inconsistency with scipy w.r.t interpolation around image edges.  When we hit a coordinate outside of the image and mode is set to 'constant' with `cval` 0, then we simply take that point to be zero and complete the interpolation.  Scipy, on the other hand, does not even try to complete the interpolation, but immediately returns `cval`.  Should this be considered a bug or a feature on the scipy end?  And if a feature, would it be feasible to also add a mode that does the same thing scikit-image currently does?

@stefanv the `ndimage.map_coordinates` behavior is what I'd expect (maybe because I'm used to it). Also, I'm not even sure what "complete the interpolation" even means. If you've filled the actual point you're interpolating to with `cval`, then any order interpolation still should yield `cval` as the final value. The figure just above looks like `cval` is completely ignored, but I guess I'm missing something.

Maybe you can make a concrete proposal for the string value to add to `mode` and what it should do? 

Perhaps this sketch will help to identify the issue:

![skimage_interpolation](https://cloud.githubusercontent.com/assets/45071/7762872/9caa80c8-ffed-11e4-9530-a03ab8fc26ef.png)

Red == known values, black == cval

@stefanv @rgommers an alternate example: `image = np.ones((2, 2))`. [A pixel is not a little square](http://alvyray.com/Memos/CG/Microsoft/6_pixel.pdf), but rather a point sample at integer coordinates. So @stefanv argues that `cval=0` means that the samples extending the grid e.g. at (-1, 0) and (-1, 1), are taken to be equal to 0, but a sample at (-0.5, 0.5) needs to be interpolated. Using a linear interpolation, that sample's value would be equal to 0.5: the average of 1 at (0, 0), 1 at (0, 1), 0 at (-1, 0) and 0 at (-1, 1).

However, `map_coordinates` returns cval in this case, regardless of the order of the interpolation.

It's unclear to me what the "correct" behaviour should be, but smoothness arguments would favour @stefanv's expectation...

@stefanv, @jni, @rgommers 
I thought about this some more and I think using `cval` in the interpolation is technically correct for warping to coordinates outside the image range. As a second point of agreement, I get the same result as in scikit.image's linear interpolation when using OpenCV's remap function:

``` python
import numpy as np
import cv2
img = np.ones((7, 7))
img[2:5, 2:5] = 0.0
scale_factor = 2.0
output_shape = scale_factor * np.asarray(img.shape)
# destination coordinates
axis_slices = tuple([slice(d) for d in output_shape])
nd_coords = np.mgrid[axis_slices]
map2_coord_arrays = [1/scale_factor * (nd_coords[i] + 0.5) - 0.5
                     for i, d in enumerate(output_shape)]
cv2_map = np.concatenate((map2_coord_arrays[0][..., np.newaxis],
                          map2_coord_arrays[1][..., np.newaxis]), axis=2)
img_cv2_remap = cv2.remap(img,
                          map1=cv2_map.astype(np.float32),
                          map2=None,
                          interpolation=cv2.INTER_LINEAR,
                          borderMode=cv2.BORDER_CONSTANT,
                          borderValue=0.)
f, ax = plt.subplots(1, 2)
ax[0].imshow(img)
ax[0].set_title('original')
ax[1].imshow(img_cv2_remap)
ax[1].set_title('opencv remap')
```

![opencv_result](https://cloud.githubusercontent.com/assets/6528957/7779580/be198702-00a3-11e5-80ad-7eb40457e5d7.png)

Perhaps a simple solution to avoid potentially confusing edge results like the above during simple rescalings would be to change the default `mode` for `resize` and `rescale` to `reflect`.  In that mode, regardless which underlying interpolation is used, the results should be the same.
The result for 'reflect' intuitively looks more like one would expect for upscaling the source by two:
![opencv_reflect](https://cloud.githubusercontent.com/assets/6528957/7779668/73f3f620-00a4-11e5-815b-aee114e1e908.png)

As another OpenCV comparison:
OpenCV has a resize() function, which does not have an option to specify the border behavior.  However, in practice it seems that for most interpolation types, any coordinates that fall outside the image range get clipped to the source coordinate range  (e.g. see here (where `sx` is the floor of the destination x coordinate and gets clipped to [0, width -1] for most interpolation types)
https://github.com/Itseez/opencv/blob/8e392251baf353c7bbcf23d3c96a9f18dde3230d/modules/imgproc/src/imgwarp.cpp#L3392-3405
In practice I think that should be equivalent to `mode='reflect'`

I agree with @grlee77 - I would use `'reflect'` here and reasonably expect a simple rescaling algorithm to default to that behavior.

@JDWarner, @ahojnnes :  I would like to put a PR to use mode `symmetric` (the `np.pad` equivalent name for `reflect` to take into account the naming conventions decided on in #1583) as the default for `resize`.  This should also fix the specific example shown in #1585 (although due to a bug in `ndimage` the mode `wrap` could still give an inconsistent result between 2D and 3D cases).

The only down side I see to the change is that this will change results for any users that did not previously explicitly specify the mode argument.  Is there a way to raise a deprecation for the default 'constant' only in the case that the user didn't explicitly pass `mode='constant'`?  (i.e. can we detect if  mode was passed as an argument vs. just getting set to the default)

Generally, we deprecate improvements for two minor point releases. However,
we immediately fix and document bugs which require API changes.

Looking back at #1585, I'm inclined to consider this a bug and recommend a
breaking fix.

On Friday, August 14, 2015, Gregory R. Lee notifications@github.com wrote:

> @JDWarner https://github.com/JDWarner, @ahojnnes
> https://github.com/ahojnnes : I would like to put a PR to use mode
> symmetric (the np.pad equivalent name for reflect to take into account
> the naming conventions decided on in #1583
> https://github.com/scikit-image/scikit-image/pull/1583) as the default
> for resize. This should also fix the specific example shown in #1585
> https://github.com/scikit-image/scikit-image/issues/1585 (although due
> to a bug in ndimage the mode wrap could still give an inconsistent result
> between 2D and 3D cases).
> 
> The only down side I see to the change is that this will change results
> for any users that did not previously explicitly specify the mode argument.
> Is there a way to raise a deprecation for the default 'constant' only in
> the case that the user didn't explicitly pass mode='constant'? (i.e. can
> we detect if mode was passed as an argument vs. just getting set to the
> default)
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/scikit-image/scikit-image/issues/1514#issuecomment-131146841
> .

How about making it `mode=None`, then:

```
if mode is None:
    mode = "constant" or "reflect"
    print deprecation/breaking fix warning
```

On Fri, Aug 14, 2015 at 8:55 AM, Josh Warner notifications@github.com
wrote:

> Generally, we deprecate improvements for two minor point releases. However,
> we immediately fix and document bugs which require API changes.
> 
> Looking back at #1585, I'm inclined to consider this a bug and recommend a
> breaking fix.
> 
> On Friday, August 14, 2015, Gregory R. Lee notifications@github.com
> wrote:
> 
> > @JDWarner https://github.com/JDWarner, @ahojnnes
> > https://github.com/ahojnnes : I would like to put a PR to use mode
> > symmetric (the np.pad equivalent name for reflect to take into account
> > the naming conventions decided on in #1583
> > https://github.com/scikit-image/scikit-image/pull/1583) as the default
> > for resize. This should also fix the specific example shown in #1585
> > https://github.com/scikit-image/scikit-image/issues/1585 (although due
> > to a bug in ndimage the mode wrap could still give an inconsistent result
> > between 2D and 3D cases).
> > 
> > The only down side I see to the change is that this will change results
> > for any users that did not previously explicitly specify the mode
> > argument.
> > Is there a way to raise a deprecation for the default 'constant' only in
> > the case that the user didn't explicitly pass mode='constant'? (i.e. can
> > we detect if mode was passed as an argument vs. just getting set to the
> > default)
> > 
> > —
> > Reply to this email directly or view it on GitHub
> > <
> > https://github.com/scikit-image/scikit-image/issues/1514#issuecomment-131146841
> > 
> > .
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/scikit-image/scikit-image/issues/1514#issuecomment-131159828
> .

:+1: for this approach. Warn the user the default changed in 0.12, and how
to restore the previous behavior.
On Aug 14, 2015 11:31 AM, "Johannes Schönberger" notifications@github.com
wrote:

> How about making it `mode=None`, then:
> 
> ```
> if mode is None:
> mode = "constant" or "reflect"
> print deprecation/breaking fix warning
> ```
> 
> On Fri, Aug 14, 2015 at 8:55 AM, Josh Warner notifications@github.com
> wrote:
> 
> > Generally, we deprecate improvements for two minor point releases.
> > However,
> > we immediately fix and document bugs which require API changes.
> > 
> > Looking back at #1585, I'm inclined to consider this a bug and recommend
> > a
> > breaking fix.
> > 
> > On Friday, August 14, 2015, Gregory R. Lee notifications@github.com
> > wrote:
> > 
> > > @JDWarner https://github.com/JDWarner, @ahojnnes
> > > https://github.com/ahojnnes : I would like to put a PR to use mode
> > > symmetric (the np.pad equivalent name for reflect to take into account
> > > the naming conventions decided on in #1583
> > > https://github.com/scikit-image/scikit-image/pull/1583) as the
> > > default
> > > for resize. This should also fix the specific example shown in #1585
> > > https://github.com/scikit-image/scikit-image/issues/1585 (although
> > > due
> > > to a bug in ndimage the mode wrap could still give an inconsistent
> > > result
> > > between 2D and 3D cases).
> > > 
> > > The only down side I see to the change is that this will change results
> > > for any users that did not previously explicitly specify the mode
> > > argument.
> > > Is there a way to raise a deprecation for the default 'constant' only
> > > in
> > > the case that the user didn't explicitly pass mode='constant'? (i.e.
> > > can
> > > we detect if mode was passed as an argument vs. just getting set to the
> > > default)
> > > 
> > > —
> > > Reply to this email directly or view it on GitHub
> > > <
> > 
> > https://github.com/scikit-image/scikit-image/issues/1514#issuecomment-131146841
> > 
> > > .
> > 
> > —
> > Reply to this email directly or view it on GitHub
> > <
> > https://github.com/scikit-image/scikit-image/issues/1514#issuecomment-131159828
> > 
> > .
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/scikit-image/scikit-image/issues/1514#issuecomment-131171330
> .

I will take that approach.  thanks


Resolving patch:
diff --git a/doc/examples/transform/plot_radon_transform.py b/doc/examples/transform/plot_radon_transform.py
--- a/doc/examples/transform/plot_radon_transform.py
+++ b/doc/examples/transform/plot_radon_transform.py
@@ -68,7 +68,7 @@
 from skimage.transform import radon, rescale
 
 image = imread(data_dir + "/phantom.png", as_grey=True)
-image = rescale(image, scale=0.4)
+image = rescale(image, scale=0.4, mode='reflect')
 
 fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(8, 4.5))
 
diff --git a/doc/examples/transform/plot_seam_carving.py b/doc/examples/transform/plot_seam_carving.py
--- a/doc/examples/transform/plot_seam_carving.py
+++ b/doc/examples/transform/plot_seam_carving.py
@@ -32,7 +32,8 @@
 
 ######################################################################
 
-resized = transform.resize(img, (img.shape[0], img.shape[1] - 200))
+resized = transform.resize(img, (img.shape[0], img.shape[1] - 200),
+                           mode='reflect')
 plt.figure()
 plt.title('Resized Image')
 plt.imshow(resized)
@@ -77,7 +78,7 @@
 plt.figure()
 plt.title('Object Removed')
 out = transform.seam_carve(img, eimg, 'vertical', 90)
-resized = transform.resize(img, out.shape)
+resized = transform.resize(img, out.shape, mode='reflect')
 plt.imshow(out)
 plt.show()
 
diff --git a/skimage/transform/_warps.py b/skimage/transform/_warps.py
--- a/skimage/transform/_warps.py
+++ b/skimage/transform/_warps.py
@@ -18,7 +18,7 @@
 )
 
 
-def resize(image, output_shape, order=1, mode='constant', cval=0, clip=True,
+def resize(image, output_shape, order=1, mode=None, cval=0, clip=True,
            preserve_range=False):
     """Resize image to match a certain size.
 
@@ -49,7 +49,8 @@ def resize(image, output_shape, order=1, mode='constant', cval=0, clip=True,
         be in the range 0-5. See `skimage.transform.warp` for detail.
     mode : {'constant', 'edge', 'symmetric', 'reflect', 'wrap'}, optional
         Points outside the boundaries of the input are filled according
-        to the given mode.  Modes match the behaviour of `numpy.pad`.
+        to the given mode.  Modes match the behaviour of `numpy.pad`.  The
+        default mode is 'constant'.
     cval : float, optional
         Used in conjunction with mode 'constant', the value outside
         the image boundaries.
@@ -74,10 +75,14 @@ def resize(image, output_shape, order=1, mode='constant', cval=0, clip=True,
     >>> from skimage import data
     >>> from skimage.transform import resize
     >>> image = data.camera()
-    >>> resize(image, (100, 100)).shape
+    >>> resize(image, (100, 100), mode='reflect').shape
     (100, 100)
 
     """
+    if mode is None:
+        mode = 'constant'
+        warn("The default mode, 'constant', will be changed to 'reflect' in "
+             "skimage 0.15.")
 
     rows, cols = output_shape[0], output_shape[1]
     orig_rows, orig_cols = image.shape[0], image.shape[1]
@@ -132,7 +137,7 @@ def resize(image, output_shape, order=1, mode='constant', cval=0, clip=True,
     return out
 
 
-def rescale(image, scale, order=1, mode='constant', cval=0, clip=True,
+def rescale(image, scale, order=1, mode=None, cval=0, clip=True,
             preserve_range=False):
     """Scale image by a certain factor.
 
@@ -161,7 +166,8 @@ def rescale(image, scale, order=1, mode='constant', cval=0, clip=True,
         be in the range 0-5. See `skimage.transform.warp` for detail.
     mode : {'constant', 'edge', 'symmetric', 'reflect', 'wrap'}, optional
         Points outside the boundaries of the input are filled according
-        to the given mode.  Modes match the behaviour of `numpy.pad`.
+        to the given mode.  Modes match the behaviour of `numpy.pad`.  The
+        default mode is 'constant'.
     cval : float, optional
         Used in conjunction with mode 'constant', the value outside
         the image boundaries.
@@ -178,9 +184,9 @@ def rescale(image, scale, order=1, mode='constant', cval=0, clip=True,
     >>> from skimage import data
     >>> from skimage.transform import rescale
     >>> image = data.camera()
-    >>> rescale(image, 0.1).shape
+    >>> rescale(image, 0.1, mode='reflect').shape
     (51, 51)
-    >>> rescale(image, 0.5).shape
+    >>> rescale(image, 0.5, mode='reflect').shape
     (256, 256)
 
     """


Test patch:
diff --git a/skimage/transform/tests/test_radon_transform.py b/skimage/transform/tests/test_radon_transform.py
--- a/skimage/transform/tests/test_radon_transform.py
+++ b/skimage/transform/tests/test_radon_transform.py
@@ -13,7 +13,7 @@
 
 PHANTOM = imread(os.path.join(data_dir, "phantom.png"),
                    as_grey=True)[::2, ::2]
-PHANTOM = rescale(PHANTOM, 0.5, order=1)
+PHANTOM = rescale(PHANTOM, 0.5, order=1, mode='reflect')
 
 
 def _debug_plot(original, result, sinogram=None):
@@ -315,7 +315,7 @@ def test_order_angles_golden_ratio():
 def test_iradon_sart():
     debug = False
 
-    image = rescale(PHANTOM, 0.8)
+    image = rescale(PHANTOM, 0.8, mode='reflect')
     theta_ordered = np.linspace(0., 180., image.shape[0], endpoint=False)
     theta_missing_wedge = np.linspace(0., 150., image.shape[0], endpoint=True)
     for theta, error_factor in ((theta_ordered, 1.),
diff --git a/skimage/transform/tests/test_warps.py b/skimage/transform/tests/test_warps.py
--- a/skimage/transform/tests/test_warps.py
+++ b/skimage/transform/tests/test_warps.py
@@ -91,10 +91,12 @@ def test_warp_clip():
     x = np.zeros((5, 5), dtype=np.double)
     x[2, 2] = 1
 
-    outx = rescale(x, 3, order=3, clip=False)
+    with expected_warnings(['The default mode']):
+        outx = rescale(x, 3, order=3, clip=False)
     assert outx.min() < 0
 
-    outx = rescale(x, 3, order=3, clip=True)
+    with expected_warnings(['The default mode']):
+        outx = rescale(x, 3, order=3, clip=True)
     assert_almost_equal(outx.min(), 0)
     assert_almost_equal(outx.max(), 1)
 
@@ -160,7 +162,8 @@ def test_rescale():
     # same scale factor
     x = np.zeros((5, 5), dtype=np.double)
     x[1, 1] = 1
-    scaled = rescale(x, 2, order=0)
+    with expected_warnings(['The default mode']):
+        scaled = rescale(x, 2, order=0)
     ref = np.zeros((10, 10))
     ref[2:4, 2:4] = 1
     assert_almost_equal(scaled, ref)
@@ -168,7 +171,8 @@ def test_rescale():
     # different scale factors
     x = np.zeros((5, 5), dtype=np.double)
     x[1, 1] = 1
-    scaled = rescale(x, (2, 1), order=0)
+    with expected_warnings(['The default mode']):
+        scaled = rescale(x, (2, 1), order=0)
     ref = np.zeros((10, 5))
     ref[2:4, 1] = 1
     assert_almost_equal(scaled, ref)
@@ -177,7 +181,8 @@ def test_rescale():
 def test_resize2d():
     x = np.zeros((5, 5), dtype=np.double)
     x[1, 1] = 1
-    resized = resize(x, (10, 10), order=0)
+    with expected_warnings(['The default mode']):
+        resized = resize(x, (10, 10), order=0)
     ref = np.zeros((10, 10))
     ref[2:4, 2:4] = 1
     assert_almost_equal(resized, ref)
@@ -187,11 +192,13 @@ def test_resize3d_keep():
     # keep 3rd dimension
     x = np.zeros((5, 5, 3), dtype=np.double)
     x[1, 1, :] = 1
-    resized = resize(x, (10, 10), order=0)
+    with expected_warnings(['The default mode']):
+        resized = resize(x, (10, 10), order=0)
     ref = np.zeros((10, 10, 3))
     ref[2:4, 2:4, :] = 1
     assert_almost_equal(resized, ref)
-    resized = resize(x, (10, 10, 3), order=0)
+    with expected_warnings(['The default mode']):
+        resized = resize(x, (10, 10, 3), order=0)
     assert_almost_equal(resized, ref)
 
 
@@ -199,7 +206,8 @@ def test_resize3d_resize():
     # resize 3rd dimension
     x = np.zeros((5, 5, 3), dtype=np.double)
     x[1, 1, :] = 1
-    resized = resize(x, (10, 10, 1), order=0)
+    with expected_warnings(['The default mode']):
+        resized = resize(x, (10, 10, 1), order=0)
     ref = np.zeros((10, 10, 1))
     ref[2:4, 2:4] = 1
     assert_almost_equal(resized, ref)
@@ -209,7 +217,8 @@ def test_resize3d_2din_3dout():
     # 3D output with 2D input
     x = np.zeros((5, 5), dtype=np.double)
     x[1, 1] = 1
-    resized = resize(x, (10, 10, 1), order=0)
+    with expected_warnings(['The default mode']):
+        resized = resize(x, (10, 10, 1), order=0)
     ref = np.zeros((10, 10, 1))
     ref[2:4, 2:4] = 1
     assert_almost_equal(resized, ref)
@@ -220,7 +229,7 @@ def test_resize3d_bilinear():
     x = np.zeros((5, 5, 2), dtype=np.double)
     x[1, 1, 0] = 0
     x[1, 1, 1] = 1
-    resized = resize(x, (10, 10, 1), order=1)
+    resized = resize(x, (10, 10, 1), order=1, mode='constant')
     ref = np.zeros((10, 10, 1))
     ref[1:5, 1:5, :] = 0.03125
     ref[1:5, 2:4, :] = 0.09375
@@ -316,16 +325,19 @@ def test_slow_warp_nonint_oshape():
 def test_keep_range():
     image = np.linspace(0, 2, 25).reshape(5, 5)
 
-    out = rescale(image, 2, preserve_range=False, clip=True, order=0)
+    with expected_warnings(['The default mode']):
+        out = rescale(image, 2, preserve_range=False, clip=True, order=0)
     assert out.min() == 0
     assert out.max() == 2
 
-    out = rescale(image, 2, preserve_range=True, clip=True, order=0)
+    with expected_warnings(['The default mode']):
+        out = rescale(image, 2, preserve_range=True, clip=True, order=0)
     assert out.min() == 0
     assert out.max() == 2
 
-    out = rescale(image.astype(np.uint8), 2, preserve_range=False,
-                  clip=True, order=0)
+    with expected_warnings(['The default mode']):
+        out = rescale(image.astype(np.uint8), 2, preserve_range=False,
+                      clip=True, order=0)
     assert out.min() == 0
     assert out.max() == 2 / 255.0