Repository: scikit-image/scikit-image

Issue:
Nearest neighbor resize of integer array changes the unique integer values
## Description

<!--
(Note: for guidance on how to use `scikit-image`, please post instead on https://forum.image.sc/tag/scikit-image)
-->

When downscale integer array, using `order=0` still can cause the result to contain integers that do not exist in the original array. I know that `skimage` would move everything to float and scale to 0~1 when `resize`. But I still don't understand why nearest neighbor would lead to this behavior. This behavior means it cannot be used to manipulate arrays representing instance mask.

## Way to reproduce
```python
arr = np.zeros((12,12), dtype=np.int16)
arr[4:8, 1:4] = 5
arr[4:8, 7:10] = 7

resized = skimage.transform.resize(
    arr,
    (8, 8),
    mode="constant",
    preserve_range=True,
    order=0,
).astype(np.int16)
print(f"original arr:\n {arr}")
print(f"resized arr:\n {resized}")
```
And the output is
```python
original arr:
 [[0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0]
 [0 5 5 5 0 0 0 7 7 7 0 0]
 [0 5 5 5 0 0 0 7 7 7 0 0]
 [0 5 5 5 0 0 0 7 7 7 0 0]
 [0 5 5 5 0 0 0 7 7 7 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0 0 0]]
resized arr:
 [[0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]
 [0 5 4 0 0 7 6 0]
 [0 5 4 0 0 7 6 0]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0]]
```


## Version information
```python
# Paste the output of the following python commands
from __future__ import print_function
import sys; print(sys.version)
import platform; print(platform.platform())
import skimage; print(f'scikit-image version: {skimage.__version__}')
import numpy; print(f'numpy version: {numpy.__version__}')
```

```python
3.9.7 (default, Sep 16 2021, 13:09:58) 
[GCC 7.5.0]
Linux-4.14.275-142.503.amzn1.x86_64-x86_64-with-glibc2.17
scikit-image version: 0.19.2
numpy version: 1.21.5
```



Discussion:
upon further investigation, looks like this might be due to the fact that anti aliasing is default to be true and it causes the change. On one hand it makes sense to me how this happened. On the other hand personally I feel like nearest neighbor should be prioritized over anti aliasing, or at least default to no anti aliasing like when data is boolean. I think the current behavior is not obvious and can be confusing. 

On a side note, tensorflow behaves differently
```python
resized = tf.image.resize(
    arr[np.newaxis,:,:,np.newaxis],
    size=(8,8),
    method=tf.image.ResizeMethod.NEAREST_NEIGHBOR,
    antialias=True
)[0,:,:,0]
print(f"{resized=}")
```

```
resized=<tf.Tensor: shape=(8, 8), dtype=int16, numpy=
array([[0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 5, 5, 0, 0, 7, 7, 0],
       [0, 5, 5, 0, 0, 7, 7, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0]], dtype=int16)>
```
Thank you for reporting this and sorry for the delayed response! I can reproduce this with current `main`. I'm yet a little unclear what precisely you want changed...

> looks like this might be due to the fact that anti aliasing is default to be true

Yes, the smoothing seems to be the source of the pixel value 4. 

> On the other hand personally I feel like nearest neighbor should be prioritized over anti aliasing, or at least default to no anti aliasing like when data is boolean.

What do you mean by prioritizing nearest neighbor over anti-aliasing? As far as I'm aware this function does not use nearest neighbor interpolation. For scipy >= 1.6.0, spline interpolation in [scipy.ndimage.zoom](https://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.zoom.html) is used.

Also for boolean images anti-aliasing already defaults to no anti aliasing.

> tensorflow behaves differently

According to [tf.image.resize's documentation](https://www.tensorflow.org/api_docs/python/tf/image/resize),  "antialias has no effect when used with nearest neighbor interpolation." That could explain why this does not happen in tensorflow.

> This behavior means it cannot be used to manipulate arrays representing instance mask.

Not sure what you mean by instance mask, but if you want to preserve integer values I would expect that anti-aliasing has to be turned of explicitly. What would you like to see changed about the current behavior?
Thank you for your response! I should have been more clear about my suggested behavior.

My main issue is with the fact that anti aliasing is default to be true when `order==0`, and it affects the result. In the doc, `order=0` is explained as nearest-neighbor. My opinion is that user expects unique values to stay the same when using nearest-neighbor.

> Also for boolean images anti-aliasing already defaults to no anti aliasing.

Exactly! For boolean images, anti-aliasing defaults to be false makes sense. Similarly, I think anti-aliasing default to be false makes sense when use nearest-neighbor. Because when people use nearest-neighbor, usually it means this is no such thing as "visual artifact".

> According to [tf.image.resize's documentation](https://www.tensorflow.org/api_docs/python/tf/image/resize), "antialias has no effect when used with nearest neighbor interpolation." That could explain why this does not happen in tensorflow.

Exactly! Personally I agree with behavior of `tensorflow`, i.e., anti-aliasing doesn't have any effect for nearest-neighbor even when it is turned on specifically. Because again, aliasing is usually not a concern when use nearest-neighbor as the method. But I am ok with it to be default false as I mentioned above.

> Not sure what you mean by instance mask, but if you want to preserve integer values I would expect that anti-aliasing has to be turned of explicitly. What would you like to see changed about the current behavior?

By instance mask, what I mean is when a mask is used to represent instance segmentation. A binary mask (boolean mask) can be used to represent semantic segmentation. An array with a bunch of integer values can be used to represent instances of instance segmentation: 0 represent background, and positive integer represents each object. So if a number 4 somehow appears next to the region of 5, it represents a pixel of a different object is there.

So my main opinion is: when people choose to use nearest-neighbor as the interpolation method, they usually don't care about the discontinuity or discretization. And aliasing is usually not a concern. I advocate for behavior same as tensorflow, which is to completely disable anti-aliasing for nearest neighbor. But I am also OK with making anti-aliasing default to be false when using nearest-neighbor. I believe this is the most common and natural use case, and it is the expected behavior.
> My main issue is with the fact that anti aliasing is default to be true when order==0, and it affects the result. In the doc, order=0 is explained as nearest-neighbor.

Ah, that's true. Sorry I didn't pay attention to the `order` part. There's indeed something to clarify fix here. In #5173 the implementation does no longer use `warp` under the hood if scipy>=1.6.0. @grlee77, does the reference to `warp` in [skimage.transform.resize's docstring](https://scikit-image.org/docs/dev/api/skimage.transform.html?highlight=resize#skimage.transform.resize) still make sense? I can't find a reference to "nearest neighbors" in [scipy's zoom](https://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.zoom.html).
> reference to warp in skimage.transform.resize's docstring still make sense? I can't find a reference to "nearest neighbors" in scipy's zoom

For SciPy and scikit-image `order=0` means nearest neighbor. I agree that it would be sensible to switch to not doing anti-aliasing in that case for the reason described.
Would we need a deprecation cycle for this switch? This would subtly change the behavior of for users using `order=0` without setting `anti_aliasing=False` explicitly. So rather something for skimage2?
This may be considered as a bug as `anti_aliasing` default value is not appropriate in the case described in this issue...
I will open a PR to manage this :wink: 

Resolving patch:
diff --git a/skimage/transform/_warps.py b/skimage/transform/_warps.py
--- a/skimage/transform/_warps.py
+++ b/skimage/transform/_warps.py
@@ -118,6 +118,8 @@ def resize(image, output_shape, order=None, mode='reflect', cval=0, clip=True,
         to downsampling. It is crucial to filter when downsampling
         the image to avoid aliasing artifacts. If not specified, it is set to
         True when downsampling an image whose data type is not bool.
+        It is also set to False when using nearest neighbor interpolation
+        (``order`` == 0) with integer input data type.
     anti_aliasing_sigma : {float, tuple of floats}, optional
         Standard deviation for Gaussian filtering used when anti-aliasing.
         By default, this value is chosen as (s - 1) / 2 where s is the
@@ -150,8 +152,10 @@ def resize(image, output_shape, order=None, mode='reflect', cval=0, clip=True,
         image = image.astype(np.float32)
 
     if anti_aliasing is None:
-        anti_aliasing = (not input_type == bool and
-                         any(x < y for x, y in zip(output_shape, input_shape)))
+        anti_aliasing = (
+            not input_type == bool and
+            not (np.issubdtype(input_type, np.integer) and order == 0) and
+            any(x < y for x, y in zip(output_shape, input_shape)))
 
     if input_type == bool and anti_aliasing:
         raise ValueError("anti_aliasing must be False for boolean images")


Test patch:
diff --git a/skimage/transform/tests/test_warps.py b/skimage/transform/tests/test_warps.py
--- a/skimage/transform/tests/test_warps.py
+++ b/skimage/transform/tests/test_warps.py
@@ -986,6 +986,17 @@ def test_resize_local_mean_dtype():
                              preserve_range=True).dtype == x_f32.dtype
 
 
+def test_nn_resize_int_img():
+    """Issue #6467"""
+    img = np.zeros((12, 12), dtype=np.int16)
+    img[4:8, 1:4] = 5
+    img[4:8, 7:10] = 7
+
+    resized = resize(img, (8, 8), order=0)
+
+    assert np.array_equal(np.unique(resized), np.unique(img))
+
+
 @pytest.mark.parametrize("_type", [tuple, np.asarray, list])
 def test_output_shape_arg_type(_type):
     img = np.random.rand(3, 3)