Repository: scikit-image/scikit-image

Issue:
New canny implementation silently fails with integer images.
### Description:

The new `skimage.feature.canny` implementation silently fails if given an integer image.  This worked on `scikit-image<=0.19`, and no longer works with `scikit-image=0.20`.  The documentation says that any dtype should work:
```
    image : 2D array
        Grayscale input image to detect edges on; can be of any dtype.
```


### Way to reproduce:

```
from skimage.feature import canny
import numpy as np

im = np.zeros((100, 100))
im[0: 50, 0: 50] = 1.0
print("Edge pixels with float input: ", canny(im, low_threshold=0, high_threshold=1).sum())
print("Edge pixels with int input: ", canny(im.astype(np.int64), low_threshold=0, high_threshold=1).sum())
```
This prints on new skimage (0.20):
```
Edge pixels with float input:  182
Edge pixels with int input:  0
```
And on old skimage (0.19):
```
Edge pixels with float input:  144
Edge pixels with int input:  144
```

As I write this test case I also need to ask ... why did the number of pixels change?

### Version information:

```Shell
3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:08:06) [GCC 11.3.0]
Linux-3.10.0-1160.88.1.el7.x86_64-x86_64-with-glibc2.17
scikit-image version: 0.20.0
numpy version: 1.23.5
```



Discussion:
Yikes, thanks for catching this!

On latest skimage I see:

```
Edge pixels with float input:  144
Edge pixels with int input:  0
```

So, at least the float version seems OK.
This seems to be the cause:

```
        low_threshold /= dtype_max
```

The correct comparison is probably:

```
from skimage.feature import canny
import numpy as np

im = np.zeros((100, 100))
im[0: 50, 0: 50] = 1.0
print("Edge pixels with float input: ", canny(im, low_threshold=0, high_threshold=1).sum())
print("Edge pixels with int input: ", canny((im * 255).astype(np.uint8), low_threshold=0, high_threshold=1).sum())
```

I think we should clarify what `low_threshold` and `high_threshold` are in the docstring too.
Note that with `(im * 255).astype(np.uint8)` as input, we already achieve the same result in v0.20 as with v0.19.

@rfezzani Could you confirm whether `low_threshold /= dtype_max` is wanted and necessary? We'll probably have to take this out for skimage v2 anyway, since we cannot rely on the dtype being meaningful.

It seems that the division by `dtype_max` was introduced in this [commit](https://github.com/scikit-image/scikit-image/commit/c97b1b6c974da5ed01e09ab257322e7096ad1aaa) after merging #2983. It was proposed as a fix for #2981.
This issue reminds me the #3009 and #4442 issues. It at least needs fix/clarification/warning in the documentation as pointed out by @erykoff .
I don't think the division by `dtype_max` is causing the bug... The following script gives
```python
import skimage
from skimage.feature import canny
import numpy as np

im = np.zeros((100, 100))
im[:50, :50] = 1

print(f"{skimage.__version__=}")

for dtype in ["float64", "float32",
              "uint8", "uint16", "uint32", "uint64",
              "int8", "int16", "int32", "int64"]:
    print(f"Edge pixels count with {dtype} input: ",
          np.count_nonzero(canny(im.astype(dtype),
                                 low_threshold=0, high_threshold=1)))
```
Gives 

### v0.19

```
skimage.__version__='0.19.0'
Edge pixels count with float64 input:  230
Edge pixels count with float32 input:  274
Edge pixels count with uint8 input:  190
Edge pixels count with uint16 input:  230
Edge pixels count with uint32 input:  230
Edge pixels count with uint64 input:  230
Edge pixels count with int8 input:  270
Edge pixels count with int16 input:  230
Edge pixels count with int32 input:  270
Edge pixels count with int64 input:  230
```

### v0.20

```
Edge pixels count with float64 input:  144
Edge pixels count with float32 input:  183
Edge pixels count with uint8 input:  98
Edge pixels count with uint16 input:  102
Edge pixels count with uint32 input:  142
Edge pixels count with uint64 input:  0
Edge pixels count with int8 input:  182
Edge pixels count with int16 input:  103
Edge pixels count with int32 input:  105
Edge pixels count with int64 input:  0
```

I don't understand why I see so many different values comparing with @stefanv and @erykoff  results :/
My results correspond with yours, @rfezzani.
The reason I suspected the dtype handling of being at fault, is because the magnitude of the edges are going to be very small when you input a value of `1` in an int64 image. Either way, it's clear that something's changed between the two versions.
I'm actually wondering how we were using int64 ever worked given the scaling.  And it's certainly surprising (to me!) that the floats get scaled relative to 1.0 and the ints get scaled to int_max.

But the change in numbers of pixels between 0.19/0.20 _and_ float32/float64 is even more concerning. 
The problem originated in 376d30f5ba83e98d2745622d9d4951914e0c91f7, #6348
OK, found the problem: it's related to `compute_bleedover`: before that was not done for an unmasked image, and now it's used.
```
        smoothed_image /= bleed_over
```
We apply a Gaussian filter to the mask to estimate bleed-over, but we set `mode='constant'`, which means that instead of returning mostly 1, the bleed-over varies a lot on the edges.
So, to get results compatible with previous versions, we should skip bleed_over calculation when no mask is given.

*Ideally*, however, computing bleedover should not affect the result; but I don't know what bleedover is, so I can't comment on that.

The closest I can get is to change the bleedover calculation not to use `mode='constant'`:

```python
        bleed_over = gaussian(mask.astype(float_type, copy=False), sigma=1) + np.finfo(float_type).eps
```

This does not resolve the issues with int64, btw. Even removing the range normalization, that thresholding still fails.
```patch
diff --git a/skimage/feature/_canny.py b/skimage/feature/_canny.py
index 89daa09b7..8647410e5 100644
--- a/skimage/feature/_canny.py
+++ b/skimage/feature/_canny.py
@@ -57,7 +57,7 @@ def _preprocess(image, mask, sigma, mode, cval):
     """
     gaussian_kwargs = dict(sigma=sigma, mode=mode, cval=cval,
                            preserve_range=False)
-    compute_bleedover = (mode == 'constant' or mask is not None)
+    compute_bleedover = mask is not None
     float_type = _supported_float_type(image.dtype)
     if mask is None:
         if compute_bleedover:
```

Yields:

```
Edge pixels count with float64 input:  230
Edge pixels count with float32 input:  270
Edge pixels count with uint8 input:  190
Edge pixels count with uint16 input:  230
Edge pixels count with uint32 input:  230
Edge pixels count with uint64 input:  0
Edge pixels count with int8 input:  270
Edge pixels count with int16 input:  230
Edge pixels count with int32 input:  270
```
Re: difference between `float32` and `float64`, looks like the dtype influences the calculation of the smoothed (Gaussian filtered) image, which then later impacts gradients etc.
I can reproduce the results in https://github.com/scikit-image/scikit-image/issues/6871#issuecomment-1499595076. Though, I think you omitted the last line for int64, which is still 0 and not 230 like in v0.19 (see https://github.com/scikit-image/scikit-image/issues/6871#issuecomment-1499262279):

```
skimage.__version__='0.21.0rc0.dev0+git20230406.6e51f8c7a'
Edge pixels count with float64 input:  230
Edge pixels count with float32 input:  270
Edge pixels count with uint8 input:  190
Edge pixels count with uint16 input:  230
Edge pixels count with uint32 input:  230
Edge pixels count with uint64 input:  0
Edge pixels count with int8 input:  270
Edge pixels count with int16 input:  230
Edge pixels count with int32 input:  270
Edge pixels count with int64 input:  0
```

OK, here we go: we call `gaussian` with `preserve_range=False`, so the uint64 image gets normalized before filtering :eyes: 
```
Edge pixels count with float64 input:  230
Edge pixels count with float32 input:  270
Edge pixels count with uint8 input:  230
Edge pixels count with uint16 input:  230
Edge pixels count with uint32 input:  230
Edge pixels count with uint64 input:  230
Edge pixels count with int8 input:  230
Edge pixels count with int16 input:  230
Edge pixels count with int32 input:  230
Edge pixels count with int64 input:  230
```
There's some sleuthing!  These final numbers are with some patch done?  And `int64` is missing.
Woops, fixed that.

Yes, I'll make a PR with the patch soon for review.

Resolving patch:
diff --git a/skimage/feature/_canny.py b/skimage/feature/_canny.py
--- a/skimage/feature/_canny.py
+++ b/skimage/feature/_canny.py
@@ -55,8 +55,12 @@ def _preprocess(image, mask, sigma, mode, cval):
     on the mask to recover the effect of smoothing from just the significant
     pixels.
     """
-    gaussian_kwargs = dict(sigma=sigma, mode=mode, cval=cval,
-                           preserve_range=False)
+    gaussian_kwargs = dict(
+        sigma=sigma,
+        mode=mode,
+        cval=cval,
+        preserve_range=False
+    )
     compute_bleedover = (mode == 'constant' or mask is not None)
     float_type = _supported_float_type(image.dtype)
     if mask is None:
@@ -188,6 +192,9 @@ def canny(image, sigma=1., low_threshold=None, high_threshold=None,
     # mask by one and then mask the output. We also mask out the border points
     # because who knows what lies beyond the edge of the image?
 
+    if np.issubdtype(image.dtype, np.int64) or np.issubdtype(image.dtype, np.uint64):
+        raise ValueError("64-bit integer images are not supported")
+
     check_nD(image, 2)
     dtype_max = dtype_limits(image, clip_negative=False)[1]
 


Test patch:
diff --git a/skimage/feature/tests/test_canny.py b/skimage/feature/tests/test_canny.py
--- a/skimage/feature/tests/test_canny.py
+++ b/skimage/feature/tests/test_canny.py
@@ -1,5 +1,6 @@
 import unittest
 import numpy as np
+import pytest
 from skimage._shared.testing import assert_equal
 from scipy.ndimage import binary_dilation, binary_erosion
 from skimage import data, feature
@@ -129,3 +130,24 @@ def test_dtype(self):
 
         assert_equal(feature.canny(image_float, 1.0, low, high),
                      feature.canny(image_uint8, 1.0, 255 * low, 255 * high))
+
+    def test_full_mask_matches_no_mask(self):
+        """The masked and unmasked algorithms should return the same result.
+
+        """
+        image = data.camera()
+
+        for mode in ('constant', 'nearest', 'reflect'):
+            assert_equal(
+                feature.canny(image, mode=mode),
+                feature.canny(image, mode=mode, mask=np.ones_like(image, dtype=bool))
+        )
+
+    def test_unsupported_int64(self):
+        for dtype in (np.int64, np.uint64):
+            image = np.zeros((10, 10), dtype=dtype)
+            image[3, 3] = np.iinfo(dtype).max
+            with pytest.raises(
+                    ValueError, match="64-bit integer images are not supported"
+            ):
+                feature.canny(image)