Repository: scikit-image/scikit-image

Issue:
skimage.segmentation.slic crashes when passed NaNs
## Description

When the image array passed to skimage.segmentation.slic() is a float array containing NaNs, the routine frequently crashes (most of the time, in my case). In Spyder, this causes a silent failure, with the print statement below failing to execute and the environment being reset. I've observed this behavior under both Windows and Linux.

## Way to reproduce
```python
import numpy as np
import skimage.segmentation as seg
im_slic = seg.slic(np.array([[np.nan, 0], [0, 0]]))
print(im_slic)
```


## Version information
```python
# Paste the output of the following python commands
from __future__ import print_function
import sys; print(sys.version)
import platform; print(platform.platform())
import skimage; print("scikit-image version: {}".format(skimage.__version__))
import numpy; print("numpy version: {}".format(numpy.__version__))
```

```python
3.7.4 (default, Aug  9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)]
Windows-10-10.0.17763-SP0
scikit-image version: 0.16.2
numpy version: 1.18.1
```



Discussion:
Thanks @rdinner for raising this! Seems like a serious bug. We'll investigate solutions.
@stefanv @hmaarrfk we've generally held off from

- automatically checking for nan's because it can be expensive. This would be even more of a concern with SLIC because one of the things it's known for is its speed.
- adding `nansafe=` or similar to the API because it would clutter an already cluttered API.

In this case, it's unclear what to do, then! One idea is to check whether any reduced representation can find the nans. In this case, perhaps, the initial cluster centroids, which is a set much smaller than the original pixels, could be checked for nan before proceeding further.
Thanks @jni ! A nansafe option would be nice. But even without modifying the code, documenting that the input image needs to be nan-free or that nans must be masked out (by adding a note to the docstring "image" parameter) would help others avoid this issue. Thank you!
I added a "good first issue" label for the task of adding a note in the docstring, which would already be an improvement. A further improvement would be to check if nans can be detected in the centroids as suggested by @jni.
Hey there, I'm looking for first issues to get involved with as I foray into the world of open-source :)
Is the issue about adding a note in docstring still available? Thanks!
Hi @RonakSharma1, thank you for your interest in scikit-image! Yes it still available. You can take a look at our contributing guidelines to get started https://scikit-image.org/docs/dev/contribute.html 
Hi @emmanuelle , thank you for this! I will have a read through the guidelines now and will let you know if I need any help as I progress.
Hey @emmanuelle 
Based on my basic understanding, I believe that only the docstring for the following files needs updating. Can you please confirm before I make any changes? Sorry to bother you with such trivial questions.

1. scikit-image/skimage/segmentation/slic_superpixels.py: Update 'image' docstring 
![slic_python](https://user-images.githubusercontent.com/34181525/90221975-119d2180-de03-11ea-9404-1638f50e083a.png)

2. scikit-image/skimage/segmentation/_slic.pyx: I think that this error might exist in Cython file as well and so I think 'image_zyx' docstring should be updated. What do you think?
![slic_cython](https://user-images.githubusercontent.com/34181525/90221957-0a761380-de03-11ea-97d1-c57f6076baa5.png)
Thank you for your help!

Hi @RonakSharma1 sorry I never got back to you. Don't hesitate to ping people (like @emmanuelle) in such a case since we are drowning in notifications :-). 

To answer your question: yes this is the docstring of the `.py` file, in the part you highlighted. As for the cython file, the function is not part of the public API so no need to change its docstring.
Hey @emmanuelle ,
Thank you for letting me know. I will make the changes and issue a PR. Will try it to it this weekend
Hey @emmanuelle 
I have now created a PR. Could you please review and provide me with feedback?
@emmanuelle and @jni should we close this issue now as the work is done I believe?

PR Link: https://github.com/scikit-image/scikit-image/pull/5014

Thanks for following up @RonakSharma1!  Closing.
My intuition though was that we should keep it open in case we want to address this at the implementation level later.
Err, yikes, yes---it is a crash!
Hey folks, I'd be interested in looking at coming up with some error catching 'nansafe' check on the inputted array for this as I have burned by this previously. Are you guys happy for me to take this on? 

I have two follow up questions on this also if thats the case:
1. I can't see any benchmarks specifically for error catching etc however I would assume to evaluate performance any changes to 'slic' would need to not signifcantly impact the benchmarks already established under the "benchmark_segmentation" category where 'slic' is used?
2. I can't see anything in the dev guide around where error catching functions are stored is it locally in the directory they are used in or would a nansafe function like this be stored in '_shared' directory as it could be used across the library?  

@emmanuelle @jni @emmanuelle  are you folks okay with me looking to mitigate this? 
Had a look into this with @lagru during a sprint at EuroSciPy. Here is what we found:

As it stands, it would be straightforward to add a check for NaN (and INF) pixels with essentially zero extra cost, since a `min` and `max` are computed over the entire image, and we could simply check the result of that (and raise a `ValueError`): https://github.com/scikit-image/scikit-image/blob/f0a3fd10edc7626baa220cd9977cdee25233ac5f/skimage/segmentation/slic_superpixels.py#L259-L262

However, there are two problems here: The image is scaled using the `image.min()` and `image.max()`, but:
- This includes the masked pixels, probably scaling should be done based on min and max after masking? This is the case even when there are no NaNs.
- `image.min()` (and `max`) will include NaNs from masked pixels so the entire image will be NaN, *even if all NaN pixels have been masked, as recommended in the docstring*.

Conclusion (probably?):

- "Apply" mask somehow (`numpy.ma`?), compute `min` and `max` after that.
- Check that for NaN (and INF) (cheaply, since single value, not array), then raise `ValueError`.
For reference, here are minor code changes with tests (but not addressing the two masking-related issues discussed above): [changes.txt](https://github.com/scikit-image/scikit-image/files/9476666/changes.txt)

Looks promising. Thank you @SimonHeybrock for the update. Would you like to turn you patch into a pull requests? :grin: We can address the mask issue separately in another issue or there if an easy solution comes to mind.
I don't think it makes sense to address the NaN-crash without fixing the mask issue first, since the code for the fix would have to change, depending on the solution of the mask issue. That is, fixing this here first would mean more work in total.
Fair. However, I don't think we really have to fuss around with `numpy.ma`. We might get away with indexing like `image[mask].max()` and similar if the `mask` is given. Do you want to try a PR? :slightly_smiling_face: 
Sorry but I am very busy currently. I have not tried, but my guess is that `image[mask].max()` is slower than using `numpy.ma`, since with indexing you have to make a copy.
No worries. I just wanted to make sure not to step on your toes. I'm happy to take over. :wink: 

Resolving patch:
diff --git a/benchmarks/benchmark_segmentation.py b/benchmarks/benchmark_segmentation.py
--- a/benchmarks/benchmark_segmentation.py
+++ b/benchmarks/benchmark_segmentation.py
@@ -72,7 +72,9 @@ def setup(self):
         try:
             mask = np.zeros((64, 64)) > 0
             mask[10:-10, 10:-10] = 1
-            segmentation.slic(np.ones_like(mask), mask=mask)
+            segmentation.slic(
+                np.ones_like(mask), mask=mask, **_channel_kwarg(False)
+            )
         except TypeError:
             raise NotImplementedError("masked slic unavailable")
 
diff --git a/skimage/segmentation/slic_superpixels.py b/skimage/segmentation/slic_superpixels.py
--- a/skimage/segmentation/slic_superpixels.py
+++ b/skimage/segmentation/slic_superpixels.py
@@ -203,6 +203,12 @@ def slic(image, n_segments=100, compactness=10., max_num_iter=10, sigma=0,
         dimension is not of length 3.
     ValueError
         If ``start_label`` is not 0 or 1.
+    ValueError
+        If ``image`` contains unmasked NaN values.
+    ValueError
+        If ``image`` contains unmasked infinite values.
+    ValueError
+        If ``image`` is 2D but ``channel_axis`` is -1 (the default).
 
     Notes
     -----
@@ -214,7 +220,8 @@ def slic(image, n_segments=100, compactness=10., max_num_iter=10, sigma=0,
       and ``spacing=[5, 1, 1]``, the effective `sigma` is ``[0.2, 1, 1]``. This
       ensures sensible smoothing for anisotropic images.
 
-    * The image is rescaled to be in [0, 1] prior to processing.
+    * The image is rescaled to be in [0, 1] prior to processing (masked
+      values are ignored).
 
     * Images of shape (M, N, 3) are interpreted as 2D RGB images by default. To
       interpret them as 3D with the last dimension having length 3, use
@@ -247,6 +254,12 @@ def slic(image, n_segments=100, compactness=10., max_num_iter=10, sigma=0,
     >>> segments = slic(img, n_segments=100, compactness=20)
 
     """
+    if image.ndim == 2 and channel_axis is not None:
+        raise ValueError(
+            f"channel_axis={channel_axis} indicates multichannel, which is not "
+            "supported for a two-dimensional image; use channel_axis=None if "
+            "the image is grayscale"
+        )
 
     image = img_as_float(image)
     float_dtype = utils._supported_float_type(image.dtype)
@@ -254,12 +267,29 @@ def slic(image, n_segments=100, compactness=10., max_num_iter=10, sigma=0,
     # function input
     image = image.astype(float_dtype, copy=True)
 
+    if mask is not None:
+        # Create masked_image to rescale while ignoring masked values
+        mask = np.ascontiguousarray(mask, dtype=bool)
+        if channel_axis is not None:
+            mask_ = np.expand_dims(mask, axis=channel_axis)
+            mask_ = np.broadcast_to(mask_, image.shape)
+        else:
+            mask_ = mask
+        image_values = image[mask_]
+    else:
+        image_values = image
+
     # Rescale image to [0, 1] to make choice of compactness insensitive to
     # input image scale.
-    image -= image.min()
-    imax = image.max()
-    if imax != 0:
-        image /= imax
+    imin = image_values.min()
+    imax = image_values.max()
+    if np.isnan(imin):
+        raise ValueError("unmasked NaN values in image are not supported")
+    if np.isinf(imin) or np.isinf(imax):
+        raise ValueError("unmasked infinite values in image are not supported")
+    image -= imin
+    if imax != imin:
+        image /= (imax - imin)
 
     use_mask = mask is not None
     dtype = image.dtype
@@ -291,7 +321,7 @@ def slic(image, n_segments=100, compactness=10., max_num_iter=10, sigma=0,
     # initialize cluster centroids for desired number of segments
     update_centroids = False
     if use_mask:
-        mask = np.ascontiguousarray(mask, dtype=bool).view('uint8')
+        mask = mask.view('uint8')
         if mask.ndim == 2:
             mask = np.ascontiguousarray(mask[np.newaxis, ...])
         if mask.shape != image.shape[:3]:


Test patch:
diff --git a/skimage/segmentation/tests/test_slic.py b/skimage/segmentation/tests/test_slic.py
--- a/skimage/segmentation/tests/test_slic.py
+++ b/skimage/segmentation/tests/test_slic.py
@@ -102,6 +102,16 @@ def test_gray_2d_deprecated_multichannel():
                    start_label=0)
 
 
+def test_gray2d_default_channel_axis():
+    img = np.zeros((20, 21))
+    img[:10, :10] = 0.33
+    with pytest.raises(
+            ValueError, match="channel_axis=-1 indicates multichannel"
+    ):
+        slic(img)
+    slic(img, channel_axis=None)
+
+
 def _check_segment_labels(seg1, seg2, allowed_mismatch_ratio=0.1):
     size = seg1.size
     ndiff = np.sum(seg1 != seg2)
@@ -115,14 +125,18 @@ def test_slic_consistency_across_image_magnitude():
     img_uint16 = 256 * img_uint8.astype(np.uint16)
     img_float32 = img_as_float(img_uint8)
     img_float32_norm = img_float32 / img_float32.max()
+    img_float32_offset = img_float32 + 1000
 
     seg1 = slic(img_uint8)
     seg2 = slic(img_uint16)
     seg3 = slic(img_float32)
     seg4 = slic(img_float32_norm)
+    seg5 = slic(img_float32_offset)
 
     np.testing.assert_array_equal(seg1, seg2)
     np.testing.assert_array_equal(seg1, seg3)
+    # Assert that offset has no impact on result
+    np.testing.assert_array_equal(seg4, seg5)
     # Floating point cases can have mismatch due to floating point error
     # exact match was observed on x86_64, but mismatches seen no i686.
     # For now just verify that a similar number of superpixels are present in
@@ -220,10 +234,12 @@ def test_enforce_connectivity():
 
     segments_connected = slic(img, 2, compactness=0.0001,
                               enforce_connectivity=True,
-                              convert2lab=False, start_label=0)
+                              convert2lab=False, start_label=0,
+                              channel_axis=None)
     segments_disconnected = slic(img, 2, compactness=0.0001,
                                  enforce_connectivity=False,
-                                 convert2lab=False, start_label=0)
+                                 convert2lab=False, start_label=0,
+                                 channel_axis=None)
 
     # Make sure nothing fatal occurs (e.g. buffer overflow) at low values of
     # max_size_factor
@@ -231,7 +247,8 @@ def test_enforce_connectivity():
                                       enforce_connectivity=True,
                                       convert2lab=False,
                                       max_size_factor=0.8,
-                                      start_label=0)
+                                      start_label=0,
+                                      channel_axis=None)
 
     result_connected = np.array([[0, 0, 0, 1, 1, 1],
                                  [0, 0, 0, 1, 1, 1],
@@ -407,17 +424,18 @@ def test_enforce_connectivity_mask():
 
     segments_connected = slic(img, 2, compactness=0.0001,
                               enforce_connectivity=True,
-                              convert2lab=False, mask=msk)
+                              convert2lab=False, mask=msk, channel_axis=None)
     segments_disconnected = slic(img, 2, compactness=0.0001,
                                  enforce_connectivity=False,
-                                 convert2lab=False, mask=msk)
+                                 convert2lab=False, mask=msk, channel_axis=None)
 
     # Make sure nothing fatal occurs (e.g. buffer overflow) at low values of
     # max_size_factor
     segments_connected_low_max = slic(img, 2, compactness=0.0001,
                                       enforce_connectivity=True,
                                       convert2lab=False,
-                                      max_size_factor=0.8, mask=msk)
+                                      max_size_factor=0.8, mask=msk,
+                                      channel_axis=None)
 
     result_connected = np.array([[0, 1, 1, 2, 2, 0],
                                  [0, 1, 1, 2, 2, 0],
@@ -537,7 +555,7 @@ def test_dtype_support(dtype):
     img = np.random.rand(28, 28).astype(dtype)
 
     # Simply run the function to assert that it runs without error
-    slic(img, start_label=1)
+    slic(img, start_label=1, channel_axis=None)
 
 
 def test_start_label_fix():
@@ -558,3 +576,24 @@ def test_start_label_fix():
                   n_segments=6, compactness=0.01, enforce_connectivity=True,
                   max_num_iter=10)
     assert superp.min() == start_label
+
+
+def test_raises_ValueError_if_input_has_NaN():
+    img = np.zeros((4,5), dtype=float)
+    img[2, 3] = np.NaN
+    with pytest.raises(ValueError):
+        slic(img, channel_axis=None)
+
+    mask = ~np.isnan(img)
+    slic(img, mask=mask, channel_axis=None)
+
+
+@pytest.mark.parametrize("inf", [-np.inf, np.inf])
+def test_raises_ValueError_if_input_has_inf(inf):
+    img = np.zeros((4,5), dtype=float)
+    img[2, 3] = inf
+    with pytest.raises(ValueError):
+        slic(img, channel_axis=None)
+
+    mask = np.isfinite(img)
+    slic(img, mask=mask, channel_axis=None)