Repository: scikit-image/scikit-image

Issue:
Divergent results in threshold_multiotsu with 16bits images
## Description
Hello,
I was testing the function with tomograph images, 16 bit (uint16), and memory error occurred:

`[...] scikit-image\skimage\filters\thresholding.py", line 1107, in <listcomp>
 momP, momS, var_btwcls = [np.zeros((nbins, nbins)) for n in range(3)]
MemoryError`

One solution found was to modify [these lines](https://github.com/scikit-image/scikit-image/blob/master/skimage/filters/thresholding.py#L1093-L1096) to `hist, bin_centers = np.histogram (image.ravel (), bins = nbins)` and `prob = hist / len (bin_centers)`, that is, use the default function in numpy to calculate the histogram.

Despite this, there is another (more important) problem where the threshold values found differ from those found in the SimpleITK and MatLAB libraries.

An example using a single image, I received the following values (3 classes, 2 thresholds):

SimpleITK: (12443, 29469)
MatLAB: (12491, 29802)
Skimage: (26629, 33177)

I was unable to upload the test image, but it is available at this link: [https://www.digitalrocksportal.org/projects/35/origin_data/85/](https://www.digitalrocksportal.org/projects/35/origin_data/85/)

 (note: although it says that the images are 8-bit, they are actually 16-bit).

## Way to reproduce
```python
# code to reproduce MemoryError

from skimage.filters import threshold_multiotsu
import numpy as np
import pytiff

names = ['AT CARB_0.7 Micron scan_recon_Export0001.tiff']

imgs = np.empty((1, 1008, 980), np.uint16)
for i, fpath in enumerate(names):
    with pytiff.Tiff(fpath) as handle:
        for page in handle.pages:
            imgs[i] = page[:]

print(threshold_multiotsu(imgs))
```


## Version information
```python
# Paste the output of the following python commands
from __future__ import print_function
import sys; print(sys.version)
import platform; print(platform.platform())
import skimage; print("scikit-image version: {}".format(skimage.__version__))
import numpy; print("numpy version: {}".format(numpy.__version__))
```

```python
# your output here
3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)]
Windows-10-10.0.17134-SP0
scikit-image version: 0.16.dev0
numpy version: 1.16.2
```


threshold_otsu and threshold_multiotsu give different results
## Description
thank you very much for implementing threshold_multiotsu, it is super helpful for my project.
However, I found some inconsistency, using threshold_otsu and threshold_multiotsu with two classes produces different results.

## Way to reproduce
```python
import numpy as np
from skimage.filters import threshold_otsu, threshold_multiotsu

array = np.array([-730.,  371.,  485.,  514.,  640.,  687.,  699.,  825.,  828., 1025., 1082., 1129., 1233.])
thresh = threshold_multiotsu(array, 2)
print(f"Multiotsu: {thresh[0]}")
# 830.43164062

thresh = threshold_otsu(array)
print(f"Singleotsu: {thresh}")
# -726.166015625

```

## Version information

```python
# Paste the output of the following python commands
from __future__ import print_function
import sys; print(sys.version)
import platform; print(platform.platform())
import skimage; print("scikit-image version: {}".format(skimage.__version__))
import numpy; print("numpy version: {}".format(numpy.__version__))
```

```python
3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
Windows-10-10.0.17134-SP0
scikit-image version: 0.16.dev0
numpy version: 1.16.4
```




Discussion:
Thank you for the detailed report. I Cc @alexandrejaguar who is the leading author of this, to bring him in the loop.
Re MemoryError: I'm not sure this algorithm is suited for 16 bit data (i.e. using 16-bit histograms). Runtime and memory usage are excessive. To work around this, convert the data to floating point first.

Runtime can be significantly improved by Cythonizing the calculation of the variance between classes, e.g.
```diff
diff --git a/skimage/filters/_multiotsu.pyx b/skimage/filters/_multiotsu.pyx
index 6e1a2668a..552840f0a 100644
--- a/skimage/filters/_multiotsu.pyx
+++ b/skimage/filters/_multiotsu.pyx
@@ -8,6 +8,44 @@ cimport numpy as cnp
 cimport cython
 from itertools import combinations

+
+def _init_var_btwcls(double [::1] prob, double [:, ::1] var_btwcls):
+    """Initialize variance between classes array."""                                                                   +    cdef:
+        ssize_t u, v
+        ssize_t nbins = prob.size
+        double [:, ::1] momP  # zeroth moments, cumulative probability
+        double [:, ::1] momS  # first moments, mean
+
+    momP = np.zeros_like(var_btwcls)
+    momS = np.zeros_like(var_btwcls)
+
+    # building the lookup tables.
+    # step 1: calculating the diagonal.
+    for u in range(1, nbins):
+        momP[u, u] = prob[u]
+        momS[u, u] = u * prob[u]
+
+    # step 2: calculating the first row.
+    for u in range(1, nbins-1):
+        momP[1, u+1] = momP[1, u] + prob[u+1]
+        momS[1, u+1] = momS[1, u] + (u+1)*prob[u+1]
+
+    # step 3: calculating the other rows recursively.
+    for u in range(2, nbins):
+        for v in range(u+1, nbins):
+            momP[u, v] = momP[1, v] - momP[1, u-1]
+            momS[u, v] = momS[1, v] - momS[1, u-1]
+
+    # step 4: calculating the between class variance.
+    for u in range(1, nbins):
+        for v in range(u+1, nbins):                                                                                    +            if (momP[u, v] != 0):
+                var_btwcls[u, v] = momS[u, v] * momS[u, v] / momP[u, v]
+            else:
+                var_btwcls[u, v] = 0
+
+
 def _find_threshold_multiotsu(double [:, ::1] var_btwcls,
                               Py_ssize_t classes,
                               Py_ssize_t bins):
diff --git a/skimage/filters/thresholding.py b/skimage/filters/thresholding.py
index a20055fa6..d2a0bc504 100644
--- a/skimage/filters/thresholding.py
+++ b/skimage/filters/thresholding.py
@@ -8,7 +8,7 @@ from ..exposure import histogram
 from .._shared.utils import assert_nD, warn, deprecated
 from ..transform import integral_image
 from ..util import crop, dtype_limits
-from ..filters._multiotsu import _find_threshold_multiotsu
+from ..filters._multiotsu import _find_threshold_multiotsu, _init_var_btwcls


 __all__ = ['try_all_threshold',
@@ -1097,35 +1097,10 @@ def threshold_multiotsu(image, classes=3, nbins=256):
     # histogram ignores nbins for integer arrays.
     nbins = len(bin_centers)

-    # defining arrays to store the zeroth (momP, cumulative probability)
-    # and first (momS, mean) moments, and the variance between classes
-    # (var_btwcls).
-    momP, momS, var_btwcls = [np.zeros((nbins, nbins)) for n in range(3)]
-
-    # building the lookup tables.
-    # step 1: calculating the diagonal.
-    for u in range(1, nbins):
-        momP[u, u] = prob[u]
-        momS[u, u] = u * prob[u]
-
-    # step 2: calculating the first row.
-    for u in range(1, nbins-1):                                                                                        -        momP[1, u+1] = momP[1, u] + prob[u+1]
-        momS[1, u+1] = momS[1, u] + (u+1)*prob[u+1]
-
-    # step 3: calculating the other rows recursively.
-    for u in range(2, nbins):
-        for v in range(u+1, nbins):
-            momP[u, v] = momP[1, v] - momP[1, u-1]
-            momS[u, v] = momS[1, v] - momS[1, u-1]
-
-    # step 4: calculating the between class variance.
-    for u in range(1, nbins):
-        for v in range(u+1, nbins):
-            if (momP[u, v] != 0):
-                var_btwcls[u, v] = momS[u, v]**2 / momP[u, v]
-            else:
-                var_btwcls[u, v] = 0
+    # defining array to store the variance between classes.
+    var_btwcls = np.zeros((nbins, nbins))
+
+    _init_var_btwcls(prob, var_btwcls)

     # finding max threshold candidates, depending on classes.
     # number of thresholds is equal to number of classes - 1.

```

I looked at it briefly. So, we have a 16bits uint image. Therefore, the value of nbins is ignored (as documented) and the number of bins is huge (2^16).

Proposed fix: convert the input image to a float dtype such that the nbins value is never ignored. Otherwise, in this situation, the user might have difficulties to do so.

This raises the question to maintain or not the nbins behavior in skimage.exposure.histogram.
2^16 is not huge anymore. Might be worth allowing users to specify nbins, but the default set to old value?

`sqrt( 2 ^ 16) = 256`. so it is as large as a 256 x 256 image.
Hi @btguilherme @cgohlke,
I am checking the code now, and it relates to #3980. I would need a little time to check how to solve this.
@hmaarrfk the algorithm I based this on is aimed to `int` images. I guess @sciunto is right: `int16` is a no-go for this one, specifically.
Hi @Miiiluuu,
thank you very much for your bug report. I am trying to identify what could cause this effect, and I'll get back to you later, OK?
With our images, I found two that replicate the issue

```
names = ('camera', 'moon', 'coins', 'text', 'clock', 'page')

for name in names:
    img = getattr(data, name)()
    otsu = threshold_otsu(img)
    multi = threshold_multiotsu(img, 2)[0]
    if otsu != multi:
        print(name, otsu, multi)
```

> moon 87 89



```
names = ('chelsea', 'coffee', 'astronaut', 'rocket')

from skimage.color import rgb2gray

for name in names:
    img = rgb2gray(getattr(data, name)())
    otsu = threshold_otsu(img)
    multi = threshold_multiotsu(img, 2)[0]
    if otsu != multi:
        print(name, otsu, multi)
```

> astronaut 0.388671875 0.439453125
When developing speedups for otsu, I found that there often exists two points that give the same result mathematically. The one that would be chosen would be implementation specific. I had to corner case a few of the points in the test. I think I left a link to a gist in the otsu test that made things more explicit.
All images pass if I call skimage.util.invert() prior thresholding, which seems to comfort your observation @hmaarrfk 
Actually, it could be a nice note to say that the uniqueness of the threshold (likely also true for some other algorithms) can be checked by inverting the image.

Resolving patch:
diff --git a/skimage/filters/thresholding.py b/skimage/filters/thresholding.py
--- a/skimage/filters/thresholding.py
+++ b/skimage/filters/thresholding.py
@@ -8,7 +8,8 @@
 from .._shared.utils import check_nD, warn
 from ..transform import integral_image
 from ..util import crop, dtype_limits
-from ..filters._multiotsu import _find_threshold_multiotsu
+from ..filters._multiotsu import (_get_multiotsu_thresh_indices_lut,
+                                  _get_multiotsu_thresh_indices)
 
 
 __all__ = ['try_all_threshold',
@@ -1104,9 +1105,15 @@ def threshold_multiotsu(image, classes=3, nbins=256):
 
     Returns
     -------
-    idx_thresh : array
+    thresh : array
         Array containing the threshold values for the desired classes.
 
+    Raises
+    ------
+    ValueError
+         If ``image`` contains less grayscale value then the desired
+         number of classes.
+
     Notes
     -----
     This implementation relies on a Cython function whose complexity
@@ -1114,12 +1121,15 @@ def threshold_multiotsu(image, classes=3, nbins=256):
     is the number of histogram bins and :math:`C` is the number of
     classes desired.
 
+    The input image must be grayscale.
+
     References
     ----------
     .. [1] Liao, P-S., Chen, T-S. and Chung, P-C., "A fast algorithm for
            multilevel thresholding", Journal of Information Science and
            Engineering 17 (5): 713-727, 2001. Available at:
            <http://ftp.iis.sinica.edu.tw/JISE/2001/200109_01.pdf>
+           :DOI:`10.6688/JISE.2001.17.5.1`
     .. [2] Tosa, Y., "Multi-Otsu Threshold", a java plugin for ImageJ.
            Available at:
            <http://imagej.net/plugins/download/Multi_OtsuThreshold.java>
@@ -1132,49 +1142,38 @@ def threshold_multiotsu(image, classes=3, nbins=256):
     >>> thresholds = threshold_multiotsu(image)
     >>> regions = np.digitize(image, bins=thresholds)
     >>> regions_colorized = label2rgb(regions)
+
     """
+
+    if len(image.shape) > 2 and image.shape[-1] in (3, 4):
+        msg = ("threshold_multiotsu is expected to work correctly only for "
+               "grayscale images; image shape {0} looks like an RGB image")
+        warn(msg.format(image.shape))
+
     # calculating the histogram and the probability of each gray level.
-    hist, bin_centers = histogram(image.ravel(),
+    prob, bin_centers = histogram(image.ravel(),
                                   nbins=nbins,
-                                  source_range='image')
-    prob = hist / image.size
-    # histogram ignores nbins for integer arrays.
-    nbins = len(bin_centers)
-
-    # Compute the zeroth (momP, cumulative probability) and first
-    # (momS, mean) moments
-
-    # building the lookup tables.
-    # step 1: calculating the diagonal.
-    prob[0] = 0
-    momP = np.diagflat(prob)
-    momS = np.diagflat(np.arange(nbins) * prob)
-
-    # step 2: calculating the first row.
-    momP[1, 2:] = prob[1] + np.cumsum(prob[2:])
-    momS[1, 2:] = prob[1] + np.cumsum(np.arange(2, nbins) * prob[2:])
-
-    # step 3: the other rows are recursively computed as:
-    # A[i, j] = A[1, j] - A[1, i-1] for A in {momP, momS};  i > 1 and j > i.
-    upper_tri = np.triu_indices_from(momP[2:, 2:], 1)
-
-    momP[2:, 2:][upper_tri] = (momP[1, 2:][upper_tri[1]]
-                               - np.repeat(momP[1, 1:-2],
-                                           np.arange(nbins - 3, 0, -1)))
-    momS[2:, 2:][upper_tri] = (momS[1, 2:][upper_tri[1]]
-                               - np.repeat(momS[1, 1:-2],
-                                           np.arange(nbins - 3, 0, -1)))
-
-    # step 4: calculating the between class variance.
-    var_btwcls = np.zeros_like(momP)
-    idx = momP > 0
-    var_btwcls[idx] = momS[idx] ** 2 / momP[idx]
-
-    # finding max threshold candidates, depending on classes.
-    # number of thresholds is equal to number of classes - 1.
-    aux_thresh = _find_threshold_multiotsu(var_btwcls, classes, nbins)
-
-    # correcting threshold values.
-    idx_thresh = bin_centers[:-1][aux_thresh.astype('int')]
-
-    return idx_thresh
+                                  source_range='image',
+                                  normalize=True)
+    prob = prob.astype('float32')
+
+    nvalues = np.count_nonzero(prob)
+    if nvalues < classes:
+        msg = ("The input image has only {} different values. "
+               "It can not be thresholded in {} classes")
+        raise ValueError(msg.format(nvalues, classes))
+    elif nvalues == classes:
+        thresh_idx = np.where(prob > 0)[0][:-1]
+    else:
+        # Get threshold indices
+        try:
+            thresh_idx = _get_multiotsu_thresh_indices_lut(prob, classes - 1)
+        except MemoryError:
+            # Don't use LUT if the number of bins is too large (if the
+            # image is uint16 for example): in this case, the
+            # allocated memory is too large.
+            thresh_idx = _get_multiotsu_thresh_indices(prob, classes - 1)
+
+    thresh = bin_centers[thresh_idx]
+
+    return thresh


Test patch:
diff --git a/skimage/filters/tests/test_thresholding.py b/skimage/filters/tests/test_thresholding.py
--- a/skimage/filters/tests/test_thresholding.py
+++ b/skimage/filters/tests/test_thresholding.py
@@ -4,9 +4,10 @@
 
 from skimage import util
 from skimage import data
-from skimage import color
+from skimage.color import rgb2gray
 from skimage.draw import circle
 from skimage._shared._warnings import expected_warnings
+from skimage.exposure import histogram
 from skimage.filters.thresholding import (threshold_local,
                                           threshold_otsu,
                                           threshold_li,
@@ -21,6 +22,8 @@
                                           try_all_threshold,
                                           _mean_std,
                                           _cross_entropy)
+from skimage.filters._multiotsu import (_get_multiotsu_thresh_indices_lut,
+                                        _get_multiotsu_thresh_indices)
 from skimage._shared import testing
 from skimage._shared.testing import assert_equal, assert_almost_equal
 from skimage._shared.testing import assert_array_equal
@@ -536,10 +539,13 @@ def test_niblack_sauvola_pathological_image():
 
 
 def test_bimodal_multiotsu_hist():
-    image = data.camera()
-    thr_otsu = threshold_otsu(image)
-    thr_multi = threshold_multiotsu(image, classes=2)
-    assert thr_otsu == thr_multi
+    for name in ['camera', 'moon', 'coins', 'text', 'clock', 'page']:
+        img = getattr(data, name)()
+        assert threshold_otsu(img) == threshold_multiotsu(img, 2)
+
+    for name in ['chelsea', 'coffee', 'astronaut', 'rocket']:
+        img = rgb2gray(getattr(data, name)())
+        assert threshold_otsu(img) == threshold_multiotsu(img, 2)
 
 
 def test_check_multiotsu_results():
@@ -561,18 +567,26 @@ def test_multiotsu_output():
     for coor, val in zip(coords, values):
         rr, cc = circle(coor[1], coor[0], 20)
         image[rr, cc] = val
-    t1, t2 = threshold_multiotsu(image)
-    # since the image only contains values 64, 128, 192, and the image is
-    # compared to the threshold with a strictly greater than check, thresholds
-    # 64, 65, ..., 127 are all equivalent, and thresholds 128, 129, ..., 191
-    # are all equivalent. The overly stringent check of t1 == 64 was causing
-    # some 32-bit builds to fail. See e.g.
-    # https://travis-ci.org/scikit-image/scikit-image-wheels/jobs/595911861#L3896-L3911
-    assert t1 in range(64, 128)
-    assert t2 in range(128, 192)
+    thresholds = [0, 64, 128]
+    assert np.array_equal(thresholds, threshold_multiotsu(image, classes=4))
+
 
-    image = color.rgb2gray(data.astronaut())
-    assert_almost_equal(threshold_multiotsu(image, 2), 0.43945312)
+def test_multiotsu_astro_image():
+    img = util.img_as_ubyte(data.astronaut())
+    with expected_warnings(['grayscale']):
+        assert_almost_equal(threshold_multiotsu(img), [58, 149])
+
+
+def test_multiotsu_more_classes_then_values():
+    img = np.ones((10, 10), dtype=np.uint8)
+    with testing.raises(ValueError):
+        threshold_multiotsu(img, classes=2)
+    img[:, 3:] = 2
+    with testing.raises(ValueError):
+        threshold_multiotsu(img, classes=3)
+    img[:, 6:] = 3
+    with testing.raises(ValueError):
+        threshold_multiotsu(img, classes=4)
 
 
 @pytest.mark.parametrize("thresholding, lower, upper", [
@@ -588,3 +602,19 @@ def test_thresholds_dask_compatibility(thresholding, lower, upper):
     import dask.array as da
     dask_camera = da.from_array(data.camera(), chunks=(256, 256))
     assert lower < float(thresholding(dask_camera)) < upper
+
+
+def test_multiotsu_lut():
+    for classes in [2, 3, 4]:
+        for name in ['camera', 'moon', 'coins', 'text', 'clock', 'page']:
+            img = getattr(data, name)()
+            prob, bin_centers = histogram(img.ravel(),
+                                          nbins=256,
+                                          source_range='image',
+                                          normalize=True)
+            prob = prob.astype('float32')
+
+            result_lut = _get_multiotsu_thresh_indices_lut(prob, classes - 1)
+            result = _get_multiotsu_thresh_indices(prob, classes - 1)
+
+            assert np.array_equal(result_lut, result)