Repository: scikit-image/scikit-image

Issue:
System-dependent doctest failures
I originally reported these on the [mailing list](https://groups.google.com/forum/#!topic/scikit-image/v8xjq7_2xq4), adding them here for tracking. The solution will in part depend on explicitly setting the print precision in numpy for doctests.

Running `skimage.doctest()` on my system (OSX 10.9, Anaconda) results in failures, apparently due to precision issues (see below).

```
........................................F.F...................................F./Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/measure/_label.py:4: skimage_deprecation: Call to deprecated function ``label``. Use ``skimage.measure.label`` instead.
  return _label(input, neighbors, background, return_num)
..F.......................................................
======================================================================
FAIL: Doctest: skimage.feature.texture.greycoprops
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/nuneziglesiasj/anaconda/lib/python2.7/doctest.py", line 2226, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for skimage.feature.texture.greycoprops
  File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/feature/texture.py", line 125, in greycoprops

----------------------------------------------------------------------
File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/feature/texture.py", line 176, in skimage.feature.texture.greycoprops
Failed example:
    contrast
Expected:
    array([[ 0.58333333,  1.        ],
           [ 1.25      ,  2.75      ]])
Got:
    array([[ 0.5833333333,  1.          ],
           [ 1.25        ,  2.75        ]])


======================================================================
FAIL: Doctest: skimage.filter._gaussian.gaussian_filter
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/nuneziglesiasj/anaconda/lib/python2.7/doctest.py", line 2226, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for skimage.filter._gaussian.gaussian_filter
  File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/filter/_gaussian.py", line 13, in gaussian_filter

----------------------------------------------------------------------
File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/filter/_gaussian.py", line 73, in skimage.filter._gaussian.gaussian_filter
Failed example:
    gaussian_filter(a, sigma=0.4)  # mild smoothing
Expected:
    array([[ 0.00163116,  0.03712502,  0.00163116],
           [ 0.03712502,  0.84496158,  0.03712502],
           [ 0.00163116,  0.03712502,  0.00163116]])
Got:
    array([[ 0.0016311596,  0.0371250207,  0.0016311596],
           [ 0.0371250207,  0.8449615765,  0.0371250207],
           [ 0.0016311596,  0.0371250207,  0.0016311596]])
----------------------------------------------------------------------
File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/filter/_gaussian.py", line 77, in skimage.filter._gaussian.gaussian_filter
Failed example:
    gaussian_filter(a, sigma=1)  # more smooting
Expected:
    array([[ 0.05855018,  0.09653293,  0.05855018],
           [ 0.09653293,  0.15915589,  0.09653293],
           [ 0.05855018,  0.09653293,  0.05855018]])
Got:
    array([[ 0.0585501805,  0.096532928 ,  0.0585501805],
           [ 0.096532928 ,  0.1591558917,  0.096532928 ],
           [ 0.0585501805,  0.096532928 ,  0.0585501805]])
----------------------------------------------------------------------
File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/filter/_gaussian.py", line 82, in skimage.filter._gaussian.gaussian_filter
Failed example:
    gaussian_filter(a, sigma=1, mode='reflect')
Expected:
    array([[ 0.08767308,  0.12075024,  0.08767308],
           [ 0.12075024,  0.16630671,  0.12075024],
           [ 0.08767308,  0.12075024,  0.08767308]])
Got:
    array([[ 0.0876730803,  0.1207502431,  0.0876730803],
           [ 0.1207502431,  0.1663067063,  0.1207502431],
           [ 0.0876730803,  0.1207502431,  0.0876730803]])


======================================================================
FAIL: Doctest: skimage.measure._label.label
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/nuneziglesiasj/anaconda/lib/python2.7/doctest.py", line 2226, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for skimage.measure._label.label
  File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/measure/_label.py", line unknown line number, in label

----------------------------------------------------------------------
File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/measure/_label.py", line ?, in skimage.measure._label.label
Failed example:
    x = np.eye(3).astype(int)
Exception raised:
    Traceback (most recent call last):
      File "/Users/nuneziglesiasj/anaconda/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest skimage.measure._label.label[0]>", line 1, in <module>
        x = np.eye(3).astype(int)
    NameError: name 'np' is not defined
----------------------------------------------------------------------
File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/measure/_label.py", line ?, in skimage.measure._label.label
Failed example:
    print(x)
Exception raised:
    Traceback (most recent call last):
      File "/Users/nuneziglesiasj/anaconda/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest skimage.measure._label.label[1]>", line 1, in <module>
        print(x)
    NameError: name 'x' is not defined
----------------------------------------------------------------------
File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/measure/_label.py", line ?, in skimage.measure._label.label
Failed example:
    print(m.label(x, neighbors=4))
Exception raised:
    Traceback (most recent call last):
      File "/Users/nuneziglesiasj/anaconda/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest skimage.measure._label.label[2]>", line 1, in <module>
        print(m.label(x, neighbors=4))
    NameError: name 'm' is not defined
----------------------------------------------------------------------
File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/measure/_label.py", line ?, in skimage.measure._label.label
Failed example:
    print(m.label(x, neighbors=8))
Exception raised:
    Traceback (most recent call last):
      File "/Users/nuneziglesiasj/anaconda/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest skimage.measure._label.label[3]>", line 1, in <module>
        print(m.label(x, neighbors=8))
    NameError: name 'm' is not defined
----------------------------------------------------------------------
File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/measure/_label.py", line ?, in skimage.measure._label.label
Failed example:
    x = np.array([[1, 0, 0],
                  [1, 1, 5],
                  [0, 0, 0]])
Exception raised:
    Traceback (most recent call last):
      File "/Users/nuneziglesiasj/anaconda/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest skimage.measure._label.label[4]>", line 1, in <module>
        x = np.array([[1, 0, 0],
    NameError: name 'np' is not defined
----------------------------------------------------------------------
File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/measure/_label.py", line ?, in skimage.measure._label.label
Failed example:
    print(m.label(x, background=0))
Exception raised:
    Traceback (most recent call last):
      File "/Users/nuneziglesiasj/anaconda/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest skimage.measure._label.label[5]>", line 1, in <module>
        print(m.label(x, background=0))
    NameError: name 'm' is not defined


======================================================================
FAIL: Doctest: skimage.measure.fit.ransac
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/nuneziglesiasj/anaconda/lib/python2.7/doctest.py", line 2226, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for skimage.measure.fit.ransac
  File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/measure/fit.py", line 470, in ransac

----------------------------------------------------------------------
File "/Users/nuneziglesiasj/anaconda/lib/python2.7/site-packages/skimage/measure/fit.py", line 576, in skimage.measure.fit.ransac
Failed example:
    ransac_model.params
Expected:
    array([ 20.12762373,  29.73563063,   4.81499637,  10.4743584 ,   0.05217117])
Got:
    array([ 20.1276237259,  29.7356306313,   4.8149963715,  10.474358404 ,
             0.0521711691])
```



Discussion:
@JDWarner commented on the mailing list post:

> Several of these might be solved by globally forcing the expected [`numpy.set_printoptions`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.set_printoptions.html) prior to running the doctests. I'm not sure why this would differ in certain platforms, but for whatever reason it looks like on OSX 10.9 you're reliably printing an additional two digits.
> 
> The second to last error, in `skimage.measure._label.label`, looks like a real error / malformed example to me.

We could use `#  doctest: +ELLIPSIS` and add change it to:

```
array([[ 0.58333333...,  1.        ],
           [ 1.25      ,  2.75      ]])
```

Ellipsis sounds like a good solution for the floating point mismatches.

Strange: I'm on OSX 10.10 and the doctests seem to pass on my system. Is this a numpy version issue (rather than an OSX issue)? I'm on numpy 1.8.1.

Regardless, these tests shouldn't be so strict. @JDWarner's suggestion of setting `numpy.set_printoptions` sounds better than setting `# doctest: +ELLIPSIS`, in my opinion, but the doctest directive wouldn't be that bad either.

BTW, I'm not sure this issue should be a blocker for the 0.11 release: doctests aren't run by default (although maybe they should be) so I'd be surprised if they passed on 0.10.

`set_printoptions` needs an extra line of Python code in the doctest, as compared to a comment after the code for the ellipsis.

Also - the set_printoptions setting seem to last across functions when running doctests via `python -m doctest thefile.py` but not via nosetests, which is a little confusing.

I don't get any failures on numpy standard wheel installs for numpy 1.8.1 or numpy 1.9.1 - maybe it's Anaconda MKL.  In any case @jni - you may need to choose a fix for these because we can't replicate the failures.

From latest master on OSX 10.6.8, using latest Anaconda w/MKL - all doctests pass.

@jni you wouldn't happen to have your global print options set in a custom config file somewhere, would you? ;)

@JDWarner I wouldn't even know how to do that, so I doubt it. However, I'm going to agree with @matthew-brett and @tonysyu that this is peculiar to my specific machine and config, and not my OS/Python version more generally. So I'm going to remove the 0.11 tag. I'll play more when I have some time...

@jni Any updates on this?

LOL:

``` python
>>> import skimage
>>> skimage.doctest()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jni/conda/envs/ana3/lib/python3.5/site-packages/skimage/__init__.py", line 90, in _test
    with warnings.catch_warnings():
NameError: name 'warnings' is not defined
```

This appears to be due to `del warnings, ...` at the end of `skimage/__init__.py`. Perhaps the solution is to import within individual functions instead of globally?

Yep, I can confirm that importing warnings from within that function works, and the test suite passes.

On OSX 10.11.4, `Python 3.4.4 |Anaconda 2.4.1`


Resolving patch:
diff --git a/skimage/__init__.py b/skimage/__init__.py
--- a/skimage/__init__.py
+++ b/skimage/__init__.py
@@ -80,6 +80,7 @@ def _test(doctest=False, verbose=False):
     def _test(doctest=False, verbose=False):
         """Run all unit tests."""
         import nose
+        import warnings
         args = ['', pkg_dir, '--exe', '--ignore-files=^_test']
         if verbose:
             args.extend(['-v', '-s'])


Test patch: