Repository: scikit-image/scikit-image

Issue:
Request: make dask[array] an optional dependency
In #2494, `dask` was made an optional dependency; in #2572, it was removed from `requirements.txt` (following #2571).

It looks as thought for version 0.14.0, `dask` is back to being a [required dependency](https://github.com/scikit-image/scikit-image/blob/master/requirements/default.txt) again. Was this an oversight when the `requirements.txt` file was refactored, or is `dask` now considered to be a required dependency of `scikit-image`?

(Background and motivation: Enthought packages `scikit-image` for EDM and Canopy, and the `dask` dependency turns `scikit-image` into quite a heavyweight package; if we can reasonably declare that `dask` is an optional dependency, we'd be quite happy to do so. :-)




Discussion:
From some digging, it looks as though `dask` was reintroduced as a required dependency in https://github.com/scikit-image/scikit-image/pull/2647#issuecomment-341017541

The depenency is on dask[array], which is 3 packages, numpy (check), dask (pure python) and toolz.

I don't think there is much dask in scikit-image today.
Ah, thank you. I'd missed that depending on `dask[array]` doesn't imply depending on all of `dask`. That's helpful.
Currently we depend on it for cycle spinning and apply_parallel.

We presumed this would be a very light dependency (https://github.com/scikit-image/scikit-image/pull/2647#issuecomment-341192030).  If that assumption is wrong, we could make it optional again, but @mdickinson it sounds as though it is not problematic for you if it is only `dask[array]`?
> it sounds as though it is not problematic for you if it is only `dask[array]`?

Yes, agreed that the `dask[array]` dependency seems very reasonable. As it turns out, it _is_ problematic for us, because our build system doesn't (currently) support partial dependencies like `dask[array]`, so we end up pulling in all the `dask` dependencies. But I think that's clearly our problem, not yours. :-)

Thank you for the quick responses! My question's answered, so I'll close this.
@mdickinson I would reopen this, and maybe change the title to "Request: make dask[array] an optional dependency"

I think removing the dask dependency is a reasonable request. On conda for example, the scikit-image now pull in all of dask, which also makes the scikit-image package huge. 

Packaging is imperfect. I would support PRs that would aim at isolating the dask dependency from the rest of the code-base. I presume the changes would be small.

I've been trying to split off dask on conda-forge into smaller packages, but how that is done exactly seems to be tricky due to `dask[distributed]`.

https://github.com/conda-forge/dask-feedstock/pull/49

@hmaarrfk here you go. =) Super happy for skimage to increase the pressure for partial dependencies support in conda, though. =P

Resolving patch:
diff --git a/skimage/restoration/_cycle_spin.py b/skimage/restoration/_cycle_spin.py
--- a/skimage/restoration/_cycle_spin.py
+++ b/skimage/restoration/_cycle_spin.py
@@ -1,7 +1,12 @@
 from itertools import product
-
 import numpy as np
-import dask
+from .._shared.utils import warn
+
+try:
+    import dask
+    dask_available = True
+except ImportError:
+    dask_available = False
 
 
 def _generate_shifts(ndim, multichannel, max_shifts, shift_steps=1):
@@ -143,6 +148,12 @@ def _run_one_shift(shift):
         tmp = func(xs, **func_kw)
         return _roll_axes(tmp, -np.asarray(shift))
 
+    if not dask_available and (num_workers is None or num_workers > 1):
+        num_workers = 1
+        warn('The optional dask dependency is not installed. '
+             'The number of workers is set to 1. To silence '
+             'this warning, install dask or explicitly set `num_workers=1` '
+             'when calling the `cycle_spin` function')
     # compute a running average across the cycle shifts
     if num_workers == 1:
         # serial processing


Test patch:
diff --git a/skimage/restoration/tests/test_denoise.py b/skimage/restoration/tests/test_denoise.py
--- a/skimage/restoration/tests/test_denoise.py
+++ b/skimage/restoration/tests/test_denoise.py
@@ -18,6 +18,13 @@
 else:
     PYWAVELET_ND_INDEXING_WARNING = None
 
+try:
+    import dask
+except ImportError:
+    DASK_NOT_INSTALLED_WARNING = 'The optional dask dependency is not installed'
+else:
+    DASK_NOT_INSTALLED_WARNING = None
+
 
 np.random.seed(1234)
 
@@ -604,7 +611,8 @@ def test_cycle_spinning_multichannel():
         func_kw = dict(sigma=sigma, multichannel=multichannel)
 
         # max_shifts=0 is equivalent to just calling denoise_func
-        with expected_warnings([PYWAVELET_ND_INDEXING_WARNING]):
+        with expected_warnings([PYWAVELET_ND_INDEXING_WARNING,
+                                DASK_NOT_INSTALLED_WARNING]):
             dn_cc = restoration.cycle_spin(noisy, denoise_func, max_shifts=0,
                                            func_kw=func_kw,
                                            multichannel=multichannel)
@@ -613,7 +621,8 @@ def test_cycle_spinning_multichannel():
 
         # denoising with cycle spinning will give better PSNR than without
         for max_shifts in valid_shifts:
-            with expected_warnings([PYWAVELET_ND_INDEXING_WARNING]):
+            with expected_warnings([PYWAVELET_ND_INDEXING_WARNING,
+                                    DASK_NOT_INSTALLED_WARNING]):
                 dn_cc = restoration.cycle_spin(noisy, denoise_func,
                                                max_shifts=max_shifts,
                                                func_kw=func_kw,
@@ -621,7 +630,8 @@ def test_cycle_spinning_multichannel():
             assert_(compare_psnr(img, dn_cc) > compare_psnr(img, dn))
 
         for shift_steps in valid_steps:
-            with expected_warnings([PYWAVELET_ND_INDEXING_WARNING]):
+            with expected_warnings([PYWAVELET_ND_INDEXING_WARNING,
+                                    DASK_NOT_INSTALLED_WARNING]):
                 dn_cc = restoration.cycle_spin(noisy, denoise_func,
                                                max_shifts=2,
                                                shift_steps=shift_steps,
@@ -653,11 +663,13 @@ def test_cycle_spinning_num_workers():
     denoise_func = restoration.denoise_wavelet
     func_kw = dict(sigma=sigma, multichannel=True)
 
+    # same results are expected whether using 1 worker or multiple workers
     with expected_warnings([PYWAVELET_ND_INDEXING_WARNING]):
-        # same result whether using 1 worker or multiple workers
         dn_cc1 = restoration.cycle_spin(noisy, denoise_func, max_shifts=1,
                                         func_kw=func_kw, multichannel=False,
                                         num_workers=1)
+    with expected_warnings([PYWAVELET_ND_INDEXING_WARNING,
+                            DASK_NOT_INSTALLED_WARNING]):
         dn_cc2 = restoration.cycle_spin(noisy, denoise_func, max_shifts=1,
                                         func_kw=func_kw, multichannel=False,
                                         num_workers=4)