Repository: scikit-image/scikit-image

Issue:
Error installing skimage-
Hi!
I've install skimage on a MacOs 10.10, like it's said in the documentation:
pip install -U scikit-image
and it say that need scipy to work, I had to install it to work.



Discussion:
That is correct and the intended behavior. Required dependencies must be installed first, then scikit-image... at least when one uses pip.

pip itself is not a package manager. If you want that, a distribution like Anaconda or Canopy will "just work" by grabbing all necessary dependencies and then what you asked for. 

Well - I think pip is a package manager - and if you install scikit-image, pip will also install:

```
Installing collected packages: scikit-image, networkx, matplotlib, pillow, six, decorator, nose, pytz, numpy, python-dateutil, mock, pyparsing
```

It's just that scipy is not among the declared dependencies for the install.  Should it be?

Looking a little further, scipy >= 0.9 is declared in [requirements.txt](https://github.com/scikit-image/scikit-image/blob/master/requirements.txt) so it should be brought along.

I can't remember the exact source of the discussion, but I think it was deliberate that the requirements.txt requirements were different from the default pip / source install requirements.

Here is the reason for it:

```
# do not attempt to install numpy and scipy until they have eggs available
INSTALL_REQUIRES = [r for r in INSTALL_REQUIRES
                                     if not r.startswith(('scipy', 'numpy'))]
```

Hum - well - they do have 'eggs' available for OSX, but not for Windows or Linux.  So.... ?

/cc @blink1073 

We could special case `darwin` and leave it as-is on other platforms until
eggs are available.

On Mon, May 11, 2015 at 3:25 PM, Stefan van der Walt <
notifications@github.com> wrote:

> /cc @blink1073 https://github.com/blink1073
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/scikit-image/scikit-image/issues/1506#issuecomment-101037662
> .

Just to be clear - we mean 'wheels' right?   I think eggs have definitely gone off. 

Is it important that eggs are available?  Why can't we let pip 
compile scipy from source (as long as it doesn't upgrade 
unnecessarily)?

a) It will upgrade unnecessarily if someone upgrades scikit-image;
b) Scipy does take a long time and a fortran compiler - so it's pretty error prone for Linux, out of the question on Windows.  For OSX, there should always be a wheel.

I thought the upgrade happens only when you specified a version. 
Or does it _always_ upgrade?  That's pretty lousy.

It always upgrades:

https://pip.pypa.io/en/latest/user_guide.html#only-if-needed-recursive-upgrade
https://github.com/pypa/pip/issues/304

Yes, it's lousy.

Yes, I meant to say `wheels`, and yes, the behavior of `pip install -U` is quite lousy.

I'm pretty sure this problem will go away (updating numpy, scipy, etc) if one does not define version (e.g. `>=1.6.1`) in `setup.py`. The dependency will be met if scipy is installed and only `scipy` is required. This has been discussed several times, but declined as one did not want to define dependencies in two places (or write a requirements.txt parser - which I think is silly for 7 requirements).

My take is that duplicating 7 requirements is worth it, and be rid of this installation trouble.

To be clear, the proposal is replacing 

```
INSTALL_REQUIRES = [r for r in INSTALL_REQUIRES
                                     if not r.startswith(('scipy', 'numpy'))]
```

with

```
INSTALL_REQUIRES = [
    'cython',
    'matplotlib',
    'numpy',
    'scipy',
    'six',
    'networkx',
    'pillow' ]
```

From the `pip` documentation:

https://pip.pypa.io/en/latest/reference/pip_install.html#cmdoption-U

"Upgrade all specified packages to the newest available version. This process is recursive regardless of whether a dependency is already satisfied."

@arve0, even if we do not specify the version, we will still have this mess.

@blink1073 Sorry, I'm not following you. How will `pip install scikit-image` (_without_ -U) be a mess, when all dependencies are declared?

This implies that the user either will get compiled wheels for his operating system (true for all dependencies on Mac and windows I believe), or has a system with compiler (should be added to the docs how the most popular, read Ubuntu, get that installed).

Are my assumptions wrong, or what have I missed?

For people wanting to upgrade, that will be `pip install --upgrade --no-deps scikit-image`, but the original poster was having trouble _installing_, not upgrading.

But when we add a new dependency (like dask), that would not work. 

—
Sent from Mailbox

On Sat, May 23, 2015 at 2:18 AM, Arve Seljebu notifications@github.com
wrote:

> ## For people wanting to upgrade, that will be `pip install --upgrade --no-deps scikit-image`, but the original poster was having trouble _installing_, not upgrading.
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/scikit-image/scikit-image/issues/1506#issuecomment-104856131

We could recommend `pip uninstall scikit-image; pip install scikit-image`

—
Sent from Mailbox

On Sat, May 23, 2015 at 2:18 AM, Arve Seljebu notifications@github.com
wrote:

> ## For people wanting to upgrade, that will be `pip install --upgrade --no-deps scikit-image`, but the original poster was having trouble _installing_, not upgrading.
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/scikit-image/scikit-image/issues/1506#issuecomment-104856131

or `pip install --upgrade --no-deps scikit-image; pip install scikit-image` (from the pip manual).

So that should work today without any changes to our setup.py, no?

All dependencies will have to be added (scipy and numpy), but versions should be stripped to avoid upgrading system-installed packages like scipy, matplotlib, etc.

In other words; dependencies will be either soft dependencies or hard dependencies.
- Hard dependencies will require a specific version installed (this should in a perfect world only consist of python-only packages)
- Soft dependencies will only require the package (and should not force upgrades)

E.g.
- six is a hard dependency (we need above 1.3)
- numpy is a soft dependency

Might add; there is nothing wrong with having (sane) hard dependencies in requirements.txt for developers. A developer should definitely have reasonable updated numpy/scipy/etc.

@arve, the listed versions _are_ hard dependencies in that we rely on functions or an API provided by those versions or the version is the minimum we can test against. 

—
Sent from Mailbox

On Sat, May 23, 2015 at 7:49 AM, Arve Seljebu notifications@github.com
wrote:

> In other words; dependencies will be either soft dependencies or hard dependencies.
> - Hard dependencies will require a specific version installed (this should in a perfect world only consist of python-only packages)
> - Soft dependencies will only require the package (and should not force upgrades)
>   E.g.
> - six is a hard dependency (we need above 1.3)
> - numpy is a soft dependency
>   Might add; there is nothing wrong with having (sane) hard dependencies in requirements.txt for developers. A developer should definitely have reasonable updated numpy/scipy/etc.
>   ---
>   Reply to this email directly or view it on GitHub:
>   https://github.com/scikit-image/scikit-image/issues/1506#issuecomment-104889492

Well, then one should just remove

```
INSTALL_REQUIRES = [r for r in INSTALL_REQUIRES
                                     if not r.startswith(('scipy', 'numpy'))]
```

**EDITED**

Haha, sorry! I mean, remove it! :laughing: 

Agreed, with the new suggested incantation we can certainly simplify setup.py, can to take a crack at the fix and the documentation update?

—
Sent from Mailbox

On Sat, May 23, 2015 at 8:06 AM, Arve Seljebu notifications@github.com
wrote:

> ## Haha, sorry! I mean, remove it! :laughing: 
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/scikit-image/scikit-image/issues/1506#issuecomment-104892885

I can take a look at it in the evening (GMT+1).

Thanks brother, keep keeping us honest.  😜


Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -34,24 +34,10 @@
 with open('requirements.txt') as fid:
     INSTALL_REQUIRES = [l.strip() for l in fid.readlines() if l]
 
-# development versions do not have the cythonized files
-if VERSION.endswith('dev'):
-    SETUP_REQUIRES = [r for r in INSTALL_REQUIRES if r.startswith('cython')]
-else:
-    INSTALL_REQUIRES = [r for r in INSTALL_REQUIRES
-                                         if not r.startswith('cython')]
-    SETUP_REQUIRES = []
-
-
-# list requirements for PyPI
-REQUIRES = [r.replace('>=', ' (>= ') + ')'
-                      for r in INSTALL_REQUIRES + SETUP_REQUIRES]
+# requirements for those browsing PyPI
+REQUIRES = [r.replace('>=', ' (>= ') + ')' for r in INSTALL_REQUIRES]
 REQUIRES = [r.replace('==', ' (== ') for r in REQUIRES]
-
-
-# do not attempt to install numpy and scipy until they have eggs available
-INSTALL_REQUIRES = [r for r in INSTALL_REQUIRES
-                                     if not r.startswith(('scipy', 'numpy'))]
+REQUIRES = [r.replace('[array]', '') for r in REQUIRES]
 
 
 def configuration(parent_package='', top_path=None):
@@ -73,9 +59,17 @@ def configuration(parent_package='', top_path=None):
 
 
 if __name__ == "__main__":
-    # purposely fail loudly if numpy or scipy are not available
-    from numpy.distutils.core import setup
-    import scipy
+    # purposely fail if numpy is not available
+    # other dependecies will be resolved by pip (install_requires)
+    try:
+        from numpy.distutils.core import setup
+    except ImportError:
+        print('To install scikit-image from source, you will need numpy.\n' +
+              'Install numpy with pip:\n' +
+              'pip install numpy\n'
+              'Or use your operating system package manager. For more\n' +
+              'details, see http://scikit-image.org/docs/stable/install.html')
+        sys.exit(1)
 
     setup(
         name=DISTNAME,
@@ -105,8 +99,9 @@ def configuration(parent_package='', top_path=None):
         ],
 
         configuration=configuration,
-        setup_requires=SETUP_REQUIRES,
         install_requires=INSTALL_REQUIRES,
+        # install cython when running setup.py (source install)
+        setup_requires=['cython>=0.21'],
         requires=REQUIRES,
         packages=setuptools.find_packages(exclude=['doc']),
         include_package_data=True,


Test patch:
diff --git a/skimage/io/tests/test_pil.py b/skimage/io/tests/test_pil.py
--- a/skimage/io/tests/test_pil.py
+++ b/skimage/io/tests/test_pil.py
@@ -230,9 +230,9 @@ class TestSaveTIF:
     def roundtrip(self, dtype, x, compress):
         with temporary_file(suffix='.tif') as fname:
             if dtype == np.bool:
-                expected = ['low contrast']
+                expected = ['low contrast|unclosed file']
             else:
-                expected = []
+                expected = ['unclosed file|\A\Z']
             with expected_warnings(expected):
                 if compress > 0:
                     imsave(fname, x, compress=compress)