Repository: mne-tools/mne-python

Issue:
Can't import mne when installing from pip
For various reasons I don't want to install MNE via the [instructions](https://martinos.org/mne/stable/install_mne_python.html) in the documentation, but rather by using pip. However, the package on PyPI is not setup correctly to install dependencies. By simply trying to `import mne`, I get the following:

```python
>>> import mne
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/depalati/miniconda3/envs/thetamod/lib/python3.6/site-packages/mne/__init__.py", line 25, in <module>
    from .io.pick import (pick_types, pick_channels,
  File "/Users/depalati/miniconda3/envs/thetamod/lib/python3.6/site-packages/mne/io/__init__.py", line 15, in <module>
    from .base import BaseRaw
  File "/Users/depalati/miniconda3/envs/thetamod/lib/python3.6/site-packages/mne/io/base.py", line 22, in <module>
    from ..channels.channels import (ContainsMixin, UpdateChannelsMixin,
  File "/Users/depalati/miniconda3/envs/thetamod/lib/python3.6/site-packages/mne/channels/__init__.py", line 8, in <module>
    from .montage import (read_montage, read_dig_montage, Montage, DigMontage,
  File "/Users/depalati/miniconda3/envs/thetamod/lib/python3.6/site-packages/mne/channels/montage.py", line 19, in <module>
    from ..viz import plot_montage
  File "/Users/depalati/miniconda3/envs/thetamod/lib/python3.6/site-packages/mne/viz/__init__.py", line 6, in <module>
    from .topo import plot_topo_image_epochs, iter_topography
  File "/Users/depalati/miniconda3/envs/thetamod/lib/python3.6/site-packages/mne/viz/topo.py", line 14, in <module>
    from matplotlib.colors import colorConverter
ModuleNotFoundError: No module named 'matplotlib'
```

This can be fixed by including dependencies with the `install_requires` keyword argument in `setup.py`. Alternatively, why is `mne/__init__.py` importing so many things in the first place? If only a small subset of MNE's functionality is required, why should a user have to install so many unused dependencies?


Discussion:
> This can be fixed by including dependencies with the install_requires keyword argument in setup.py. 

It used to be a bad idea to include numpy/scipy/matplotlib in `install_requires` because of complications with building them properly. Maybe nowadays with wheels it's not a big problem.

> Alternatively, why is mne/__init__.py importing so many things in the first place? If only a small subset of MNE's functionality is required, why should a user have to install so many unused dependencies?

Our assumption is that the vast majority of people using MNE will have (at minimum) a standard scientific Python stack installed, meaning numpy/scipy/matplotlib are already there. Having `import mne` import all namespaces makes things easier for users, and having top-level imports for things like `matplotlib` makes things easier for maintainers. 
> It used to be a bad idea to include numpy/scipy/matplotlib in install_requires because of complications with building them properly. Maybe nowadays with wheels it's not a big problem.

My feeling is that users who want to install with pip (or with `setup.py` directly after cloning this repo) probably know what they're getting into and can handle these complications (although as you say, with wheels it turns out not to be a big issue).

It certainly makes sense for the average MNE user to follow the conda route, although I'm not fond of the recommendation to create an environment from a YAML file since I have had issues with reproducibility there (generally related to platform-specific packages getting listed and breaking things on other platforms). On that note, I initially tried to install MNE with the conda-forge package, but that appears to be outdated and requires legacy Python.
btw matplotlib should not be a strong dependency... only numpy and scipy.

what do you recommend? we went the conda route because we had issues
with pip and mayavi/vtk.

regarding conda package if people are still tempted to do "conda
install mne" and it's not maintained we should remove the conda
package.

> btw matplotlib should not be a strong dependency... only numpy and scipy.

It is currently due to our import habits, and we list it as such in the [README](https://github.com/mne-tools/mne-python/blob/master/README.rst#dependencies). Do you propose we nest all these imports? It is certainly doable... perhaps there are not that many.

> It certainly makes sense for the average MNE user to follow the conda route, although I'm not fond of the recommendation to create an environment from a YAML file since I have had issues with reproducibility there (generally related to platform-specific packages getting listed and breaking things on other platforms). 

FWIW we use continuous integration on Linux/macOS/AppVeyor (Windows) with this file and it has been pretty stable, at least for the last ~6 mo (since we switched all to a unified 3.6-based env). I work with people who use MNE on all these platforms and it seems to work well. Of course YMMV, but we're happy to hear feedback and tweak.

Yes `conda-forge` is a bit behind because the dependency stack for `mayavi` involving `traits/traitsui/pyface` plus at least version 4.6 (which just came out) has forced 2.7 before (and/or an ancient matplotlib < 2.0?) and might still.
The conda-forge package for `mayavi` appears to work for multiple Python versions: https://anaconda.org/conda-forge/mayavi/files

Resolving patch:
diff --git a/mne/viz/topo.py b/mne/viz/topo.py
--- a/mne/viz/topo.py
+++ b/mne/viz/topo.py
@@ -8,10 +8,10 @@
 #
 # License: Simplified BSD
 
+from copy import deepcopy
 from functools import partial
 from itertools import cycle
-from copy import deepcopy
-from matplotlib.colors import colorConverter
+
 import numpy as np
 
 from ..io.constants import Bunch
@@ -341,6 +341,7 @@ def _plot_timeseries(ax, ch_idx, tmin, tmax, vmin, vmax, ylim, data, color,
                      labels=None):
     """Show time series on topo split across multiple axes."""
     import matplotlib.pyplot as plt
+    from matplotlib.colors import colorConverter
     picker_flag = False
     for data_, color_ in zip(data, color):
         if not picker_flag:


Test patch:
diff --git a/mne/tests/test_import_nesting.py b/mne/tests/test_import_nesting.py
--- a/mne/tests/test_import_nesting.py
+++ b/mne/tests/test_import_nesting.py
@@ -1,7 +1,7 @@
 import sys
 from subprocess import Popen, PIPE
 
-from mne.utils import run_tests_if_main, requires_version
+from mne.utils import run_tests_if_main
 
 
 run_script = """
@@ -22,35 +22,37 @@
                        and sys.modules[x] is not None)
 bad = scipy_submodules - ok_scipy_submodules
 if len(bad) > 0:
-    out.append('Found un-nested scipy submodules: %s' % list(bad))
+    out.append('scipy submodules: %s' % list(bad))
 
 # check sklearn and others
-_sklearn = _pandas = _nose = _mayavi = False
+_sklearn = _pandas = _nose = _mayavi = _matplotlib = False
 for x in sys.modules.keys():
     if x.startswith('sklearn') and not _sklearn:
-        out.append('Found un-nested sklearn import')
+        out.append('sklearn')
         _sklearn = True
     if x.startswith('pandas') and not _pandas:
-        out.append('Found un-nested pandas import')
+        out.append('pandas')
         _pandas = True
     if x.startswith('nose') and not _nose:
-        out.append('Found un-nested nose import')
+        out.append('nose')
         _nose = True
     if x.startswith('mayavi') and not _mayavi:
-        out.append('Found un-nested mayavi import')
+        out.append('mayavi')
         _mayavi = True
+    if x.startswith('matplotlib') and not _matplotlib:
+        out.append('matplotlib')
+        _matplotlib = True
 if len(out) > 0:
-    print('\\n' + '\\n'.join(out), end='')
+    print('\\nFound un-nested imports for:\\n' + '\\n'.join(out), end='')
     exit(1)
 """
 
 
-@requires_version('scipy', '0.11')  # old ones not organized properly
 def test_module_nesting():
-    """Test that module imports are necessary
-    """
+    """Test that module imports are properly nested."""
     proc = Popen([sys.executable, '-c', run_script], stdout=PIPE, stderr=PIPE)
     stdout, stderr = proc.communicate()
+    stdout = stdout.decode('utf-8')
     if proc.returncode:
         raise AssertionError(stdout)