Repository: gammapy/gammapy

Issue:
Gammapy test fails with Python 2 and Numpy 1.14
We have a bunch of CI fails that are all Python 2 and appeared with the recent Numpy 1.14 release:
https://travis-ci.org/gammapy/gammapy/builds/328225839

I've looked through them, and they are all due to one bug that was introduced in `np.einsum`, see my comments here https://github.com/astropy/astropy/issues/7051#issuecomment-357468791
It's already fixed in Numpy master (see https://github.com/numpy/numpy/pull/10371), but I'm not sure if that will go out via Numpy 1.14.1 and if so when that will be.

I'm not sure how to get a green CI build again with minimal effort. @bsipocz - Would pinning Python 2 CI builds to Numpy 1.13 be easy to achieve? I think adding skip or xfail markers for all the tests that fail on Python 2 now would be a bit annoying, but of course that could be done as well.



Discussion:
I've temporarily disabled Python 2 builds in our CI in e4e0e972 .
Will turn back on once Numpy 1.14.1 is out, which should be in two weeks (end of January).

Resolving patch:
diff --git a/ah_bootstrap.py b/ah_bootstrap.py
--- a/ah_bootstrap.py
+++ b/ah_bootstrap.py
@@ -19,9 +19,14 @@
 contains an option called ``auto_use`` with a value of ``True``, it will
 automatically call the main function of this module called
 `use_astropy_helpers` (see that function's docstring for full details).
-Otherwise no further action is taken (however,
-``ah_bootstrap.use_astropy_helpers`` may be called manually from within the
-setup.py script).
+Otherwise no further action is taken and by default the system-installed version
+of astropy-helpers will be used (however, ``ah_bootstrap.use_astropy_helpers``
+may be called manually from within the setup.py script).
+
+This behavior can also be controlled using the ``--auto-use`` and
+``--no-auto-use`` command-line flags. For clarity, an alias for
+``--no-auto-use`` is ``--use-system-astropy-helpers``, and we recommend using
+the latter if needed.
 
 Additional options in the ``[ah_boostrap]`` section of setup.cfg have the same
 names as the arguments to `use_astropy_helpers`, and can be used to configure
@@ -137,7 +142,6 @@
 
 from setuptools import Distribution
 from setuptools.package_index import PackageIndex
-from setuptools.sandbox import run_setup
 
 from distutils import log
 from distutils.debug import DEBUG
@@ -147,6 +151,11 @@
 DIST_NAME = 'astropy-helpers'
 PACKAGE_NAME = 'astropy_helpers'
 
+if PY3:
+    UPPER_VERSION_EXCLUSIVE = None
+else:
+    UPPER_VERSION_EXCLUSIVE = '3'
+
 # Defaults for other options
 DOWNLOAD_IF_NEEDED = True
 INDEX_URL = 'https://pypi.python.org/simple'
@@ -287,6 +296,18 @@ def parse_command_line(cls, argv=None):
             config['offline'] = True
             argv.remove('--offline')
 
+        if '--auto-use' in argv:
+            config['auto_use'] = True
+            argv.remove('--auto-use')
+
+        if '--no-auto-use' in argv:
+            config['auto_use'] = False
+            argv.remove('--no-auto-use')
+
+        if '--use-system-astropy-helpers' in argv:
+            config['auto_use'] = False
+            argv.remove('--use-system-astropy-helpers')
+
         return config
 
     def run(self):
@@ -464,9 +485,10 @@ def _directory_import(self):
             # setup.py exists we can generate it
             setup_py = os.path.join(path, 'setup.py')
             if os.path.isfile(setup_py):
-                with _silence():
-                    run_setup(os.path.join(path, 'setup.py'),
-                              ['egg_info'])
+                # We use subprocess instead of run_setup from setuptools to
+                # avoid segmentation faults - see the following for more details:
+                # https://github.com/cython/cython/issues/2104
+                sp.check_output([sys.executable, 'setup.py', 'egg_info'], cwd=path)
 
                 for dist in pkg_resources.find_distributions(path, True):
                     # There should be only one...
@@ -501,16 +523,32 @@ def get_option_dict(self, command_name):
         if version:
             req = '{0}=={1}'.format(DIST_NAME, version)
         else:
-            req = DIST_NAME
+            if UPPER_VERSION_EXCLUSIVE is None:
+                req = DIST_NAME
+            else:
+                req = '{0}<{1}'.format(DIST_NAME, UPPER_VERSION_EXCLUSIVE)
 
         attrs = {'setup_requires': [req]}
 
+        # NOTE: we need to parse the config file (e.g. setup.cfg) to make sure
+        # it honours the options set in the [easy_install] section, and we need
+        # to explicitly fetch the requirement eggs as setup_requires does not
+        # get honored in recent versions of setuptools:
+        # https://github.com/pypa/setuptools/issues/1273
+
         try:
-            if DEBUG:
-                _Distribution(attrs=attrs)
-            else:
-                with _silence():
-                    _Distribution(attrs=attrs)
+
+            context = _verbose if DEBUG else _silence
+            with context():
+                dist = _Distribution(attrs=attrs)
+                try:
+                    dist.parse_config_files(ignore_option_errors=True)
+                    dist.fetch_build_eggs(req)
+                except TypeError:
+                    # On older versions of setuptools, ignore_option_errors
+                    # doesn't exist, and the above two lines are not needed
+                    # so we can just continue
+                    pass
 
             # If the setup_requires succeeded it will have added the new dist to
             # the main working_set
@@ -846,6 +884,10 @@ def flush(self):
         pass
 
 
+@contextlib.contextmanager
+def _verbose():
+    yield
+
 @contextlib.contextmanager
 def _silence():
     """A context manager that silences sys.stdout and sys.stderr."""


Test patch: