Repository: spack/spack

Issue:
can't build py-mccabe (and py-flake8) on osx
On OSX (10.11.6), I can't use spack to install `py-flake8` b/c `py-mccabe` (one of flake8's dependencies) fails.

Issue looks like a problem with `pytest-runner` being needed.

I tried adding `pytest-runner`, but it failed to build & thats as far down as the rabbit hole I had time to explore.


Full Details Below.

**spack install py-flake8**
```
==> python is already installed in /Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/python-2.7.14-5vxw6f2k4hjdnk7fi6tulhtrasixblki
==> py-enum34 is already installed in /Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-enum34-1.1.6-hdwea3dsqtekcgnnwje3sg6m7zvqvss3
==> python is already installed in /Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/python-2.7.14-5vxw6f2k4hjdnk7fi6tulhtrasixblki
==> py-pytest is already installed in /Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-pytest-3.0.7-jk7mvh7cjoj4ii4uj57aeud3bwg2b737
==> py-setuptools is already installed in /Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275
==> Installing py-mccabe
==> Using cached archive: /Users/harrison37/Work/2017_09_conduit_spack_update/spack/var/spack/cache/py-mccabe/py-mccabe-0.5.2.tar.gz
==> Already staged py-mccabe-0.5.2-2svvfdbb5hhuzqydx37fmodrxcr47a64 in /Users/harrison37/Work/2017_09_conduit_spack_update/spack/var/spack/stage/py-mccabe-0.5.2-2svvfdbb5hhuzqydx37fmodrxcr47a64
==> No patches needed for py-mccabe
==> Building py-mccabe [PythonPackage]
==> Executing phase: 'build'
==> Error: ProcessError: Command exited with status 1:
    '/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/python-2.7.14-5vxw6f2k4hjdnk7fi6tulhtrasixblki/bin/python2.7' 'setup.py' '--no-user-cfg' 'build'
See build log for details:
  /Users/harrison37/Work/2017_09_conduit_spack_update/spack/var/spack/stage/py-mccabe-0.5.2-2svvfdbb5hhuzqydx37fmodrxcr47a64/mccabe-0.5.2/spack-build.out
```

**log with py-mccabe build error (w/o pytest-runner)**

```
No local packages or working download links found for pytest-runner
Traceback (most recent call last):
  File "setup.py", line 57, in <module>
    'Topic :: Software Development :: Quality Assurance',
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/python-2.7.14-5vxw6f2k4hjdnk7fi6tulhtrasixblki/lib/python2.7/distutils/core.py", line 111, in setup
    _setup_distribution = dist = klass(attrs)
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/setuptools/dist.py", line 318, in __init__
    self.fetch_build_eggs(attrs['setup_requires'])
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/setuptools/dist.py", line 375, in fetch_build_eggs
    replace_conflicting=True,
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/pkg_resources/__init__.py", line 851, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1123, in best_match
    return self.obtain(req, installer)
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1135, in obtain
    return installer(requirement)
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/setuptools/dist.py", line 443, in fetch_build_egg
    return cmd.easy_install(req)
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 667, in easy_install
    raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pytest-runner')
```

**log of pytest-runner build error**
```
No local packages or working download links found for setuptools_scm>=1.15.0
Traceback (most recent call last):
  File "setup.py", line 49, in <module>
    setuptools.setup(**params)
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/python-2.7.14-5vxw6f2k4hjdnk7fi6tulhtrasixblki/lib/python2.7/distutils/core.py", line 111, in setup
    _setup_distribution = dist = klass(attrs)
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/setuptools/dist.py", line 318, in __init__
    self.fetch_build_eggs(attrs['setup_requires'])
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/setuptools/dist.py", line 375, in fetch_build_eggs
    replace_conflicting=True,
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/pkg_resources/__init__.py", line 851, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1123, in best_match
    return self.obtain(req, installer)
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1135, in obtain
    return installer(requirement)
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/setuptools/dist.py", line 443, in fetch_build_egg
    return cmd.easy_install(req)
  File "/Users/harrison37/Work/2017_09_conduit_spack_update/spack/opt/spack/darwin-elcapitan-x86_64/clang-7.3.0-apple/py-setuptools-35.0.2-2jxxsgfojlrnbs52bshmraxsjbhzt275/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 667, in easy_install
    raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('setuptools_scm>=1.15.0')

```


Discussion:
@cyrush : Not sure about above errors but I am able to build on Sierra from scratch as:

```
$ spack install py-flake8 ^python@2.7.14
...
$ spack find
==> 19 installed packages.
-- darwin-sierra-x86_64 / clang@8.1.0-apple ---------------------
bzip2@1.0.6        py-appdirs@1.4.3       py-mccabe@0.5.2       py-pyparsing@2.2.0    readline@7.0
ncurses@6.0        py-configparser@3.5.0  py-packaging@16.8     py-setuptools@35.0.2  sqlite@3.20.0
openssl@1.0.2k     py-enum34@1.1.6        py-pycodestyle@2.0.0  py-six@1.10.0         zlib@1.2.11
pkg-config@0.29.2  py-flake8@3.0.4        py-pyflakes@1.2.3     python@2.7.14
```
@pramodk, thanks just tried with a fresh spack clone and still encountered the error. 
Doesn't look like an osx version specific issue, but who knows ... 

Anything in your `packages.yaml` (as external package) that affecting build?

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/py-flake8/package.py b/var/spack/repos/builtin/packages/py-flake8/package.py
--- a/var/spack/repos/builtin/packages/py-flake8/package.py
+++ b/var/spack/repos/builtin/packages/py-flake8/package.py
@@ -66,3 +66,7 @@ class PyFlake8(PythonPackage):
 
     # TODO: Add test dependencies
     # depends_on('py-nose', type='test')
+
+    def patch(self):
+        """Filter pytest-runner requirement out of setup.py."""
+        filter_file("['pytest-runner']", "[]", 'setup.py', string=True)
diff --git a/var/spack/repos/builtin/packages/py-mccabe/package.py b/var/spack/repos/builtin/packages/py-mccabe/package.py
--- a/var/spack/repos/builtin/packages/py-mccabe/package.py
+++ b/var/spack/repos/builtin/packages/py-mccabe/package.py
@@ -47,3 +47,7 @@ class PyMccabe(PythonPackage):
 
     # TODO: Add test dependencies
     # depends_on('py-pytest', type='test')
+
+    def patch(self):
+        """Filter pytest-runner requirement out of setup.py."""
+        filter_file("['pytest-runner']", "[]", 'setup.py', string=True)


Test patch: