Repository: spack/spack

Issue:
Installation issue: py-multiqc -- help debugging concretization/dependency quandary
A friend reported trouble building py-multiqc on his mac, which set me on a wild goose chase.  He claims it builds on linux, but I'm w/out a linux system at the moment to verify.

My immediate question is, how can I usefully debug a problem with dependencies?

On a mac running 10.13.6, on `develop`, in a tree that I updated this afternoon....

```console
(alice)[17:02:25]spack>>./bin/spack spec py-multiqc
Input spec
--------------------------------
py-multiqc

Concretized
--------------------------------
==> Warning: clang@9.1.0-apple cannot build optimized binaries for "haswell". Using best target possible: "x86_64"
==> Error: An unsatisfiable version constraint has been detected for spec:

    python@3.7.4%clang@9.1.0-apple+bz2+ctypes+dbm+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline~shared+sqlite3+ssl~tix~tkinter~ucs4~uuid+zlib arch=darwin-highsierra-x86_64
        ^bzip2@1.0.8%clang@9.1.0-apple+shared arch=darwin-highsierra-x86_64
            ^diffutils@3.7%clang@9.1.0-apple arch=darwin-highsierra-x86_64
                ^libiconv@1.16%clang@9.1.0-apple arch=darwin-highsierra-x86_64
        ^expat
        ^gdbm
            ^readline
                ^ncurses
                    ^pkgconf@1.6.3%clang@9.1.0-apple arch=darwin-highsierra-x86_64
        ^gettext@0.20.1%clang@9.1.0-apple+bzip2+curses+git~libunistring+libxml2+tar+xz arch=darwin-highsierra-x86_64
            ^libxml2
                ^xz
                ^zlib@1.2.11%clang@9.1.0-apple+optimize+pic+shared arch=darwin-highsierra-x86_64
            ^tar
        ^libffi
        ^openssl@1.0.2:
            ^perl@5.14.0:
        ^sqlite@3.0.8:


while trying to concretize the partial spec:

    py-backports-functools-lru-cache


py-backports-functools-lru-cache requires python version 2.6.0:3.3.99, but spec asked for 3.7.4
(alice)[17:02:29]spack>>
``` 

I started off dropping calls to `pdb.set_trace()` into various bits of `spec.py` and poking around, but it's all twisty turny.  It seems that `py-matplotlib` is constrained to versions that require python2-ish, which is what drags in the `py-backports-functools-lru-cache`, but `py-matplotlib` *also* wants `py-numpy` and it seems to want python3-ish.

Here's an example trying to build a spec that i picked out of the `py-multiqc` spec while poking around:

```console
(alice)[17:12:17]spack>>./bin/spack spec py-matplotlib@2.0.0:2.9.9
Input spec
--------------------------------
py-matplotlib@2.0.0:2.9.9

Concretized
--------------------------------
==> Warning: clang@9.1.0-apple cannot build optimized binaries for "haswell". Using best target possible: "x86_64"
==> Error: An unsatisfiable version constraint has been detected for spec:

    python@3.7.4%clang@9.1.0-apple+bz2+ctypes+dbm+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline~shared+sqlite3+ssl~tix~tkinter~ucs4~uuid+zlib arch=darwin-highsierra-x86_64
        ^bzip2@1.0.8%clang@9.1.0-apple+shared arch=darwin-highsierra-x86_64
            ^diffutils@3.7%clang@9.1.0-apple arch=darwin-highsierra-x86_64
                ^libiconv@1.16%clang@9.1.0-apple arch=darwin-highsierra-x86_64
        ^expat
        ^gdbm
            ^readline
                ^ncurses
                    ^pkgconf@1.6.3%clang@9.1.0-apple arch=darwin-highsierra-x86_64
        ^gettext@0.20.1%clang@9.1.0-apple+bzip2+curses+git~libunistring+libxml2+tar+xz arch=darwin-highsierra-x86_64
            ^libxml2
                ^xz
                ^zlib@1.2.11%clang@9.1.0-apple+optimize+pic+shared arch=darwin-highsierra-x86_64
            ^tar
        ^libffi
        ^openssl@1.0.2:
            ^perl@5.14.0:
        ^sqlite@3.0.8:


while trying to concretize the partial spec:

    py-backports-functools-lru-cache


py-backports-functools-lru-cache requires python version 2.6.0:3.3.99, but spec asked for 3.7.4
```

Is there some way to trace the progress of concretization?

Is there some way to see the dependency graph as it stands when the error happened?

Is there some way to see where the "spec asked for 3.7.4", because the input certainly didn't.

**Thanks**, as always.




Discussion:
ping @odoublewen 
> It seems that `py-matplotlib` is constrained to versions that require python2-ish,

I seemed to have missed the ':' in the version spec in for python in py-matplotlib, so this would seem to not be accurate (but the empirical observations still stand).
> Is there some way to trace the progress of concretization?

There is currently no way to print out the intermediate decisions of the concretizer as it proceeds along. We could add that (that's usually what I insert myself when it comes to tracking concretizer bugs).

> Is there some way to see the dependency graph as it stands when the error happened?

Currently no; great idea though (we could add a try catch at the top level to print out the spec at the time of the error)

My current strategy mainly involves `grep` relating to the associated complaints, for example looking for packages that depend on `py-functools-backports-lru-cache` (and likewise I may do this for the conflicting versions that it mentions too). In this case that is `py-matplotlib`.

I think in this case the issue is that `py-matplotlib` dependency on `py-functools-backports-lru-cache` needs to be conditioned on `^python@:2` like:

```
    depends_on('py-backports-functools-lru-cache', type=('build', 'run'),
            when='@2.1.0:2.999.999 ^python@:2')
```

That is, when building `@2.1.0:2.999.999` with `python@3:` this was still including the backports library. This doesn't specifically address the more interesting question you have though about how we could report this better and present more information to the user (and I think your first two questions suggest an easy path towards improving this somewhat).

As a side note `py-multiqc` appears to similarly have an unqualified dependency on `py-enum34`.
@scheibelp -- That's pretty  much what I did, starting with a breakpoint [here in `_merge_dependency`](https://github.com/spack/spack/blob/develop/lib/spack/spack/spec.py#L2506) (based on ripgrepping around a bit).  But printing out the various things from the debugger is messy; I suspect that there are helper methods or *etc.* that would make e.g. `spec_deps` print out prettyily.

I probably have the time to PR something towards the interesting question (not that *your* problem's *not* interesting, @odoublewen...) if you can make some suggestions.
I'm happy to work on it myself if folks have suggestions (and what we have already I think is a good start toward improving the debugging process), although I wouldn't be able to start until December most likely.
Your suggestion re py-matplotlib fixes the immediate problem.  I'll PR it and see if @odoublewen can test it in a couple of scenarios.

> As a side note `py-multiqc` appears to similarly have an unqualified dependency on `py-enum34`.

So, [this pypi page](https://pypi.org/project/enum34/) suggests that it's only needed for `^python@:3.3`.  Is that correct?  I can PR that too.  Or, since we don't have a failure scenario for it I should just drop in a comment?
> I'm happy to work on it myself

Don't let me steal your toys.  I'm not in any hurry (indeed, I'm not using Spack for anything at the moment...).

On the other hand, I have spare time so if you run out of TUITs, let me know.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/py-enum34/package.py b/var/spack/repos/builtin/packages/py-enum34/package.py
--- a/var/spack/repos/builtin/packages/py-enum34/package.py
+++ b/var/spack/repos/builtin/packages/py-enum34/package.py
@@ -15,6 +15,5 @@ class PyEnum34(PythonPackage):
     version('1.1.6', sha256='8ad8c4783bf61ded74527bffb48ed9b54166685e4230386a9ed9b1279e2df5b1')
 
     depends_on('python')
-    conflicts('python@3.4:')
     depends_on('py-ordereddict', when='^python@:2.6', type=('build', 'run'))
     depends_on('py-setuptools', type='build')
diff --git a/var/spack/repos/builtin/packages/py-matplotlib/package.py b/var/spack/repos/builtin/packages/py-matplotlib/package.py
--- a/var/spack/repos/builtin/packages/py-matplotlib/package.py
+++ b/var/spack/repos/builtin/packages/py-matplotlib/package.py
@@ -85,7 +85,7 @@ class PyMatplotlib(PythonPackage):
     depends_on('py-subprocess32', type=('build', 'run'), when='^python@:2.7')
     depends_on('py-functools32', type=('build', 'run'), when='@:2.0.999 ^python@2.7')
     depends_on('py-backports-functools-lru-cache', type=('build', 'run'),
-               when='@2.1.0:2.999.999')
+               when='@2.1.0:2.999.999 ^python@:2')
     depends_on('py-six@1.9.0:', type=('build', 'run'), when='@:2')
 
     # Optional backend dependencies
diff --git a/var/spack/repos/builtin/packages/py-multiqc/package.py b/var/spack/repos/builtin/packages/py-multiqc/package.py
--- a/var/spack/repos/builtin/packages/py-multiqc/package.py
+++ b/var/spack/repos/builtin/packages/py-multiqc/package.py
@@ -31,5 +31,6 @@ class PyMultiqc(PythonPackage):
     depends_on('py-pyyaml', type=('build', 'run'))
     depends_on('py-simplejson', type=('build', 'run'))
     depends_on('py-requests', type=('build', 'run'), when='@1.5:')
-    depends_on('py-enum34', type=('build', 'run'), when='@1.5:')
+    depends_on('py-enum34', type=('build', 'run'), when='@1.4:1.5 ^python@:3.3')
+    depends_on('py-enum34', type=('build', 'run'), when='@1.3')
     depends_on('py-markdown', type=('build', 'run'), when='@1.5:')


Test patch: