Repository: spack/spack

Issue:
Installation issue: py-horovod
The py-horovod package doesn't seem to actually install anything.  Maybe @rvinaybharadwaj or @adamjstewart have an idea on how to fix it?

### Spack version
```console
$ spack --version
0.14.0-247-958f260
```

### Steps to reproduce the issue

```console
$ spack install py-horovod
$ find ~/workspace/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx | grep -v \\.spack
~/workspace/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx
~/workspace/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/lib
~/workspace/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/lib/perl5
~/workspace/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/lib/python3.7
~/workspace/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/lib/python3.7/site-packages
```

### Platform and user environment

```commandline
$ uname -a 
Linux --- 3.10.0-1062.9.1.1chaos.ch6.x86_64 #1 SMP Mon Dec 16 13:50:10 PST 2019 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -d
Description:	Red Hat Enterprise Linux Server release 7.7 (Maipo)
``` 

### Additional information

```console
$ spack spec --install-status py-horovod
Input spec
--------------------------------
 -   py-horovod

Concretized
--------------------------------
[+]  py-horovod@0.19.0%gcc@7.3.0+cuda~gloo+mpi~mxnet+pytorch arch=linux-rhel7-haswell
[+]      ^cuda@10.1.168%gcc@7.3.0 arch=linux-rhel7-haswell
[+]      ^mvapich2@2.3%gcc@7.3.0~alloca ch3_rank_bits=32 ~cuda~debug fabrics=mrail file_systems=auto process_managers=auto +regcache threads=multiple +wrapperrpath arch=linux-rhel7-haswell
[+]      ^py-cloudpickle@1.2.1%gcc@7.3.0 arch=linux-rhel7-haswell
[+]          ^py-setuptools@41.4.0%gcc@7.3.0 arch=linux-rhel7-haswell
[+]              ^python@3.7.6%gcc@7.3.0+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4~uuid+zlib arch=linux-rhel7-haswell
[+]                  ^bzip2@1.0.8%gcc@7.3.0+shared arch=linux-rhel7-haswell
[+]                      ^diffutils@3.7%gcc@7.3.0 arch=linux-rhel7-haswell
[+]                          ^libiconv@1.16%gcc@7.3.0 arch=linux-rhel7-haswell
[+]                  ^expat@2.2.9%gcc@7.3.0+libbsd arch=linux-rhel7-haswell
[+]                      ^libbsd@0.10.0%gcc@7.3.0 arch=linux-rhel7-haswell
[+]                  ^gdbm@1.18.1%gcc@7.3.0 arch=linux-rhel7-haswell
[+]                      ^readline@8.0%gcc@7.3.0 arch=linux-rhel7-haswell
[+]                          ^ncurses@6.2%gcc@7.3.0~symlinks+termlib arch=linux-rhel7-haswell
[+]                              ^pkgconf@1.6.3%gcc@7.3.0 arch=linux-rhel7-haswell
[+]                  ^gettext@0.20.1%gcc@7.3.0+bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-rhel7-haswell
[+]                      ^libxml2@2.9.9%gcc@7.3.0~python arch=linux-rhel7-haswell
[+]                          ^xz@5.2.4%gcc@7.3.0 arch=linux-rhel7-haswell
[+]                          ^zlib@1.2.11%gcc@7.3.0+optimize+pic+shared arch=linux-rhel7-haswell
[+]                      ^tar@1.32%gcc@7.3.0 arch=linux-rhel7-haswell
[+]                  ^libffi@3.2.1%gcc@7.3.0 arch=linux-rhel7-haswell
[+]                  ^openssl@1.1.1d%gcc@7.3.0+systemcerts arch=linux-rhel7-haswell
[+]                      ^perl@5.30.1%gcc@7.3.0+cpanm+shared+threads arch=linux-rhel7-haswell
[+]                  ^sqlite@3.30.1%gcc@7.3.0~column_metadata+fts~functions~rtree arch=linux-rhel7-haswell
[+]      ^py-pip@19.3%gcc@7.3.0 arch=linux-rhel7-haswell
[+]      ^py-psutil@5.6.3%gcc@7.3.0 arch=linux-rhel7-haswell
[+]      ^py-pyyaml@5.1.2%gcc@7.3.0 arch=linux-rhel7-haswell
[+]          ^libyaml@0.2.2%gcc@7.3.0 arch=linux-rhel7-haswell
[+]      ^py-six@1.12.0%gcc@7.3.0 arch=linux-rhel7-haswell
[+]      ^py-torch@1.4.0%gcc@7.3.0~binary~caffe2+cuda cuda_arch=37 +cudnn+distributed~fbgemm~ffmpeg~gloo~leveldb~lmdb~magma~miopen+mkldnn+nccl~nnpack~opencv+openmp~qnnpack~redis~tbb~test~zstd arch=linux-rhel7-haswell
[+]          ^cmake@3.12.1%gcc@7.3.0~doc~ncurses~openssl+ownlibs patches=93488edf3f89cef6f3c77e821007c022e88e9f1f566ae9bf824a87ecbac97ebb,dd3a40d4d92f6b2158b87d6fb354c277947c776424aa03f6dc8096cf3135f5d0 ~qt arch=linux-rhel7-haswell
[+]          ^cudnn@7.6.5.32-10.1-linux-x64%gcc@7.3.0 arch=linux-rhel7-haswell
[+]          ^eigen@3.3.7%gcc@7.3.0 build_type=RelWithDebInfo arch=linux-rhel7-haswell
[+]          ^nccl@2.4.8-1%gcc@7.3.0 patches=42778c78eb9875dacddf5eca20f7f6a077773fcbee41e51174f81b3143684b6d arch=linux-rhel7-haswell
[+]          ^ninja@1.10.0%gcc@7.3.0 arch=linux-rhel7-haswell
[+]          ^openblas@0.3.6%gcc@7.3.0~consistent_fpcsr~ilp64+pic+shared threads=openmp arch=linux-rhel7-haswell
[+]          ^protobuf@3.7.1%gcc@7.3.0 build_type=Release +shared arch=linux-rhel7-haswell
[+]          ^py-numpy@1.16.2%gcc@7.3.0+blas+lapack patches=196dc7cc8e1367de199726d4e005bba50c3d13fb2843c4753c5f07f7894f10c9 arch=linux-rhel7-haswell
[+]          ^py-pybind11@2.4.3%gcc@7.3.0 build_type=RelWithDebInfo arch=linux-rhel7-haswell
```

### General information

- [X] I have run `spack --version` and reported the version of Spack
- [X] I have searched the issues of this repo and believe this is not a duplicate



Discussion:
What happens when you remove the `phases` and `install` methods and use the defaults?
Lots of errors complaining about gloo which should be disabled.

```console
$ spack install -j1 py-horovod
==> Warning: Missing a source id for cuda@10.1.168
==> cuda@10.1.168 : externally installed in /usr/tce/packages/cuda/cuda-10.1.168
==> cuda@10.1.168 : already registered in DB
[+] /usr/tce/packages/cuda/cuda-10.1.168
==> mvapich2@2.3 : externally installed in /usr/tce/packages/mvapich2/mvapich2-2.3-gcc-7.3.0
==> mvapich2@2.3 : already registered in DB
[+] /usr/tce/packages/mvapich2/mvapich2-2.3-gcc-7.3.0
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/libiconv-1.16-l3icexstz6tu2sny5evx3ajl67zijcrc
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/libbsd-0.10.0-jyortedd7orfkvxlxczjaly7lmpqa73y
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/pkgconf-1.6.3-zv2mrlwzio5623s7b7tfvkfrwohpty6o
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/xz-5.2.4-tvwc7ln3jp7rvdtb63ioqwnbe5uw2kc4
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/zlib-1.2.11-5ahcbau7qgzvredwqnvvdsdcamgmylva
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/libffi-3.2.1-53azu7oryfzqnutxrroidskuzyzfde4l
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/libyaml-0.2.2-avdzbmm2goeqvxnzzb43w2pgf3oer37t
==> cmake@3.12.1 : externally installed in /usr/tce/packages/cmake/cmake-3.12.1
==> cmake@3.12.1 : already registered in DB
[+] /usr/tce/packages/cmake/cmake-3.12.1
==> cudnn@7.6.5.32-10.1-linux-x64 : externally installed in /collab/usr/global/tools/nvidia/cudnn/toss_3_x86_64_ib/cudnn-7.6.5
==> cudnn@7.6.5.32-10.1-linux-x64 : already registered in DB
[+] /collab/usr/global/tools/nvidia/cudnn/toss_3_x86_64_ib/cudnn-7.6.5
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/openblas-0.3.6-mo2eqkltg46cfk4rpjiq6heh63osdf7b
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/nccl-2.4.8-1-jqb6nrnwtjzmb2hgi4ftreasgfyra2hy
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/diffutils-3.7-cfr5q3ulxnhsfhcxc2g22lgdcajjgkf3
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/tar-1.32-c3yg6bsalyrg6zm7elpzmbvbc55rqtx6
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/expat-2.2.9-dvi4ehmtbsgkmv5bmhj7vhg5wakonwea
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/ncurses-6.2-yafmux3oawr3f5oxn5iiueqzoz7gsv6y
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/libxml2-2.9.9-fgdzkl4undeizhkvr3ir7u4stfqe7cx4
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/protobuf-3.7.1-uaeegb2ow6rf7byhcxnaqsmsrshjuvp7
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/eigen-3.3.7-e2zt37k7kwsnchmqzugzurh3c55776ea
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/bzip2-1.0.8-guntphakfzwdadwoarytrs6sfwnmjkj4
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/readline-8.0-zvy7s4ajexgti6z6s435tj5uym3kbzv5
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gettext-0.20.1-ynbfinwmvvow7r2zb6ett4drfpplfhrg
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/sqlite-3.30.1-wa6o2aujgndhs3sanjoehef2quowolma
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gdbm-1.18.1-ajku7bufwfyf2e3nmytn6cugntvoipjg
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/perl-5.30.1-ev5w3zz7grkdv7v7ikwr773qc5ta7dcm
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/openssl-1.1.1d-jswws3imh7jr7233frgiwsmt2blijjjm
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-setuptools-41.4.0-nwdy4bqkant2kvsfc2pnlinxd7twjq44
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-pyyaml-5.1.2-wto5bqcy3hlydi6tobgjpie7r36nyugy
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-six-1.12.0-qf6bxlusmk2rhpduqotagqcfrgd56dcd
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/ninja-1.10.0-7wbth22ceo5tpof74z6htxduw3mwbcpx
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-numpy-1.16.2-r7eqijjvzq3g27gfxgcr5scqy5dsxqmb
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-psutil-5.6.3-cbxggt3eouvetjmalrbxumswl4rke3lm
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-pybind11-2.4.3-htlmnefx32ihaazqzjbkm5fnqvktxtgx
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-cloudpickle-1.2.1-hxegbqzs3i7da2g5p4lf6x56f4p2mgnm
[+] /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-torch-1.4.0-ywoseu2zix3fhsj7daryg4mernnesvsy
==> 28206: Installing py-horovod
==> Using cached archive: /usr/WS2/blake14/spack/var/spack/cache/_source-cache/archive/0e/0e9fec11cd7f5f39a09f0785d1097cb51c44537ae14c9b4b2578b5cdd21efb9b.tar.gz
==> Staging archive: /var/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/v0.19.0.tar.gz
==> Created stage in /var/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx
==> No patches needed for py-horovod
==> 28206: py-horovod: Building py-horovod [PythonPackage]
==> 28206: py-horovod: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 1:
    '/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/bin/python3.7' '-s' 'setup.py' '--no-user-cfg' 'build'

3 errors found in build log:
     115    /usr/tce/packages/gcc/gcc-7.3.0/bin/gcc -pthread -shared -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gettext-0.20.1-ynbfi
            nwmvvow7r2zb6ett4drfpplfhrg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/readline-8.0-zvy7s4ajexgti6z6s435tj5uym3kbzv5
            /lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/ncurses-6.2-yafmux3oawr3f5oxn5iiueqzoz7gsv6y/lib -L/usr/WS2/blake14/spac
            k/opt/spack/linux-rhel7-haswell/gcc-7.3.0/openssl-1.1.1d-jswws3imh7jr7233frgiwsmt2blijjjm/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-h
            aswell/gcc-7.3.0/sqlite-3.30.1-wa6o2aujgndhs3sanjoehef2quowolma/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gdbm-1.18
            .1-ajku7bufwfyf2e3nmytn6cugntvoipjg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/zlib-1.2.11-5ahcbau7qgzvredwqnvvdsdca
            mgmylva/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/bzip2-1.0.8-guntphakfzwdadwoarytrs6sfwnmjkj4/lib -L/usr/WS2/blake
            14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/xz-5.2.4-tvwc7ln3jp7rvdtb63ioqwnbe5uw2kc4/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-
            haswell/gcc-7.3.0/expat-2.2.9-dvi4ehmtbsgkmv5bmhj7vhg5wakonwea/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/libffi-3.2
            .1-53azu7oryfzqnutxrroidskuzyzfde4l/lib64 build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.o -L/usr/WS2/blake14/spack/opt/spack/linux-r
            hel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/lib -o build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.so
     116    /usr/WS2/blake14/spack/lib/spack/env/gcc/gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -fPIC -I/usr/WS2/blake14/sp
            ack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/include/python3.7m -c build/temp.linux-x86_64-3.7/te
            st_compile/test_link_flags.cc -o build/temp.linux-x86_64-3.7/test_compile/test_link_flags.o
     117    /usr/tce/packages/gcc/gcc-7.3.0/bin/gcc -pthread -shared -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gettext-0.20.1-ynbfi
            nwmvvow7r2zb6ett4drfpplfhrg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/readline-8.0-zvy7s4ajexgti6z6s435tj5uym3kbzv5
            /lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/ncurses-6.2-yafmux3oawr3f5oxn5iiueqzoz7gsv6y/lib -L/usr/WS2/blake14/spac
            k/opt/spack/linux-rhel7-haswell/gcc-7.3.0/openssl-1.1.1d-jswws3imh7jr7233frgiwsmt2blijjjm/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-h
            aswell/gcc-7.3.0/sqlite-3.30.1-wa6o2aujgndhs3sanjoehef2quowolma/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gdbm-1.18
            .1-ajku7bufwfyf2e3nmytn6cugntvoipjg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/zlib-1.2.11-5ahcbau7qgzvredwqnvvdsdca
            mgmylva/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/bzip2-1.0.8-guntphakfzwdadwoarytrs6sfwnmjkj4/lib -L/usr/WS2/blake
            14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/xz-5.2.4-tvwc7ln3jp7rvdtb63ioqwnbe5uw2kc4/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-
            haswell/gcc-7.3.0/expat-2.2.9-dvi4ehmtbsgkmv5bmhj7vhg5wakonwea/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/libffi-3.2
            .1-53azu7oryfzqnutxrroidskuzyzfde4l/lib64 -Wl,--version-script=horovod.lds build/temp.linux-x86_64-3.7/test_compile/test_link_flags.o -L/usr/W
            S2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/lib -o build/temp.linux-x86_64-3.7/test
            _compile/test_link_flags.so
     118    No CUDA runtime is found, using CUDA_HOME='/usr/tce/packages/cuda/cuda-10.1.168'
     119    /usr/WS2/blake14/spack/lib/spack/env/gcc/gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -fPIC -std=c++14 -fPIC -O2
            -Wall -fassociative-math -ffast-math -ftree-vectorize -funsafe-math-optimizations -mf16c -mavx -mfma -I/usr/tce/packages/mvapich2/mvapich2-2.3
            -gcc-7.3.0/include -L/usr/tce/packages/mvapich2/mvapich2-2.3-gcc-7.3.0/lib -lmpicxx -Wl,-rpath -Wl,/usr/tce/packages/mvapich2/mvapich2-2.3-gcc
            -7.3.0/lib -lmpi -I/usr/tce/packages/mvapich2/mvapich2-2.3-gcc-7.3.0/include -L/usr/tce/packages/mvapich2/mvapich2-2.3-gcc-7.3.0/lib -lmpicxx
            -Wl,-rpath -Wl,/usr/tce/packages/mvapich2/mvapich2-2.3-gcc-7.3.0/lib -lmpi -Ithird_party/HTTPRequest/include -Ithird_party/boost/assert/includ
            e -Ithird_party/boost/config/include -Ithird_party/boost/core/include -Ithird_party/boost/detail/include -Ithird_party/boost/iterator/include
            -Ithird_party/boost/lockfree/include -Ithird_party/boost/mpl/include -Ithird_party/boost/parameter/include -Ithird_party/boost/predef/include
            -Ithird_party/boost/preprocessor/include -Ithird_party/boost/static_assert/include -Ithird_party/boost/type_traits/include -Ithird_party/boost
            /utility/include -Ithird_party/eigen -Ithird_party/flatbuffers/include -Ithird_party/lbfgs/include -Ithird_party/gloo -I/usr/WS2/blake14/spack
            /opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-torch-1.4.0-ywoseu2zix3fhsj7daryg4mernnesvsy/lib/python3.7/site-packages/torch/include -I/usr/WS2/
            blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-torch-1.4.0-ywoseu2zix3fhsj7daryg4mernnesvsy/lib/python3.7/site-packages/torch/includ
            e/torch/csrc/api/include -I/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-torch-1.4.0-ywoseu2zix3fhsj7daryg4mernnesvsy/lib/
            python3.7/site-packages/torch/include/TH -I/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-torch-1.4.0-ywoseu2zix3fhsj7daryg
            4mernnesvsy/lib/python3.7/site-packages/torch/include/THC -I/usr/tce/packages/cuda/cuda-10.1.168/include -I/usr/WS2/blake14/spack/opt/spack/li
            nux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/include/python3.7m -c build/temp.linux-x86_64-3.7/test_compile/test_
            torch_cuda.cc -o build/temp.linux-x86_64-3.7/test_compile/test_torch_cuda.o
     120    /usr/tce/packages/gcc/gcc-7.3.0/bin/gcc -pthread -shared -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gettext-0.20.1-ynbfi
            nwmvvow7r2zb6ett4drfpplfhrg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/readline-8.0-zvy7s4ajexgti6z6s435tj5uym3kbzv5
            /lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/ncurses-6.2-yafmux3oawr3f5oxn5iiueqzoz7gsv6y/lib -L/usr/WS2/blake14/spac
            k/opt/spack/linux-rhel7-haswell/gcc-7.3.0/openssl-1.1.1d-jswws3imh7jr7233frgiwsmt2blijjjm/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-h
            aswell/gcc-7.3.0/sqlite-3.30.1-wa6o2aujgndhs3sanjoehef2quowolma/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gdbm-1.18
            .1-ajku7bufwfyf2e3nmytn6cugntvoipjg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/zlib-1.2.11-5ahcbau7qgzvredwqnvvdsdca
            mgmylva/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/bzip2-1.0.8-guntphakfzwdadwoarytrs6sfwnmjkj4/lib -L/usr/WS2/blake
            14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/xz-5.2.4-tvwc7ln3jp7rvdtb63ioqwnbe5uw2kc4/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-
            haswell/gcc-7.3.0/expat-2.2.9-dvi4ehmtbsgkmv5bmhj7vhg5wakonwea/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/libffi-3.2
            .1-53azu7oryfzqnutxrroidskuzyzfde4l/lib64 build/temp.linux-x86_64-3.7/test_compile/test_torch_cuda.o -L/usr/WS2/blake14/spack/opt/spack/linux-
            rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/lib -o build/temp.linux-x86_64-3.7/test_compile/test_torch_cuda.so
  >> 121    CMake Error: The source directory "/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spack-src/third_par
            ty/gloo" does not appear to contain CMakeLists.txt.
     122    Specify --help for usage, or press the help button on the CMake GUI.
  >> 123    CMake Error: The source directory "/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spack-src/third_par
            ty/gloo" does not appear to contain CMakeLists.txt.
     124    Specify --help for usage, or press the help button on the CMake GUI.
     125    /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/lib/python3.7/distutils/dist.py:2
            74: UserWarning: Unknown distribution option: 'test_requires'
     126      warnings.warn(msg)
     127    INFO: Unable to build TensorFlow plugin, will skip it.
     128
     129    Traceback (most recent call last):

     ...

     171      File "setup.py", line 1384, in build_cmake
     172        cwd=build_temp)
     173      File "/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/lib/python3.7/subprocess.
            py", line 363, in check_call
     174        raise CalledProcessError(retcode, cmd)
     175    subprocess.CalledProcessError: Command '['cmake', '/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spa
            ck-src/third_party/gloo', '-DUSE_MPI=ON', '-DCMAKE_BUILD_TYPE=Release', "-DCMAKE_CXX_FLAGS_RELEASE:STRING=''", '-DCMAKE_LIBRARY_OUTPUT_DIRECTO
            RY_RELEASE=/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spack-src/build/lib.linux-x86_64-3.7', '-DC
            MAKE_ARCHIVE_OUTPUT_DIRECTORY_RELEASE=/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spack-src/build/
            temp.linux-x86_64-3.7/lib/mxnet']' returned non-zero exit status 1.
     176
  >> 177    error: None of TensorFlow, PyTorch, or MXNet plugins were built. See errors above.

See build log for details:
  /var/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spack-build-out.txt

==> Error: Failed to install py-horovod due to ChildError: ProcessError: Command exited with status 1:
    '/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/bin/python3.7' '-s' 'setup.py' '--no-user-cfg' 'build'
3 errors found in build log:
     115    /usr/tce/packages/gcc/gcc-7.3.0/bin/gcc -pthread -shared -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gettext-0.20.1-ynbfi
            nwmvvow7r2zb6ett4drfpplfhrg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/readline-8.0-zvy7s4ajexgti6z6s435tj5uym3kbzv5
            /lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/ncurses-6.2-yafmux3oawr3f5oxn5iiueqzoz7gsv6y/lib -L/usr/WS2/blake14/spac
            k/opt/spack/linux-rhel7-haswell/gcc-7.3.0/openssl-1.1.1d-jswws3imh7jr7233frgiwsmt2blijjjm/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-h
            aswell/gcc-7.3.0/sqlite-3.30.1-wa6o2aujgndhs3sanjoehef2quowolma/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gdbm-1.18
            .1-ajku7bufwfyf2e3nmytn6cugntvoipjg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/zlib-1.2.11-5ahcbau7qgzvredwqnvvdsdca
            mgmylva/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/bzip2-1.0.8-guntphakfzwdadwoarytrs6sfwnmjkj4/lib -L/usr/WS2/blake
            14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/xz-5.2.4-tvwc7ln3jp7rvdtb63ioqwnbe5uw2kc4/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-
            haswell/gcc-7.3.0/expat-2.2.9-dvi4ehmtbsgkmv5bmhj7vhg5wakonwea/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/libffi-3.2
            .1-53azu7oryfzqnutxrroidskuzyzfde4l/lib64 build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.o -L/usr/WS2/blake14/spack/opt/spack/linux-r
            hel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/lib -o build/temp.linux-x86_64-3.7/test_compile/test_cpp_flags.so
     116    /usr/WS2/blake14/spack/lib/spack/env/gcc/gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -fPIC -I/usr/WS2/blake14/sp
            ack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/include/python3.7m -c build/temp.linux-x86_64-3.7/te
            st_compile/test_link_flags.cc -o build/temp.linux-x86_64-3.7/test_compile/test_link_flags.o
     117    /usr/tce/packages/gcc/gcc-7.3.0/bin/gcc -pthread -shared -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gettext-0.20.1-ynbfi
            nwmvvow7r2zb6ett4drfpplfhrg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/readline-8.0-zvy7s4ajexgti6z6s435tj5uym3kbzv5
            /lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/ncurses-6.2-yafmux3oawr3f5oxn5iiueqzoz7gsv6y/lib -L/usr/WS2/blake14/spac
            k/opt/spack/linux-rhel7-haswell/gcc-7.3.0/openssl-1.1.1d-jswws3imh7jr7233frgiwsmt2blijjjm/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-h
            aswell/gcc-7.3.0/sqlite-3.30.1-wa6o2aujgndhs3sanjoehef2quowolma/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gdbm-1.18
            .1-ajku7bufwfyf2e3nmytn6cugntvoipjg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/zlib-1.2.11-5ahcbau7qgzvredwqnvvdsdca
            mgmylva/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/bzip2-1.0.8-guntphakfzwdadwoarytrs6sfwnmjkj4/lib -L/usr/WS2/blake
            14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/xz-5.2.4-tvwc7ln3jp7rvdtb63ioqwnbe5uw2kc4/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-
            haswell/gcc-7.3.0/expat-2.2.9-dvi4ehmtbsgkmv5bmhj7vhg5wakonwea/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/libffi-3.2
            .1-53azu7oryfzqnutxrroidskuzyzfde4l/lib64 -Wl,--version-script=horovod.lds build/temp.linux-x86_64-3.7/test_compile/test_link_flags.o -L/usr/W
            S2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/lib -o build/temp.linux-x86_64-3.7/test
            _compile/test_link_flags.so
     118    No CUDA runtime is found, using CUDA_HOME='/usr/tce/packages/cuda/cuda-10.1.168'
     119    /usr/WS2/blake14/spack/lib/spack/env/gcc/gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -fPIC -std=c++14 -fPIC -O2
            -Wall -fassociative-math -ffast-math -ftree-vectorize -funsafe-math-optimizations -mf16c -mavx -mfma -I/usr/tce/packages/mvapich2/mvapich2-2.3
            -gcc-7.3.0/include -L/usr/tce/packages/mvapich2/mvapich2-2.3-gcc-7.3.0/lib -lmpicxx -Wl,-rpath -Wl,/usr/tce/packages/mvapich2/mvapich2-2.3-gcc
            -7.3.0/lib -lmpi -I/usr/tce/packages/mvapich2/mvapich2-2.3-gcc-7.3.0/include -L/usr/tce/packages/mvapich2/mvapich2-2.3-gcc-7.3.0/lib -lmpicxx
            -Wl,-rpath -Wl,/usr/tce/packages/mvapich2/mvapich2-2.3-gcc-7.3.0/lib -lmpi -Ithird_party/HTTPRequest/include -Ithird_party/boost/assert/includ
            e -Ithird_party/boost/config/include -Ithird_party/boost/core/include -Ithird_party/boost/detail/include -Ithird_party/boost/iterator/include
            -Ithird_party/boost/lockfree/include -Ithird_party/boost/mpl/include -Ithird_party/boost/parameter/include -Ithird_party/boost/predef/include
            -Ithird_party/boost/preprocessor/include -Ithird_party/boost/static_assert/include -Ithird_party/boost/type_traits/include -Ithird_party/boost
            /utility/include -Ithird_party/eigen -Ithird_party/flatbuffers/include -Ithird_party/lbfgs/include -Ithird_party/gloo -I/usr/WS2/blake14/spack
            /opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-torch-1.4.0-ywoseu2zix3fhsj7daryg4mernnesvsy/lib/python3.7/site-packages/torch/include -I/usr/WS2/
            blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-torch-1.4.0-ywoseu2zix3fhsj7daryg4mernnesvsy/lib/python3.7/site-packages/torch/includ
            e/torch/csrc/api/include -I/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-torch-1.4.0-ywoseu2zix3fhsj7daryg4mernnesvsy/lib/
            python3.7/site-packages/torch/include/TH -I/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/py-torch-1.4.0-ywoseu2zix3fhsj7daryg
            4mernnesvsy/lib/python3.7/site-packages/torch/include/THC -I/usr/tce/packages/cuda/cuda-10.1.168/include -I/usr/WS2/blake14/spack/opt/spack/li
            nux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/include/python3.7m -c build/temp.linux-x86_64-3.7/test_compile/test_
            torch_cuda.cc -o build/temp.linux-x86_64-3.7/test_compile/test_torch_cuda.o
     120    /usr/tce/packages/gcc/gcc-7.3.0/bin/gcc -pthread -shared -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gettext-0.20.1-ynbfi
            nwmvvow7r2zb6ett4drfpplfhrg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/readline-8.0-zvy7s4ajexgti6z6s435tj5uym3kbzv5
            /lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/ncurses-6.2-yafmux3oawr3f5oxn5iiueqzoz7gsv6y/lib -L/usr/WS2/blake14/spac
            k/opt/spack/linux-rhel7-haswell/gcc-7.3.0/openssl-1.1.1d-jswws3imh7jr7233frgiwsmt2blijjjm/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-h
            aswell/gcc-7.3.0/sqlite-3.30.1-wa6o2aujgndhs3sanjoehef2quowolma/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/gdbm-1.18
            .1-ajku7bufwfyf2e3nmytn6cugntvoipjg/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/zlib-1.2.11-5ahcbau7qgzvredwqnvvdsdca
            mgmylva/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/bzip2-1.0.8-guntphakfzwdadwoarytrs6sfwnmjkj4/lib -L/usr/WS2/blake
            14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/xz-5.2.4-tvwc7ln3jp7rvdtb63ioqwnbe5uw2kc4/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-
            haswell/gcc-7.3.0/expat-2.2.9-dvi4ehmtbsgkmv5bmhj7vhg5wakonwea/lib -L/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/libffi-3.2
            .1-53azu7oryfzqnutxrroidskuzyzfde4l/lib64 build/temp.linux-x86_64-3.7/test_compile/test_torch_cuda.o -L/usr/WS2/blake14/spack/opt/spack/linux-
            rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/lib -o build/temp.linux-x86_64-3.7/test_compile/test_torch_cuda.so
  >> 121    CMake Error: The source directory "/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spack-src/third_par
            ty/gloo" does not appear to contain CMakeLists.txt.
     122    Specify --help for usage, or press the help button on the CMake GUI.
  >> 123    CMake Error: The source directory "/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spack-src/third_par
            ty/gloo" does not appear to contain CMakeLists.txt.
     124    Specify --help for usage, or press the help button on the CMake GUI.
     125    /usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/lib/python3.7/distutils/dist.py:2
            74: UserWarning: Unknown distribution option: 'test_requires'
     126      warnings.warn(msg)
     127    INFO: Unable to build TensorFlow plugin, will skip it.
     128
     129    Traceback (most recent call last):

     ...

     171      File "setup.py", line 1384, in build_cmake
     172        cwd=build_temp)
     173      File "/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/lib/python3.7/subprocess.
            py", line 363, in check_call
     174        raise CalledProcessError(retcode, cmd)
     175    subprocess.CalledProcessError: Command '['cmake', '/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spa
            ck-src/third_party/gloo', '-DUSE_MPI=ON', '-DCMAKE_BUILD_TYPE=Release', "-DCMAKE_CXX_FLAGS_RELEASE:STRING=''", '-DCMAKE_LIBRARY_OUTPUT_DIRECTO
            RY_RELEASE=/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spack-src/build/lib.linux-x86_64-3.7', '-DC
            MAKE_ARCHIVE_OUTPUT_DIRECTORY_RELEASE=/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spack-src/build/
            temp.linux-x86_64-3.7/lib/mxnet']' returned non-zero exit status 1.
     176
  >> 177    error: None of TensorFlow, PyTorch, or MXNet plugins were built. See errors above.

See build log for details:
  /var/tmp/blake14/spack-stage/spack-stage-py-horovod-0.19.0-6bfj56gcf2pegrhj333s3glwks6zvtdx/spack-build-out.txt
Traceback (most recent call last):
  File "/usr/WS2/blake14/spack/lib/spack/spack/build_environment.py", line 801, in child_process
    return_value = function()
  File "/usr/WS2/blake14/spack/lib/spack/spack/installer.py", line 1114, in build_process
    phase(pkg.spec, pkg.prefix)
  File "/usr/WS2/blake14/spack/lib/spack/spack/package.py", line 108, in phase_wrapper
    phase(spec, prefix)
  File "/usr/WS2/blake14/spack/lib/spack/spack/build_systems/python.py", line 153, in build
    self.setup_py('build', *args)
  File "/usr/WS2/blake14/spack/lib/spack/spack/build_systems/python.py", line 121, in setup_py
    self.python('-s', setup, '--no-user-cfg', *args, **kwargs)
  File "/usr/WS2/blake14/spack/lib/spack/spack/build_systems/python.py", line 115, in python
    inspect.getmodule(self).python(*args, **kwargs)
  File "/usr/WS2/blake14/spack/lib/spack/spack/util/executable.py", line 189, in __call__
    proc.returncode, long_msg)
spack.util.executable.ProcessError: Command exited with status 1:
    '/usr/WS2/blake14/spack/opt/spack/linux-rhel7-haswell/gcc-7.3.0/python-3.7.6-7ytlrgx4k2o34advynhberykf4utw27h/bin/python3.7' '-s' 'setup.py' '--no-user-cfg' 'build'

```
That's because we aren't setting the right environment variables. Let me hack on this package, I think there's a lot that's still missing.
Anything I can do to help?  I'm trying to be a better spack contributor.
@rblake-llnl you can test and review my PR when it's finished, should be done any minute now.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/mxnet/package.py b/var/spack/repos/builtin/packages/mxnet/package.py
--- a/var/spack/repos/builtin/packages/mxnet/package.py
+++ b/var/spack/repos/builtin/packages/mxnet/package.py
@@ -13,6 +13,8 @@ class Mxnet(MakefilePackage):
     homepage = "http://mxnet.io"
     url      = "https://github.com/apache/incubator-mxnet/releases/download/1.3.0/apache-mxnet-src-1.3.0-incubating.tar.gz"
 
+    maintainers = ['adamjstewart']
+
     version('1.3.0', sha256='c00d6fbb2947144ce36c835308e603f002c1eb90a9f4c5a62f4d398154eed4d2')
 
     variant('cuda', default=False, description='Enable CUDA support')
diff --git a/var/spack/repos/builtin/packages/py-horovod/package.py b/var/spack/repos/builtin/packages/py-horovod/package.py
--- a/var/spack/repos/builtin/packages/py-horovod/package.py
+++ b/var/spack/repos/builtin/packages/py-horovod/package.py
@@ -3,10 +3,6 @@
 #
 # SPDX-License-Identifier: (Apache-2.0 OR MIT)
 
-from spack import *
-import fnmatch
-import os
-
 
 class PyHorovod(PythonPackage):
     """Horovod is a distributed deep learning training framework for
@@ -14,47 +10,111 @@ class PyHorovod(PythonPackage):
 
     homepage = "https://github.com/horovod"
     git      = "https://github.com/horovod/horovod.git"
-    url      = "https://github.com/horovod/horovod/archive/v0.19.0.tar.gz"
-
-    version('master', branch='master', submodules='True')
-    version('0.19.0',       sha256='0e9fec11cd7f5f39a09f0785d1097cb51c44537ae14c9b4b2578b5cdd21efb9b')
-    version('0.18.2',       sha256='a073e08cec65474afdb2d011486b4cb6c7ac8fcb1eca3e02b169e1e7b4a66da6')
-    version('0.18.1',       sha256='26e236d1f60955e9dd12b9f0a836f0691296a010fcd1ac72295970a780f4e4fb')
-    version('0.18.0',       sha256='94f13e7110c5f3fd1aa194b9d886b5bb91c9bc02ade31bcb84fc6e7f9c043455')
-    version('0.17.1',       sha256='14eea5744eda9c62988ffa278a9a5472cebbc6a287eca9ed48cacfcd177e8978')
-    version('0.17.0.post1', sha256='220b230611e22dc69777f1be4d9788a07e73a0722e511091fa156cdf68ca798b')
-    version('0.17.0',       sha256='4bb121dda6cdaa1677535470adc1836493a9c4930ab19f6b491254ea47a12a4f')
-    version('0.16.4',       sha256='c0168dfeb31a56ede52eae115f43fa2d06a5db55a37201064ef901c8000d708d')
-    version('0.16.3',       sha256='1857cf1b335723366cc71e4bcd0583f2dde0c821212cda0e1b6bddfe4ba1ea0d')
-    version('0.16.2',       sha256='baa9754e59ab0ee72d3b5769cf77e06a2c7b0a2d9626e0e14ca2ab131934ce74')
-
-    variant('pytorch', default=True, description='Enables PyTorch')
-    variant('mxnet', default=False, description='Enables mxnet')
-    variant('mpi', default=True, description='Enables MPI build')
+
+    maintainers = ['adamjstewart']
+
+    version('master', branch='master', submodules=True)
+    version('0.19.1', tag='v0.19.1', submodules=True)
+    version('0.19.0', tag='v0.19.0', submodules=True)
+    version('0.18.2', tag='v0.18.2', submodules=True)
+    version('0.18.1', tag='v0.18.1', submodules=True)
+    version('0.18.0', tag='v0.18.0', submodules=True)
+    version('0.17.1', tag='v0.17.1', submodules=True)
+    version('0.17.0', tag='v0.17.0', submodules=True)
+    version('0.16.4', tag='v0.16.4', submodules=True)
+    version('0.16.3', tag='v0.16.3', submodules=True)
+    version('0.16.2', tag='v0.16.2', submodules=True)
+
+    # Deep learning frameworks
+    variant('pytorch',    default=True,  description='Enables PyTorch')
+    variant('tensorflow', default=False, description='Enables TensorFlow')
+    variant('mxnet',      default=False, description='Enables Apache MXNet')
+
+    # Distributed support
     variant('gloo', default=False, description='Enables features related to distributed support')
+    variant('mpi',  default=True,  description='Enables MPI build')
+
+    # GPU support
     variant('cuda', default=True, description='Enables CUDA build')
+    variant('gpu_allreduce', default='mpi',
+            description='Backend to use for GPU_ALLREDUCE',
+            values=('mpi', 'nccl'), multi=False)  # DDL support is deprecated
+    variant('gpu_allgather', default='mpi',
+            description='Backend to use for GPU_ALLGATHER',
+            values=('mpi',), multi=False)
+    variant('gpu_broadcast', default='mpi',
+            description='Backend to use for GPU_BROADCAST',
+            values=('mpi', 'nccl'), multi=False)
 
-    depends_on('python', type=('build', 'run'))
+    # Required dependencies
     depends_on('py-setuptools', type='build')
     depends_on('py-cloudpickle', type=('build', 'run'))
     depends_on('py-psutil', type=('build', 'run'))
     depends_on('py-pyyaml', type=('build', 'run'))
     depends_on('py-six', type=('build', 'run'))
-    depends_on('py-torch', type=('build', 'run'))
-    depends_on('py-pip', type=('build'))
 
-    # Optional dependencies
-    depends_on('cuda', when='+cuda')
-    depends_on('nccl', when='+nccl')
-    depends_on('gloo', when='+gloo')
+    # Deep learning frameworks
+    depends_on('py-torch@0.4.0:', type=('build', 'run'), when='+pytorch')
+    depends_on('py-torch+cuda', type=('build', 'run'), when='+pytorch+cuda')
+    depends_on('py-cffi@1.4.0:', type=('build', 'run'), when='+pytorch')
+    depends_on('py-tensorflow@1.1.0:', type=('build', 'link', 'run'), when='+tensorflow')
+    depends_on('mxnet@1.4.0:+python', type=('build', 'link', 'run'), when='+mxnet')
+    depends_on('mxnet+cuda', type=('build', 'link', 'run'), when='+mxnet+cuda')
+
+    # Distributed support
+    # There does not appear to be a way to use an external Gloo installation
+    depends_on('cmake', type='build', when='+gloo')
     depends_on('mpi', when='+mpi')
-    depends_on('py-torch', type=('build', 'run'), when='+pytorch')
-    depends_on('mxnet', when='+mxnet')
+    depends_on('mpi', when='gpu_allreduce=mpi')
+    depends_on('mpi', when='gpu_allgather=mpi')
+    depends_on('mpi', when='gpu_broadcast=mpi')
+
+    # GPU support
+    depends_on('cuda', when='+cuda')
+    depends_on('nccl@2.0:', when='gpu_allreduce=nccl')
+    depends_on('nccl@2.0:', when='gpu_broadcast=nccl')
+
+    # Test dependencies
+    depends_on('py-mock', type='test')
+    depends_on('py-pytest', type='test')
+    depends_on('py-pytest-forked', type='test')
+
+    conflicts('+gloo', when='platform=darwin', msg='Gloo cannot be compiled on MacOS')
+    conflicts('~gloo~mpi', msg='One of Gloo or MPI are required for Horovod to run')
+    conflicts('~pytorch~tensorflow~mxnet', msg='At least one deep learning backend is required')
+
+    def setup_build_environment(self, env):
+        # Deep learning frameworks
+        if '~pytorch' in self.spec:
+            env.set('HOROVOD_WITHOUT_PYTORCH', 1)
+        if '~tensorflow' in self.spec:
+            env.set('HOROVOD_WITHOUT_TENSORFLOW', 1)
+        if '~mxnet' in self.spec:
+            env.set('HOROVOD_WITHOUT_MXNET', 1)
 
-    phases = ['clean', 'sdist', 'install']
+        # Distributed support
+        if '~gloo' in self.spec:
+            env.set('HOROVOD_WITHOUT_GLOO', 1)
+        if '+mpi' in self.spec:
+            env.set('HOROVOD_WITH_MPI', 1)
+        else:
+            env.set('HOROVOD_WITHOUT_MPI', 1)
 
-    def install(self, spec, prefix):
-        pip = which('pip')
-        for file in os.listdir(prefix):
-            if fnmatch.fnmatch(file, 'horovod-*.tar.gz'):
-                pip('install', file, '--prefix={0}'.format(prefix))
+        # GPU support
+        if '+cuda' in self.spec:
+            env.set('HOROVOD_CUDA_HOME', self.spec['cuda'].prefix)
+            env.set('HOROVOD_CUDA_INCLUDE',
+                    self.spec['cuda'].headers.directories[0])
+            env.set('HOROVOD_CUDA_LIB', self.spec['cuda'].libs.directories[0])
+        if '^nccl' in self.spec:
+            env.set('HOROVOD_NCCL_HOME', self.spec['nccl'].prefix)
+            env.set('HOROVOD_NCCL_INCLUDE',
+                    self.spec['nccl'].headers.directories[0])
+            env.set('HOROVOD_NCCL_LIB', self.spec['nccl'].libs.directories[0])
+        env.set('HOROVOD_GPU_ALLREDUCE',
+                self.spec.variants['gpu_allreduce'].value.upper())
+        env.set('HOROVOD_GPU_ALLGATHER',
+                self.spec.variants['gpu_allgather'].value.upper())
+        env.set('HOROVOD_GPU_BROADCAST',
+                self.spec.variants['gpu_broadcast'].value.upper())
+        env.set('HOROVOD_ALLOW_MIXED_GPU_IMPL', 1)
diff --git a/var/spack/repos/builtin/packages/py-torch/package.py b/var/spack/repos/builtin/packages/py-torch/package.py
--- a/var/spack/repos/builtin/packages/py-torch/package.py
+++ b/var/spack/repos/builtin/packages/py-torch/package.py
@@ -105,22 +105,22 @@ class PyTorch(PythonPackage, CudaPackage):
     cuda_arch_conflict = ('This version of Torch/Caffe2 only supports compute '
                           'capabilities ')
 
-    conflicts('cuda_arch=none', when='+cuda+caffe2',
+    conflicts('cuda_arch=none', when='+cuda',
               msg='Must specify CUDA compute capabilities of your GPU, see '
               'https://developer.nvidia.com/cuda-gpus')
-    conflicts('cuda_arch=52', when='@1.3.0:+cuda+caffe2',
+    conflicts('cuda_arch=52', when='@1.3.0:+cuda',
               msg=cuda_arch_conflict + '>=5.3')
-    conflicts('cuda_arch=50', when='@1.3.0:+cuda+caffe2',
+    conflicts('cuda_arch=50', when='@1.3.0:+cuda',
               msg=cuda_arch_conflict + '>=5.3')
-    conflicts('cuda_arch=35', when='@1.3.0:+cuda+caffe2',
+    conflicts('cuda_arch=35', when='@1.3.0:+cuda',
               msg=cuda_arch_conflict + '>=5.3')
-    conflicts('cuda_arch=32', when='@1.3.0:+cuda+caffe2',
+    conflicts('cuda_arch=32', when='@1.3.0:+cuda',
               msg=cuda_arch_conflict + '>=5.3')
-    conflicts('cuda_arch=30', when='@1.3.0:+cuda+caffe2',
+    conflicts('cuda_arch=30', when='@1.3.0:+cuda',
               msg=cuda_arch_conflict + '>=5.3')
-    conflicts('cuda_arch=30', when='@1.2.0:+cuda+caffe2',
+    conflicts('cuda_arch=30', when='@1.2.0:+cuda',
               msg=cuda_arch_conflict + '>=3.2')
-    conflicts('cuda_arch=20', when='@1.0.0:+cuda+caffe2',
+    conflicts('cuda_arch=20', when='@1.0.0:+cuda',
               msg=cuda_arch_conflict + '>=3.0')
 
     # Required dependencies


Test patch: