Repository: spack/spack

Issue:
Installation issue: py-horovod missing dependency when framework is tensorflow
### Steps to reproduce the issue

It seems that py-horovod is missing dependency `py-tensorflow-estimator`. 
Currently if I build it with `frameworks=tensorflow`, the output of `horovodrun --check-build` is
```console
Horovod v0.23.0:

Available Frameworks:
    [ ] TensorFlow
    [X] PyTorch
    [ ] MXNet

Available Controllers:
    [ ] MPI
    [ ] Gloo

Available Tensor Operations:
    [ ] NCCL
    [ ] DDL
    [ ] CCL
    [ ] MPI
    [ ] Gloo 
```

Using verbose output shows the exception:
```console
ModuleNotFoundError: No module named 'tensorflow_estimator'
Extension tensorflow was NOT built.
```

Add `py-tensorflow-estimator` as dependency seems to solve the issue.

### Information on your system

* **Spack:** 0.17.1-957-bed3a69498
* **Python:** 3.8.10
* **Platform:** linux-ubuntu20.04-zen2
* **Concretizer:** clingo

### Additional information

@adamjstewart  @aweits  @tgaddair

The build succeeds, but it is not usable, giving
```console
ValueError: Neither MPI nor Gloo support has been built. Try reinstalling Horovod ensuring that either MPI is installed (MPI) or CMake is installed (Gloo).
```
since each time it runs, it will first check installation which imports `tensorflow_estimator`
[spack-build-env.txt](https://github.com/spack/spack/files/7957256/spack-build-env.txt)
[spack-build-out.txt](https://github.com/spack/spack/files/7957257/spack-build-out.txt)


### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have run `spack maintainers <name-of-the-package>` and **@mentioned** any maintainers
- [X] I have uploaded the build log and environment files
- [X] I have searched the issues of this repo and believe this is not a duplicate


Discussion:
@aweits @glennpj I think this relates to some of the hacks we use in our TF packages. Horovod itself doesn't import `tensorflow_estimator` anywhere as far as I can tell. The package only depends on TF, and TF should have a dep on `tensorflow_estimator`. I'm not sure where we should handle this.
TF can not have a dependency on TF-estimator as that would be a circular dependency. TF-estimator has been masked out of the TF package for a while. TF-estimator definitely depends on TF. The py-horovod package will need a dependency on TF-estimator. Something like:
```
depends_on('py-tensorflow-estimator',  type='run', when='frameworks=tensorflow')
```
But I don't think `py-horovod` uses `py-tensorflow-estimator`, there are no imports in the code from what I can tell. @asesidaa any idea where that error message is actually coming from?
> But I don't think `py-horovod` uses `py-tensorflow-estimator`, there are no imports in the code from what I can tell. @asesidaa any idea where that error message is actually coming from?

It seems that `tf.estimator` is used when intializing the tensorflow module in horovod.
https://github.com/horovod/horovod/blob/e5725478d00ff434ce68e2cc579611a2e4481faa/horovod/tensorflow/__init__.py#L310



> TF can not have a dependency on TF-estimator as that would be a circular dependency. TF-estimator has been masked out of the TF package for a while. TF-estimator definitely depends on TF. The py-horovod package will need a dependency on TF-estimator. Something like:
> 
> ```
> depends_on('py-tensorflow-estimator',  type='run', when='frameworks=tensorflow')
> ```
Add this dependency solves the problem.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/py-horovod/package.py b/var/spack/repos/builtin/packages/py-horovod/package.py
--- a/var/spack/repos/builtin/packages/py-horovod/package.py
+++ b/var/spack/repos/builtin/packages/py-horovod/package.py
@@ -63,6 +63,7 @@ class PyHorovod(PythonPackage, CudaPackage):
     # Framework dependencies
     depends_on('py-tensorflow@1.1.0:',  type=('build', 'link', 'run'), when='frameworks=tensorflow')
     depends_on('py-tensorflow@1.15:',   type=('build', 'link', 'run'), when='frameworks=tensorflow @0.20:')
+    depends_on('py-tensorflow-estimator', type=('build', 'run'), when='frameworks=tensorflow')
     depends_on('py-torch@0.4.0:',       type=('build', 'link', 'run'), when='frameworks=pytorch')
     depends_on('py-torch@1.2:',         type=('build', 'link', 'run'), when='frameworks=pytorch @0.20:')
     depends_on('py-torchvision',        type=('build', 'run'),         when='frameworks=pytorch @:0.19.1')


Test patch: