Repository: spack/spack

Issue:
R packages with non-R dependencies broken by recent commit
### Steps to reproduce

I was working on R package updates and noticed that all R packages with non-R dependencies are no longer able to find headers for those dependencies. I ran `git bisect` and determined that the problem was introduced with PR #30738 (@alalazo).

I can potentially fix every R package that has non-R dependencies by patching the `configure` scripts and setting `configure_args` but it seems like that should not be necessary. Also, I am not sure if this problem is limited to R packages or is more widespread.

### Error message

This is one example.

```console
$ spack install --fresh r-units
...
==> Installing r-units-0.8-0-pc2hrpgkxfkziemzlihkckrle6tet5mk
==> No binary for r-units-0.8-0-pc2hrpgkxfkziemzlihkckrle6tet5mk found: installing from source
==> Using cached archive: /opt/packages/gpjohnsn/spack/var/spack/cache/_source-cache/archive/9c/9c46fe138e8c1c3d3a51268776412f02d09673656516148cccb71b1071beb21a.tar.gz
==> No patches needed for r-units
==> r-units: Executing phase: 'install'
==> Error: ProcessError: Command exited with status 1:
    '/opt/packages/gpjohnsn/spack/opt/spack/linux-centos7-cascadelake/gcc-9.5.0/r-4.2.1-o5dgur6hxdbqitf3zvrdyo4a2qpiaiwb/bin/R' '--vanilla' 'CMD' 'INSTALL' '--library=/opt/packages/gpjohnsn/spack/opt/spack/linux-centos7-cascadelake/gcc-9.5.0/r-units-0.8-0-pc2hrpgkxfkziemzlihkckrle6tet5mk/rlib/R/library' '/tmp/gpjohnsn/spack-stage/spack-stage-r-units-0.8-0-pc2hrpgkxfkziemzlihkckrle6tet5mk/spack-src'

2 errors found in build log:
     192    checking udunits2.h presence... no
     193    checking for udunits2.h... no
     194    checking udunits2/udunits2.h usability... no
     195    checking udunits2/udunits2.h presence... no
     196    checking for udunits2/udunits2.h... no
     197    checking for ut_read_xml in -ludunits2... yes
  >> 198    configure: error: in `/tmp/gpjohnsn/spack-stage/spack-stage-r-units-0.8-0-pc2hrpgkxfkziemzlihkckrle6tet5mk/spack-src':
  >> 199    configure: error:
     200    --------------------------------------------------------------------------------
     201      Configuration failed because udunits2.h was not found. Try installing:
     202        * deb: libudunits2-dev (Debian, Ubuntu, ...)
     203        * rpm: udunits2-devel (Fedora, EPEL, ...)
     204        * brew: udunits (OSX)
     205      If udunits2 is already installed in a non-standard location, use:
---
```


### Information on your system

* **Spack:** 0.19.0.dev0
* **Python:** 3.6.8
* **Platform:** linux-centos7-cascadelake
* **Concretizer:** clingo

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:
How were we finding those dependencies before? I took a look at the current procedure and couldn't find anything that is missing at a first glance. The functions that should be called (e.g. `setup_dependent_build_environment`) are called, and code seems to be executed correctly.
@glennpj Can you post the entire output as a file, in my case I can reproduce but I also see:
```console
checking for gcc... /usr/bin/gcc
[ ... ]
```
Was that the case before?
I checked many other specs, and they all seem to use our compiler wrapper during configuration, as expected. `r-units`, instead, picks up `/usr/bin/gcc` on my laptop.
I think I got what happened, currently trying to confirm it. We have this line in `r`'s recipe:
```python
filter_compiler_wrappers("Makeconf", relative_root=os.path.join("rlib", "R", "etc"))
```
This is needed to use a Spack built R even outside of Spack - so it substitutes Spack compiler wrappers with the bare compilers after installation. We also have this other function in `r`'s recipe:
```python
@run_after("install")
def copy_makeconf(self):
    # Make a copy of Makeconf because it will be needed to properly build R
    # dependencies in Spack.
    src_makeconf = join_path(self.etcdir, "Makeconf")
    dst_makeconf = join_path(self.etcdir, "Makeconf.spack")
    install(src_makeconf, dst_makeconf)
```
I think the reworking in #30738 inverted the order of execution of the two functions (even though we never granted, as far as I can tell, that `filter_compiler_wrapper` would operate in a specific order with respect to other functions). That result in both `Makeconf` and `Makeconf.spack` being the same and pointing to the bare compilers _without_ the wrappers.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/r/package.py b/var/spack/repos/builtin/packages/r/package.py
--- a/var/spack/repos/builtin/packages/r/package.py
+++ b/var/spack/repos/builtin/packages/r/package.py
@@ -107,8 +107,6 @@ def url_for_version(self, version):
         url = "https://cloud.r-project.org/src/base"
         return url + "/R-%s/R-%s.tar.gz" % (version.up_to(1), version)
 
-    filter_compiler_wrappers("Makeconf", relative_root=os.path.join("rlib", "R", "etc"))
-
     @property
     def etcdir(self):
         return join_path(prefix, "rlib", "R", "etc")
@@ -187,6 +185,9 @@ def copy_makeconf(self):
         dst_makeconf = join_path(self.etcdir, "Makeconf.spack")
         install(src_makeconf, dst_makeconf)
 
+    # To respect order of execution, we should filter after we made the copy above
+    filter_compiler_wrappers("Makeconf", relative_root=os.path.join("rlib", "R", "etc"))
+
     # ========================================================================
     # Set up environment to make install easy for R extensions.
     # ========================================================================


Test patch: