Repository: spack/spack

Issue:
Compiler Wrappers Erroneously in Path for `spack setup`; breaks things sometimes.
I am following [Developing Software with Spack](https://spack.readthedocs.io/en/latest/workflows.html#build-with-spack) section from documentation and see following error:

``` bash
$ mkdir build; cd build
$ ../spconfig.py ..
-- The C compiler identification is GNU 4.9.3
-- The CXX compiler identification is GNU 4.9.3
-- Checking whether C compiler has -isysroot
-- Checking whether C compiler has -isysroot - yes
-- Checking whether C compiler supports OSX deployment target flag
-- Checking whether C compiler supports OSX deployment target flag - yes
-- Check for working C compiler: /Users/kumbhar/workarena/software/sources/spack/opt/spack/darwin-elcapitan-x86_64/gcc-4.2.1/gcc-4.9.3-3yghgmzvmbiqmem7lpu2dmcqz6qoqcjd/bin/gcc
-- Check for working C compiler: /Users/kumbhar/workarena/software/sources/spack/opt/spack/darwin-elcapitan-x86_64/gcc-4.2.1/gcc-4.9.3-3yghgmzvmbiqmem7lpu2dmcqz6qoqcjd/bin/gcc -- broken
….
  /Users/kumbhar/workarena/software/sources/spack/opt/spack/darwin-elcapitan-x86_64/gcc-4.2.1/gcc-4.9.3-3yghgmzvmbiqmem7lpu2dmcqz6qoqcjd/bin/gcc
  -Wl,-search_paths_first -Wl,-headerpad_max_install_names
  CMakeFiles/cmTC_afeb5.dir/testCCompiler.c.o -o cmTC_afeb5

  Spack compiler must be run from Spack! Input 'SPACK_PREFIX' is
  missing.collect2: error: ld returned 1 exit status
```

In some other thread (#1111) I came across `spack env`. Using this I get all environment as:

```
spack env mylib@develop bash
```

and then it works fine. Is above the missing step from the workflow document? 

I have another question related to extending `CMakePackage` and  `spconfig.py`:

When I started writing packages for CMake based projects, I have extended packages from `Package`. Now when I do `spack setup my_spec` in the source directory of the project, `spconfig.py` is not generated. But I have `install` method defined in the `package.py` with `spack-build` as build directory. So when I invoke`spack setup`, it build and install the package. And then for development purpose I just do:

``` bash
cd speck-build
spack env spec_with_spaces - -  bash  

# change code - make - change code .....
```

which works fine! Is there any drawback of this compared to having `spconfig.py`?

@citibeth : any suggestion?



Discussion:
A bit of background... for a variety of reasons, Spack wraps your compiler in shell-script wrappers, designed to be run only from within Spack.  The `spconfig.py` script is supposed to do exactly the same build as what Spack would do if you called `spack install`, EXCEPT that it calls the compiler directly, rather than Spack's wrappers (which only work from within Spack).  On my system, that's how it's always worked for me.  I have no idea why `spconfig.py` is trying to call the compiler wrappers in your case, maybe @tgamblin has some ideas.  Can you report on the following to try to debug?
1. Share your `package.py`, `spconfig.py`, `packages.yaml` and `compilers.yaml`.
2. Do you get the same behavior building other stuff with `spack setup`, i.e. is this an issue with your particular package?  Try building the `ibmisc` package with `spack setup`, which others have had success with.
3. Are you able to try the same build on a non-Mac?

It looks like the `spack env` is a workaround that makes the Spack compiler wrappers "work" even from outside of Spack?

---

> When I started writing packages for CMake based projects, I have extended packages from Package

Yes... `spack setup` needs to get CMake arguments without running the instlall.  So the `install()` method was broken up into phases to enable this.  Hence the need to subclass from `CMakePackage`.  `spack setup` will not work if you subclass from `Package` directly.

> Is there any drawback of this compared to having spconfig.py?

Yes:
1. The behavior of `spack setup` on packages subclassed directly from `Package` is undefined.
2. You have to run a new shell with a particular environment in order to build.  That is a little more cumbersome than having the required environment in `spconfig.py`
3. Depending on the complexity of your project, running Spack can be really slow.  The `spconfig.py` approach minimizes the number of times Spack must be run.

That said... I was not aware of the possibilities you brought up, thank you for mentioning them!

Good news, I ran into this problem too.  But only on SOME computers... I don't yet know why some but not all.  Anyway... look in the generated `spconfig.py` files, I see the following:

```
env['PATH'] = ":".join(cmdlist("""
    /gpfsm/dnb53/rpfische/spack3/opt/spack/linux-SuSE11-x86_64/gcc-5.3.0/cmake-3.6.1-xfzruydrrrsee7v264k7sjlzxtsvedhg/bin
    /gpfsm/dnb53/rpfische/spack3/opt/spack/linux-SuSE11-x86_64/gcc-5.3.0/python-3.5.2-d5igtz5ukjiqtaxlfyror562fvygfiv4/bin
    /gpfsm/dnb53/rpfische/spack3/opt/spack/linux-SuSE11-x86_64/gcc-5.3.0/netcdf-cxx4-4.3.0-6sb6flocof7yu2jis2jyhp3xhc2yzg4d/bin
    /gpfsm/dnb53/rpfische/spack3/opt/spack/linux-SuSE11-x86_64/gcc-5.3.0/py-numpy-1.11.1-lhvrnmqb3f6phgnd4gs2pgqo562elian/bin
    /gpfsm/dnb53/rpfische/spack3/opt/spack/linux-SuSE11-x86_64/gcc-5.3.0/udunits2-2.2.20-rvmcxyelfy6yhao4g4xdki4tts6numbe/bin
    /gpfsm/dnb53/rpfische/spack3/opt/spack/linux-SuSE11-x86_64/gcc-5.3.0/proj-4.9.2-f654335yo4t4uz3fdd67vytqnmvo5n4p/bin
    /gpfsm/dnb53/rpfische/spack3/lib/spack/env
    /gpfsm/dnb53/rpfische/spack3/lib/spack/env/case-insensitive
    /gpfsm/dnb53/rpfische/spack3/lib/spack/env/gcc
    /gpfsm/dnb53/rpfische/spack3/opt/spack/linux-SuSE11-x86_64/gcc-5.3.0/binutils-2.27-vdugmzyoeozobcy2zppn6kjpyyorjfef/bin
    /home/rpfische/git/modele-control/bin
    /usr/local/other/SLES11.3/openmpi/1.10.1/gcc-5.3/bin
    /usr/local/other/SLES11.3/gcc/5.3.0/bin
    /usr/local/other/SLES11.3/git/2.7.4/libexec/git-core
    /usr/local/other/SLES11.3/git/2.7.4/bin
    /usr/local/other/SLES11.3/asciidoc/8.6.8/bin
    /home/rpfische/spack3/bin
    /usr/local/bin
    /usr/bin
    /bin
    /usr/bin/X11
    /usr/X11R6/bin
    /usr/games
    /opt/kde3/bin
    /opt/ibutils/bin
    /usr/lib/mit/bin
    /usr/lib/mit/sbin
    /usr/slurm/bin
    /home/rpfische/spack2/bin
"""))
```

The compiler wrappers are found in the following locations:

```
    /gpfsm/dnb53/rpfische/spack3/lib/spack/env
    /gpfsm/dnb53/rpfische/spack3/lib/spack/env/case-insensitive
    /gpfsm/dnb53/rpfische/spack3/lib/spack/env/gcc
```

These come BEFORE the real compiler, which is found at:

```
    /usr/local/other/SLES11.3/gcc/5.3.0/bin
```

@tgamblin @alalazo 
The workaround for now is to remove the three compiler wrapper lines from `spconfig.py`, then it works (for me) like a charm!

The long-term solution?  I could certainly add some code to make sure those three lines don't make it into `spconfigpy`.  But I'm wondering if this is related to commit e9d4780b on Oct 4.  I'd like to get a second opinion on what's going on here.

```
e9d4780b (Todd Gamblin       2016-10-04 09:40:28 -0700 263)     # These variables control compiler wrapper be
havior
e9d4780b (Todd Gamblin       2016-10-04 09:40:28 -0700 264)     env.set_path(SPACK_RPATH_DEPS, [d.prefix for 
d in get_rpath_deps(pkg)])
e9d4780b (Todd Gamblin       2016-10-04 09:40:28 -0700 265)     env.set_path(SPACK_LINK_DEPS, [
e9d4780b (Todd Gamblin       2016-10-04 09:40:28 -0700 266)         d.prefix for d in pkg.spec.traverse(root=
False, deptype=('link'))])
```

OK, I don't know why it's not working.  Later on in `spconfig.py` is:

```
    -DCMAKE_C_COMPILER=/usr/local/other/SLES11.3/gcc/5.3.0/bin/gcc
    -DCMAKE_CXX_COMPILER=/usr/local/other/SLES11.3/gcc/5.3.0/bin/g++
    -DCMAKE_Fortran_COMPILER=/usr/local/other/SLES11.3/gcc/5.3.0/bin/gfortran
```

On the computer where it does work, these lines seem to be causing CMake to pick up the real compiler, not the Spack wrappers.  The `spconfig.py` is essentially the same on the computer that doesn't work; but there, CMake picks up the Spack wrappers instead.  In BOTH cases, the compiler is NOT in a system location.  On the working computer, it's Spack-built; on the non-working computer it's handbuilt and requires a module to run properly.

Thanks @citibeth for detailed information! I don't have my dev environment setup at the moment, will provide my configurations soon.

Here are some additional points:
- I have packages inheriting `Package` class. When I do `spack setup`, it install the packages. But as `build` directory is created in the source, I can use it after `spack env`.
- For `autotools` based projects this works fine i.e. `spack setup` build and install the package. From `build` directory I can just do `make/make install` without `spack env`.

I think I found the problem, don't think we need additional details.

On Sun, Oct 23, 2016 at 9:48 AM, Pramod Kumbhar notifications@github.com
wrote:

> Thanks @citibeth https://github.com/citibeth for detailed information!
> I don't have my dev environment setup at the moment, will provide my
> configurations soon.
> 
> Here are some additional points:
> - I have packages inheriting Package class. When I do spack setup, it
>   install the packages. But as build directory is created in the source,
>   I can use it after spack env.
> - For autotools based projects this works fine i.e. spack setup build
>   and install the package. From build directory I can just do make/make
>   install without spack env.
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> https://github.com/LLNL/spack/issues/2055#issuecomment-255589663, or mute
> the thread
> https://github.com/notifications/unsubscribe-auth/AB1cd7rNdt7Rd-g8MYIYue6eRww4hVRjks5q22WqgaJpZM4KbW-j
> .


Resolving patch:
diff --git a/lib/spack/spack/cmd/setup.py b/lib/spack/spack/cmd/setup.py
--- a/lib/spack/spack/cmd/setup.py
+++ b/lib/spack/spack/cmd/setup.py
@@ -37,6 +37,7 @@ def setup_parser(subparser):
 
     cd_group = subparser.add_mutually_exclusive_group()
     arguments.add_common_arguments(cd_group, ['clean', 'dirty'])
+    subparser.epilog = 'DEPRECATED: use `spack dev-build` instead'
 
 
 def write_spconfig(package, dirty):
@@ -98,6 +99,8 @@ def cmdlist(str):
 
 
 def setup(self, args):
+    tty.warn('DEPRECATED: use `spack dev-build` instead')
+
     if not args.spec:
         tty.die("spack setup requires a package spec argument.")
 
diff --git a/lib/spack/spack/installer.py b/lib/spack/spack/installer.py
--- a/lib/spack/spack/installer.py
+++ b/lib/spack/spack/installer.py
@@ -466,7 +466,6 @@ def log(pkg):
     packages_dir = spack.store.layout.build_packages_path(pkg.spec)
 
     # Remove first if we're overwriting another build
-    # (can happen with spack setup)
     try:
         # log and env install paths are inside this
         shutil.rmtree(packages_dir)


Test patch: