Repository: spack/spack

Issue:
gcc or binutils seems to have been broken recently
My daily build two nights ago worked.  A fresh clone of github.com/llnl/spack late yesterday afternoon failed to build, as did my build last night.

I'm building gcc@5.4.0 with the CentOS system compiler, gcc@4.8.[35].

The compiler appears to be failing.

Here's a bit from a `config.log` file.  The interesting bit is `n--eh-frame-hdr`, it looks as if there might be a stray 'n' somewhere that's getting pasted onto the front of the `--eh-frame-hdr` switch:

```
configure:3851: /scratch/hartzelg/working-dir/workspace/daily-build/spack/lib/spack/env/gcc/gcc    conftest.c  >&5
/blah/apps/spack/v0.0.10/opt/spack/linux-centos7-x86_64/gcc-4.8.5/binutils-2.28-66ufycihw5nglnjuy4xf6hqpwrru6bx4/bin/ld: cannot find n--eh-frame-hdr: No
 such file or directory
collect2: error: ld returned 1 exit status
```

I've been fishing for the change, but perhaps it will ring a bell.



Discussion:
Is it caused by #4144?
> Is it caused by #4144?

i think the only real change is `lib64`->`lib`, otherwise I can't really see how that PR can break something... But that would be my best guess as well...
I'll dig.  The changes in the `binutils_options` look possible....
I didn't change anything in `binutils_options`. At least I don't think I did. But I've never been able to install `gcc+binutils` before, so I didn't test it.
I'm not saying that there is anything *meaningful* here, but this is the bit that caught my eye: https://github.com/LLNL/spack/pull/4144/files#diff-38bf8d6101b9bc88270ac094f38565e1R199 (since what I'm seeing feels like something's screwed up in the way that gcc's building it's command line for `ld`).
But we were already doing that. I'm not sure what changed.
> but this is the bit that caught my eye: https://github.com/LLNL/spack/pull/4144/files#diff-38bf8d6101b9bc88270ac094f38565e1R199

I don't thinks @adamjstewart changed this from the original version... At least I see no changes in the `binutils` part of the PR
[using my confused voice, not my snarky one, in case the tone doesn't come through the email?]

I think, as I stare at it, that there are no *substantive* changes here, but doesn't the red on the left hand side and the green on the right hand side mean that there were changes?

![screen shot 2017-05-09 at 9 16 03 am](https://cloud.githubusercontent.com/assets/312978/25861007/499ace38-3498-11e7-9b97-3a77c76cb14c.jpg)

My working hypothesis is that there's a stray `vi` or `emacs` keystroke sitting in a file somewhere (an `n`, in particular`, that's ending up in the template string used to build the ld command line.
Nope, at least not in the binutils section. Does `gcc~binutils` build for you?
@adamjstewart -- "nope" what?  The red and green bits in that diff from the PR don't represent changes?  (again, reading through them, they seem like simple transformations of Python string formatting and a bit of `prefix` to `prefix.bin` jiggering), but it's not like I'm looking at the wrong PR or something, is it?

By "there are no changes" I mean "there are no changes that matter". As in `'foo' == "foo"` and `prefix + '/lib' == prefix.lib`.
I have a build running for `gcc` from the commit before the gcc 7 PR.  I've never ever needed to build `~binutils` (unless I've been doing it by default or something) and do have `binutils` in my spack installed packages list....

I can try that next if it'd be more useful that bisecting to find the problematic commit.
If pre-GCC 7 PR works, then we should definitely figure out what changed. The lib -> lib64 suspicion seems like a good one to test. If pre-GCC 7 works, then I would try changing that back next. You'll need to re-add [this line](https://github.com/LLNL/spack/pull/4144/files#diff-38bf8d6101b9bc88270ac094f38565e1L119) and switch [this line](https://github.com/LLNL/spack/pull/4144/files#diff-38bf8d6101b9bc88270ac094f38565e1R234) from prefix.lib to prefix.lib64.
> doesn't the red on the left hand side and the green on the right hand side mean that there were changes?

there were changes in the code (thus red/green), but the result should be **exactly the same as before**.
I need to id the file (found this by running a find across the gcc tree and then strings and then grep), but since I'm looking for an 'n' glued onto the front of a `--eh-frame-hdr`, I think that this is my culprit....

![screen shot 2017-05-09 at 9 33 39 am](https://cloud.githubusercontent.com/assets/312978/25861793/9f191c78-349a-11e7-8ccf-06296a0e41e3.jpg)


Here's the ascii version:

```
-rpath /scratch/hartzelg/spack-cime-sc1-working-dir/workspace/sc1-daily-build/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-5.4.0-mqlhbvmx6lar5z6rmmf5hrqme2ogbu4v/lib:/scratch/hartzelg/spack-cime-sc1-working-dir/workspace/sc1-daily-build/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-5.4.0-mqlhbvmx6lar5z6rmmf5hrqme2ogbu4v/lib64 \n%{!static:--eh-frame-hdr} %{!mandroid|tno-android-ld:%{m16|m32|mx32:;:-m elf_x86_64}
```
LIne 253?

https://github.com/adamjstewart/spack/blob/83f324c7cf9b7d91120446462fd7311200ed4d20/var/spack/repos/builtin/packages/gcc/package.py#L253

@hartzell That HAS to be it. I wasn't sure how to convert `\\\n` to its raw equivalent. I must have messed up. Let's just revert that line. I'll send out a PR for you to test.
I have a build just starting up with this:

```diff
--- a/var/spack/repos/builtin/packages/gcc/package.py
+++ b/var/spack/repos/builtin/packages/gcc/package.py
@@ -250,6 +250,7 @@ class Gcc(AutotoolsPackage):
             for line in lines:
                 out.write(line + '\n')
                 if line.startswith('*link:'):
-                    out.write(r'-rpath {0}:{1} \n'.format(
+                    out.write(r'-rpath {0}:{1}'.format(
                               self.prefix.lib, self.prefix.lib64))
+                    out.write('\n')
         set_install_permissions(specs_file)
```

Is there a prettier change I might make?
I don't think that would make a difference. Like, there is a change, but the result should be exactly the same as before 😉 
That's me being `--python`, thought perhaps it was a bad interaction with the `r'...'` and the `\n`.

Would it be `"\n"` (in Perl, that'd get it transmogrified....)?
I actually have no idea what it would be. But I assume the way it was before will work.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/gcc/package.py b/var/spack/repos/builtin/packages/gcc/package.py
--- a/var/spack/repos/builtin/packages/gcc/package.py
+++ b/var/spack/repos/builtin/packages/gcc/package.py
@@ -250,6 +250,6 @@ def write_rpath_specs(self):
             for line in lines:
                 out.write(line + '\n')
                 if line.startswith('*link:'):
-                    out.write(r'-rpath {0}:{1} \n'.format(
+                    out.write('-rpath {0}:{1} '.format(
                               self.prefix.lib, self.prefix.lib64))
         set_install_permissions(specs_file)


Test patch: