Repository: spack/spack

Issue:
Installation issue: Bazel@0.29
<!-- Thanks for taking the time to report this build failure. To proceed with the report please:

1. Title the issue "Installation issue: bazel".
2. Provide the information required below.

We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively! -->

I am trying to build `bazel@0.29.1` on my way to `py-tensorflow@2.1` on `lassen` (`power9`) and running into a lot of issues. First, I ran into [this](https://github.com/bazelbuild/bazel/issues/9263) and ended up creating a patch to fix this. After that, I am running into a linking error -- i think my compiler is set up correctly since this is the only time I see this issue.

Here is the tail of the `spack-build.out` -- also attached
```
ERROR: /var/tmp/bazel_x9NXNCtZ/out/external/bazel_tools/third_party/ijar/BUILD:74:1: Linking of rule '@bazel_tools//third_party/ijar:zipper' failed (Exit 1)
[1,163 / 1,813] 15 actions running
    @com_google_protobuf//:protobuf; 4s local
    @com_google_protobuf//:protobuf; 2s local
    @com_google_protobuf//:protobuf; 2s local
    @com_google_protobuf//:protobuf; 1s local
    @com_google_protobuf//:protobuf; 1s local
    @com_google_protobuf//:protobuf; 1s local
    @com_google_protobuf//:protobuf; 1s local
    @com_google_protobuf//:protobuf; 1s local ...









bazel-out/host/bin/external/bazel_tools/src/main/cpp/util/_objs/filesystem/path_posix.o:path_posix.cc:function blaze_util::SplitPath(std::string const&): error: undefined reference to 'std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&, unsigned long, std::allocator<char> const&)'
bazel-out/host/bin/external/bazel_tools/src/main/cpp/util/_objs/filesystem/path_posix.o:path_posix.cc:function blaze_util::SplitPath(std::string const&): error: undefined reference to 'std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&, unsigned long, std::allocator<char> const&)'
collect2: error: ld returned 1 exit status
```

### Steps to reproduce the issue

<!-- Fill in the exact spec you are trying to build and the relevant part of the error message -->
```console
$ spack install -v bazel@0.29.0%gcc@7.3.1
...
```

### Information on your system

<!-- Please include the output of `spack debug report` -->
```
***Spack:** 0.16.0-2-aed2422ce
***Python:** 3.8.6
***Platform:** linux-rhel7-power9le
```

<!-- If you have any relevant configuration detail (custom `packages.yaml` or `modules.yaml`, etc.) you can add that here as well. -->
```
  jdk:
    externals:
    - spec: jdk@1.8 %gcc@7.3.1 arch=linux-rhel7-power9le
      prefix: /usr/lib/jvm/java-1.8.0-openjdk
```

### Additional information

<!-- Please upload the following files. They should be present in the stage directory of the failing build. Also upload any config.log or similar file if one exists. -->

* [spack-build-env.txt](https://github.com/spack/spack/files/5699656/spack-build-env.txt)
* [spack-build-out.txt](https://github.com/spack/spack/files/5699657/spack-build-out.txt)

<!-- Some packages have maintainers who have volunteered to debug build failures. Run `spack maintainers <name-of-the-package>` and @mention them here if they exist. -->

@adamjstewart 

### General information

<!-- These boxes can be checked by replacing [ ] with [x] or by clicking them after submitting the issue. -->
- [x] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have run `spack maintainers <name-of-the-package>` and @mentioned any maintainers
- [x] I have uploaded the build log and environment files
- [x] I have searched the issues of this repo and believe this is not a duplicate



Discussion:
Sorry, I don't have access to a power9 system to test this on. I haven't encountered this myself, but from my experience, TF and Bazel are a nightmare to build on any system that Google doesn't officially support. For example, the build should work just fine on Ubuntu but seems to fail for everyone on RHEL/CentOS. There may be some workarounds in old Spack issues or Bazel/TF issues. I would suggest bugging the developers, although you're very unlikely to get a response from them.
@adamjstewart , thanks for a prompt response. I am looking through the package for finding solutions.

One silly thing that is happening is, that during linking stage is failing. I suspect this is because the `LD_LIBARRY_PATH` is not correct. I have added the compiler to spack after loading as a module. 

```
SPACK_LD_LIBRARY_PATH=/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/py-pip-20.2-dgmjqgnd2g53zruwe3tcmpdp2mvhaa7y/lib:/usr/WS1/bhatia4/dec2020_s$
ack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/py-setuptools-50.3.2-5ijjyyxfzp3qwi62vcm7e2av3ioexjj6/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gc$
-7.3.1/python-3.8.6-wnhwfowh2haxdd44fipffnejvlvtroyo/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/sqlite-3.33.0-2j5sgjqrbslqdg4tf7pr34euqwprip$
o/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/openssl-1.1.1h-ictgh26hscftlt33b5osvgeyrtawqv5v/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/sp$
ck/linux-rhel7-power9le/gcc-7.3.1/libuuid-1.0.3-frwhvqe7lwm22hol6xpzrd5b6sv4gcmj/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/libffi-3.3-ly5lk$
kp4ligeokw37flq3cmcbx3pach/lib64:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/libffi-3.3-ly5lkdkp4ligeokw37flq3cmcbx3pach/lib:/usr/WS1/bhatia4/dec$
020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/gettext-0.21-yh4ybeyipxbvpxfrkmnociqva5nity7a/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-$
.3.1/libxml2-2.9.10-kxsi6mdg4qcof4lbqkdgsvbddusdqxix/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/zlib-1.2.11-wup2hw6o5bkocs2fg5desyw664m7xry6$
lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/xz-5.2.5-bspsrxj7bppw2j22uujf7r7ulyp7pgkv/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linu$
-rhel7-power9le/gcc-7.3.1/libiconv-1.16-4b7rg3ojdjboaxqhdg2deae6uo3h5dxf/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/gdbm-1.18.1-inn3u4zzchvj$
i3aqlyayixeghdlrawd/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/readline-8.0-h4hkrjmptpysquspse3ux4k3p3a5mdsi/lib:/usr/WS1/bhatia4/dec2020_sp$
ck/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/ncurses-6.2-dzfasfwmqkip6trvibvvpydk4os2fmb2/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/ex$
at-2.2.10-dxieubmg6sft6babgnqdpft2uoh6d6vj/lib:/usr/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/libbsd-0.10.0-6huorkjcjxm6hciph3ry3csza5c6yd2x/lib:/us$
/WS1/bhatia4/dec2020_spack/spack/opt/spack/linux-rhel7-power9le/gcc-7.3.1/bzip2-1.0.8-kredonmvkoeqw7376k3krjqepda35f75/lib:/usr/tcetmp/packages/fftw/fftw-3.3.8/lib:/usr/tce/packag$
s/cuda/cuda-10.2.89/lib64:/opt/ibm/spectrumcomputing/lsf/10.1.0.9/linux3.10-glibc2.17-ppc64le-csm/lib
```
note that there is no `lib` path for my compiler, `gcc/7.3.1`. do you know why it might not be showing up? I would probably try to explicitly set this in the package file unless there is a good solution.


And this is the linking error, fyi.
```
  >> 51421    bazel-out/host/bin/external/bazel_tools/src/main/cpp/util/_objs/filesystem/path_posix.o:path_posix.cc:function blaze_util::SplitPath(std::string
              const&): error: undefined reference to 'std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&,
              unsigned long, std::allocator<char> const&)'
  >> 51422    bazel-out/host/bin/external/bazel_tools/src/main/cpp/util/_objs/filesystem/path_posix.o:path_posix.cc:function blaze_util::SplitPath(std::string
              const&): error: undefined reference to 'std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&,
              unsigned long, std::allocator<char> const&)'
  >> 51423    collect2: error: ld returned 1 exit status
```
Hmm, I believe we have some logic inplace to automatically detect RPATHs for compilers. @becker33?

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/bazel/package.py b/var/spack/repos/builtin/packages/bazel/package.py
--- a/var/spack/repos/builtin/packages/bazel/package.py
+++ b/var/spack/repos/builtin/packages/bazel/package.py
@@ -108,6 +108,11 @@ class Bazel(Package):
     depends_on('python', type=('build', 'run'))
     depends_on('zip', when='platform=linux', type=('build', 'run'))
 
+    # make work on power9 (2x commits)
+    # https://github.com/bazelbuild/bazel/commit/5cff4f1edf8b95bf0612791632255852332f72b5
+    # https://github.com/bazelbuild/bazel/commit/ab62a6e097590dac5ec946ad7a796ea0e8593ae0
+    patch('linux_ppc-0.29.1.patch', when='@0.29.1')
+
     # Pass Spack environment variables to the build
     patch('bazelruleclassprovider-0.25.patch', when='@0.25:')
     patch('bazelruleclassprovider-0.14.patch', when='@0.14:0.24')
@@ -163,6 +168,11 @@ def url_for_version(self, version):
         return url.format(version)
 
     def setup_build_environment(self, env):
+        # fix the broken linking (on power9)
+        # https://github.com/bazelbuild/bazel/issues/10327
+        env.set('BAZEL_LINKOPTS', '')
+        env.set('BAZEL_LINKLIBS', '-lstdc++')
+
         env.set('EXTRA_BAZEL_ARGS',
                 # Spack's logs don't handle colored output well
                 '--color=no --host_javabase=@local_jdk//:jdk'


Test patch: