Repository: bazelbuild/bazel
Issue: Stop unconditionally adding -B/usr/bin flag to C/C++ toolchains
Labels: type: feature request
P2
team-Rules-CPP

The addition of this flag was done as a fix to Issue #760 

The problem with this is that it completely breaks any custom toolchain setup because it forces use of tools such as `ld`, `as`, `ar` and others from /usr/bin. And since this is added so early in the process it is not possible override it in the build files.

I see that the original problem in #760 was that `process-wrapper` and `build-runfiles` could not run because they failed to find libstdc++ which has a better solution to just link them statically as described in #4137

At the end of #760 there are other comments mentioning that hard coding -B/usr/bin is wrong.

Issue comments:
Thanks for the report. I don't plan to work on this until https://github.com/bazelbuild/bazel/issues/5187 is done. And I can see an universe in which we won't need so many env vars to tweak C++ toolchain. But nothing concrete yet. If this is urgent, let me know, and I will help the PR.
+1 to fixing this. This blocks using non-standard toolchains with bazel.
E.g. in our experiments with downloading a toolchain on-the-fly in Tensorflow, we can't currently use the LLD linker from the downloaded package, because -B flag is added by bazel and can't be overriden (so we end up picking LLD in /usr/bin/ if it's installed).

My intuition is that there should be a fix for https://github.com/bazelbuild/bazel/issues/760 that does not involve setting -B.

Pull request review comments:

Patch that resolved the issue:
diff --git a/tools/cpp/CROSSTOOL b/tools/cpp/CROSSTOOL
index a41b8822a964a4..5b9cd3d5356167 100644
--- a/tools/cpp/CROSSTOOL
+++ b/tools/cpp/CROSSTOOL
@@ -59,7 +59,6 @@ toolchain {
   tool_path { name: "gcc" path: "/usr/bin/gcc" }
   cxx_flag: "-std=c++0x"
   linker_flag: "-lstdc++"
-  linker_flag: "-B/usr/bin/"
 
   # TODO(bazel-team): In theory, the path here ought to exactly match the path
   # used by gcc. That works because bazel currently doesn't track files at
@@ -271,7 +270,6 @@ toolchain {
   tool_path { name: "gcc" path: "/usr/bin/clang" }
   cxx_flag: "-std=c++0x"
   linker_flag: "-lstdc++"
-  linker_flag: "-B/usr/bin/"
 
   # TODO(bazel-team): In theory, the path here ought to exactly match the path
   # used by gcc. That works because bazel currently doesn't track files at
diff --git a/tools/cpp/unix_cc_configure.bzl b/tools/cpp/unix_cc_configure.bzl
index 361931b47b88b1..0279b1939f76b8 100644
--- a/tools/cpp/unix_cc_configure.bzl
+++ b/tools/cpp/unix_cc_configure.bzl
@@ -282,8 +282,6 @@ def _crosstool_content(repository_ctx, cc, cpu_value, darwin):
                 "dynamic_lookup",
                 "-headerpad_max_install_names",
             ] if darwin else bin_search_flag + [
-                # Always have -B/usr/bin, see https://github.com/bazelbuild/bazel/issues/760.
-                "-B/usr/bin",
                 # Gold linker only? Can we enable this by default?
                 # "-Wl,--warn-execstack",
                 # "-Wl,--detect-odr-violations"
@@ -318,10 +316,7 @@ def _crosstool_content(repository_ctx, cc, cpu_value, darwin):
         ] + ((
             _add_compiler_option_if_supported(repository_ctx, cc, "-Wthread-safety") +
             _add_compiler_option_if_supported(repository_ctx, cc, "-Wself-assign")
-        ) if darwin else bin_search_flag + [
-            # Always have -B/usr/bin, see https://github.com/bazelbuild/bazel/issues/760.
-            "-B/usr/bin",
-        ]) + (
+        )) + (
             # Disable problematic warnings.
             _add_compiler_option_if_supported(repository_ctx, cc, "-Wunused-but-set-parameter") +
             # has false positives


Test patch:
diff --git a/src/test/java/com/google/devtools/build/lib/analysis/mock/MOCK_CROSSTOOL b/src/test/java/com/google/devtools/build/lib/analysis/mock/MOCK_CROSSTOOL
index 0ef230997778ed..3392cf5f3d153a 100644
--- a/src/test/java/com/google/devtools/build/lib/analysis/mock/MOCK_CROSSTOOL
+++ b/src/test/java/com/google/devtools/build/lib/analysis/mock/MOCK_CROSSTOOL
@@ -62,7 +62,6 @@ toolchain {
   tool_path { name: "gcc" path: "/usr/bin/gcc" }
   cxx_flag: "-std=c++0x"
   linker_flag: "-lstdc++"
-  linker_flag: "-B/usr/bin/"
 
   # TODO(bazel-team): In theory, the path here ought to exactly match the path
   # used by gcc. That works because bazel currently doesn't track files at
diff --git a/src/test/shell/bazel/testdata/bazel_toolchain_test_data/tools/arm_compiler/CROSSTOOL b/src/test/shell/bazel/testdata/bazel_toolchain_test_data/tools/arm_compiler/CROSSTOOL
index d9e45a4e01d109..55148f0daa1df5 100644
--- a/src/test/shell/bazel/testdata/bazel_toolchain_test_data/tools/arm_compiler/CROSSTOOL
+++ b/src/test/shell/bazel/testdata/bazel_toolchain_test_data/tools/arm_compiler/CROSSTOOL
@@ -155,8 +155,6 @@ toolchain {
   compiler_flag: "-fstack-protector"
   compiler_flag: "-Wall"
   compiler_flag: "-Wl,-z,-relro,-z,now"
-  compiler_flag: "-B/usr/bin"
-  compiler_flag: "-B/usr/bin"
   compiler_flag: "-Wunused-but-set-parameter"
   compiler_flag: "-Wno-free-nonheap-object"
   compiler_flag: "-fno-omit-frame-pointer"
@@ -173,8 +171,6 @@ toolchain {
   linker_flag: "-lstdc++"
   linker_flag: "-lm"
   linker_flag: "-Wl,-no-as-needed"
-  linker_flag: "-B/usr/bin"
-  linker_flag: "-B/usr/bin"
   linker_flag: "-pass-exit-codes"
   needsPic: true
   objcopy_embed_flag: "-I"