Repository: bazelbuild/bazel
Issue: NDK runtime libs are not part of the toolchain's all_files outputs
Labels: type: bug
P1

For `libc++`, the runtime libs are not in the outputs of the toolchain, causing linktime errors in sandboxed mode. Sample of affected files:

```
external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs
external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/x86_64
external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/x86_64/libc++.so
external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/x86_64/libc++_static.a
external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/x86_64/libc++_shared.so
external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/x86_64/libc++abi.a
external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/x86_64/libandroid_support.a
external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/x86_64/libc++.a
```

See conversation at https://github.com/bazelbuild/bazel/issues/3923#issuecomment-372750002 for more details.

Cause: the generated `%toolchain%-all-files` filegroups does not contain the `libs` folder:

Generated `external/androidndk/BUILD.bazel` snippet:
```
cc_toolchain(
    name = "aarch64-linux-android-clang5.0.300080-libcpp",
    all_files = ":aarch64-linux-android-clang5.0.300080-libcpp-all_files",
    compiler_files = ":aarch64-linux-android-clang5.0.300080-libcpp-all_files",
    cpu = "arm64-v8a",
    dwp_files = ":aarch64-linux-android-clang5.0.300080-libcpp-all_files",
    dynamic_runtime_libs = [":libcpp-arm64-v8a-dynamic-runtime-libraries"],
    linker_files = ":aarch64-linux-android-clang5.0.300080-libcpp-all_files",
    objcopy_files = ":aarch64-linux-android-clang5.0.300080-libcpp-all_files",
    static_runtime_libs = [":libcpp-arm64-v8a-static-runtime-libraries"],
    strip_files = ":aarch64-linux-android-clang5.0.300080-libcpp-all_files",
    supports_param_files = 0,
)

filegroup(
    name = "aarch64-linux-android-clang5.0.300080-libcpp-all_files",
    srcs = glob(["ndk/toolchains/llvm/**"]) + glob([
        "ndk/platforms/android-26/arch-arm64/**/*",
        "ndk/sources/android/support/include/**/*",
        "ndk/sources/cxx-stl/llvm-libc++/include/**/*",
        "ndk/sources/cxx-stl/llvm-libc++abi/include/**/*",
        "ndk/sysroot/**/*",
        "ndk/toolchains/aarch64-linux-android-4.9/prebuilt/darwin-x86_64/**/*",
    ]),
)
```

Fix: The `aarch64-linux-android-clang5.0.300080-libcpp-all_files` filegroup glob should also contain `ndk/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/*`. Same for other toolchains.

cc @steeve

Issue comments:
That's perfect.
I think the crux of the matter is at https://github.com/bazelbuild/bazel/blob/d1119fa583ff659a80828d1b67cc980b04f6b386/src/main/java/com/google/devtools/build/lib/bazel/rules/android/AndroidNdkRepositoryFunction.java#L340-L344

Which is weird because the path should be in that list as a `-L` flag ?

Or perhaps the template should be modified to add `:libcpp-arm64-v8a-dynamic-runtime-libraries`, `:libcpp-arm64-v8a-static-runtime-libraries` and so on to that list instead of globing ?
> Or perhaps the template should be modified to add :libcpp-arm64-v8a-dynamic-runtime-libraries, :libcpp-arm64-v8a-static-runtime-libraries and so on to that list instead of globing ?

Yes, I have a fix to generate this: https://gist.github.com/jin/7d9fcf924fa33ab8fbc4c43c97676ab3
@jin What is `:libcpp-arm64-v8a-static-runtime-libraries`? What is the affect of `static_runtime_libs`? Should I add libc++ `.a`s to that target, where for `dynamic_runtime_libs` I have the `.so`s?
```
$ bazel query @androidndk//:libcpp-arm64-v8a-static-runtime-libraries --output=build
# /usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/b1f050f5ec8123e101e99c9b5ef847b2/external/androidndk/BUILD.bazel:3200:1
filegroup(
  name = "libcpp-arm64-v8a-static-runtime-libraries",
  srcs = ["@androidndk//:ndk/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libandroid_support.a", "@androidndk//:ndk/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libc++.a", "@androidndk//:ndk/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libc++_static.a", "@androidndk//:ndk/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libc++abi.a"],
)

$ bazel query @androidndk//:libcpp-arm64-v8a-dynamic-runtime-libraries --output=build
# /usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/b1f050f5ec8123e101e99c9b5ef847b2/external/androidndk/BUILD.bazel:3195:1
filegroup(
  name = "libcpp-arm64-v8a-dynamic-runtime-libraries",
  srcs = ["@androidndk//:ndk/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libc++.so", "@androidndk//:ndk/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libc++_shared.so"],
)
```

These targets are meant for the internal NDK configuration to build up a CROSSTOOL so Bazel knows where the files are in the NDK. You shouldn't need to configure them. What are you intending to do here?

Pull request review comments:

Patch that resolved the issue:
diff --git a/src/main/java/com/google/devtools/build/lib/bazel/rules/android/android_ndk_cc_toolchain_template.txt b/src/main/java/com/google/devtools/build/lib/bazel/rules/android/android_ndk_cc_toolchain_template.txt
index 2019792fd335a5..7a5322b5f61957 100644
--- a/src/main/java/com/google/devtools/build/lib/bazel/rules/android/android_ndk_cc_toolchain_template.txt
+++ b/src/main/java/com/google/devtools/build/lib/bazel/rules/android/android_ndk_cc_toolchain_template.txt
@@ -22,6 +22,9 @@ filegroup(
     name = "%toolchainName%-all_files",
     srcs = glob(["ndk/toolchains/%toolchainDirectory%/**"]) + glob([
         %toolchainFileGlobs%
-    ]),
+    ]) + [
+      ":%dynamicRuntimeLibs%",
+      ":%staticRuntimeLibs%",
+    ],
 )
 


Test patch: