Repository: tensorflow
Issue: Failed to compile on aarch64
Labels: stat:awaiting tensorflower,type:bug,type:build/install,TF 2.16

### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

Yes

### Source

source

### TensorFlow version

2.16.1

### Custom code

No

### OS platform and distribution

aarch64

### Mobile device

_No response_

### Python version

3.10

### Bazel version

bazelisk using the default version requested by tensorflow

### GCC/compiler version

clang 17 (I also tried with clang16).

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current behavior?

When compiling the tensorflow source code on aarch64, I always get the following error:

```
WARN ERROR: /home/build/tensorflow/BUILD:1263:20: Linking tensorflow/libtensorflow.so.2.16.1 failed: (Exit 1): clang-17 failed: error executing command (from target //tensorflow:libtensorflow.so.2.16.1) /usr/bin/clang-17 @bazel-out/aarch64-opt/bin/tensorflow/libtensorflow.so.2.16.1-2.params
```

However the compilation works on x86_64.

``` 
export PYTHON_BIN_PATH=/usr/bin/python
      export TF_PYTHON_VERSION=3.10
      export USE_DEFAULT_PYTHON_LIB_PATH=1
      export TF_NEED_JEMALLOC=1
      export TF_NEED_KAFKA=1
      export TF_NEED_OPENCL_SYCL=0
      export TF_NEED_AWS=1
      export TF_NEED_GCP=1
      export TF_NEED_HDFS=1
      export TF_NEED_S3=1
      export TF_ENABLE_XLA=1
      export TF_NEED_GDR=0
      export TF_NEED_VERBS=0
      export TF_NEED_OPENCL=0
      export TF_NEED_MPI=0
      export TF_NEED_TENSORRT=0
      export TF_NEED_NGRAPH=0
      export TF_NEED_IGNITE=0
      export TF_NEED_ROCM=0
      export TF_SYSTEM_LIBS="boringssl,curl,gif,icu,libjpeg_turbo,nasm,png,zlib"
      export TF_SET_ANDROID_WORKSPACE=0

      ./configure

      bazel --bazelrc=.tf_configure.bazelrc build \
        --config=opt \
        --config=mkl_threadpool \
        //tensorflow:libtensorflow.so \
        //tensorflow:libtensorflow_cc.so \
        //tensorflow:install_headers \
        //tensorflow/tools/pip_package:build_pip_package
```

### Standalone code to reproduce the issue

```shell
export PYTHON_BIN_PATH=/usr/bin/python
      export TF_PYTHON_VERSION=3.10
      export USE_DEFAULT_PYTHON_LIB_PATH=1
      export TF_NEED_JEMALLOC=1
      export TF_NEED_KAFKA=1
      export TF_NEED_OPENCL_SYCL=0
      export TF_NEED_AWS=1
      export TF_NEED_GCP=1
      export TF_NEED_HDFS=1
      export TF_NEED_S3=1
      export TF_ENABLE_XLA=1
      export TF_NEED_GDR=0
      export TF_NEED_VERBS=0
      export TF_NEED_OPENCL=0
      export TF_NEED_MPI=0
      export TF_NEED_TENSORRT=0
      export TF_NEED_NGRAPH=0
      export TF_NEED_IGNITE=0
      export TF_NEED_ROCM=0
      export TF_SYSTEM_LIBS="boringssl,curl,gif,icu,libjpeg_turbo,nasm,png,zlib"
      export TF_SET_ANDROID_WORKSPACE=0

      ./configure

      bazel --bazelrc=.tf_configure.bazelrc build \
        --config=opt \
        --config=mkl_threadpool \
        //tensorflow:libtensorflow.so \
        //tensorflow:libtensorflow_cc.so \
        //tensorflow:install_headers \
        //tensorflow/tools/pip_package:build_pip_package
```


### Relevant log output

```shell
2024/05/09 13:03:51 WARN [14,447 / 14,578] Compiling tensorflow/compiler/jit/xla_platform_info.cc; 20s local ... (16 actions running)
2024/05/09 13:03:52 WARN [14,448 / 14,578] Compiling tensorflow/compiler/jit/xla_platform_info.cc; 21s local ... (16 actions running)
2024/05/09 13:03:55 WARN [14,449 / 14,578] Compiling tensorflow/compiler/jit/xla_platform_info.cc; 24s local ... (16 actions, 15 running)
2024/05/09 13:03:57 WARN [14,450 / 14,578] Compiling tensorflow/compiler/jit/xla_platform_info.cc; 26s local ... (16 actions running)
2024/05/09 13:03:58 WARN [14,452 / 14,578] Compiling tensorflow/compiler/jit/get_compiler_ir.cc; 26s local ... (16 actions running)
2024/05/09 13:04:00 WARN [14,454 / 14,578] Compiling tensorflow/compiler/jit/kernels/xla_ops.cc; 27s local ... (16 actions running)
2024/05/09 13:04:01 WARN [14,457 / 14,578] Compiling tensorflow/compiler/jit/xla_cpu_device.cc; 22s local ... (16 actions running)
2024/05/09 13:04:03 WARN [14,459 / 14,578] Compiling tensorflow/compiler/jit/xla_cpu_device.cc; 24s local ... (16 actions running)
2024/05/09 13:04:04 WARN [14,460 / 14,578] Compiling tensorflow/compiler/jit/xla_cpu_device.cc; 25s local ... (16 actions, 15 running)
2024/05/09 13:04:06 WARN [14,461 / 14,578] Compiling tensorflow/compiler/jit/xla_cpu_device.cc; 27s local ... (16 actions running)
2024/05/09 13:04:07 WARN [14,464 / 14,578] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 22s local ... (16 actions running)
2024/05/09 13:04:11 WARN [14,465 / 14,578] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 26s local ... (16 actions, 15 running)
2024/05/09 13:04:12 WARN [14,467 / 14,578] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 27s local ... (16 actions, 15 running)
2024/05/09 13:04:13 WARN [14,468 / 14,578] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 28s local ... (16 actions running)
2024/05/09 13:04:14 WARN [14,470 / 14,578] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 29s local ... (16 actions, 15 running)
2024/05/09 13:04:16 WARN [14,473 / 14,578] Compiling tensorflow/compiler/aot/codegen.cc; 19s local ... (16 actions, 15 running)
2024/05/09 13:04:17 WARN [14,476 / 14,578] Compiling tensorflow/core/common_runtime/pluggable_device/pluggable_device_plugin_init.cc; 16s local ... (16 actions running)
2024/05/09 13:04:18 WARN [14,478 / 14,578] Compiling tensorflow/core/common_runtime/pluggable_device/pluggable_device_plugin_init.cc; 17s local ... (16 actions, 15 running)
2024/05/09 13:04:20 WARN [14,479 / 14,578] Compiling tensorflow/core/common_runtime/pluggable_device/pluggable_device_plugin_init.cc; 19s local ... (16 actions, 15 running)
2024/05/09 13:04:21 WARN [14,481 / 14,578] Compiling tensorflow/core/common_runtime/pluggable_device/pluggable_device_plugin_init.cc; 20s local ... (16 actions, 14 running)
2024/05/09 13:04:22 WARN [14,482 / 14,578] Compiling tensorflow/core/common_runtime/pluggable_device/pluggable_device_plugin_init.cc; 21s local ... (16 actions, 15 running)
2024/05/09 13:04:23 WARN [14,486 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 21s local ... (16 actions running)
2024/05/09 13:04:25 WARN [14,487 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 23s local ... (16 actions running)
2024/05/09 13:04:26 WARN [14,489 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 24s local ... (16 actions running)
2024/05/09 13:04:28 WARN [14,491 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 25s local ... (16 actions, 15 running)
2024/05/09 13:04:29 WARN [14,492 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 27s local ... (16 actions, 15 running)
2024/05/09 13:04:30 WARN [14,494 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 28s local ... (16 actions, 15 running)
2024/05/09 13:04:31 WARN [14,497 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 29s local ... (14 actions running)
2024/05/09 13:04:32 WARN [14,498 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 30s local ... (13 actions running)
2024/05/09 13:04:34 WARN [14,500 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 32s local ... (11 actions running)
2024/05/09 13:04:36 WARN [14,505 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 34s local ... (6 actions running)
2024/05/09 13:04:38 WARN [14,506 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 36s local ... (5 actions running)
2024/05/09 13:04:40 WARN [14,508 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 38s local ... (3 actions running)
2024/05/09 13:04:40 WARN ERROR: /home/build/tensorflow/BUILD:1263:20: Linking tensorflow/libtensorflow.so.2.16.1 failed: (Exit 1): clang-17 failed: error executing command (from target //tensorflow:libtensorflow.so.2.16.1) /usr/bin/clang-17 @bazel-out/aarch64-opt/bin/tensorflow/libtensorflow.so.2.16.1-2.params
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN clang-17: error: linker command failed with exit code 1 (use -v to see invocation)
2024/05/09 13:04:41 WARN INFO: Elapsed time: 4130.234s, Critical Path: 216.57s
2024/05/09 13:04:41 WARN INFO: 14511 processes: 1423 internal, 13088 local.
2024/05/09 13:04:41 WARN FAILED: Build did NOT complete successfully
```
```