Undefined symbol error when compiling and loading C++ extension
Labels: module: cpp-extensions, module: cpp, triaged

### 🐛 Describe the bug

We are trying to use a custom C++ extension in our project, and it successfully compiles and runs on one Linux environment but fails with an undefined symbol error in another environment. We are JIT-compiling this extension using the `cpp_extension.load` API. For this particular extension there are no CUDA source files, only a single C++ source file. Here is the relevant part of the traceback in the failing environment:
```
Using /home/jovyan/.cache/torch_extensions as PyTorch extensions root...
Emitting ninja build file /home/jovyan/.cache/torch_extensions/segmented_lookup_cpp/build.ninja...
Building extension module segmented_lookup_cpp...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] c++ segmented_lookup.o -shared -ltorch_cpu -L/home/jovyan/ColBERT-4-fast_search/env-gpu/lib/python3.7/site-packages/torch/lib -lc10 -ltorch_cpu -ltorch -ltorch_python -o segmented_lookup_cpp.so
Loading extension module segmented_lookup_cpp...
Traceback (most recent call last):
  ...
  File "/home/jovyan/ColBERT-4-fast_search/env-gpu/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1092, in load
    keep_intermediates=keep_intermediates)
  File "/home/jovyan/ColBERT-4-fast_search/env-gpu/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1318, in _jit_compile
    return _import_module_from_library(name, build_directory, is_python_module)
  File "/home/jovyan/ColBERT-4-fast_search/env-gpu/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1701, in _import_module_from_library
    module = importlib.util.module_from_spec(spec)
ImportError: /home/jovyan/.cache/torch_extensions/segmented_lookup_cpp/segmented_lookup_cpp.so: undefined symbol: _ZNK2at6Tensor5dtypeEv
```

Things we have tried include explicitly specifying `extra_ldflags=["-ltorch_cpu"]` (as suggested by https://github.com/pytorch/pytorch/issues/60341) and clearing the `torch_extension` cache (as suggested by https://github.com/pytorch/pytorch/issues/68905), but neither of these has worked.

We suspect this issue is due to some compiler version mismatch (note that we use GCC 5.4 in the successful environment and GCC 9.3 in the failing environment) but we're not sure if there is a way to verify this without downgrading the GCC version. Any suggestions for debugging this further?

### Versions

This is the environment where the extension successfully runs:
```
PyTorch version: 1.9.0
Is debug build: False
CUDA used to build PyTorch: 11.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 16.04.6 LTS (x86_64)
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
Clang version: Could not collect
CMake version: version 3.5.1

Python version: 3.7 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 11.0.194
GPU models and configuration:
GPU 0: TITAN V
GPU 1: TITAN V
GPU 2: TITAN V
GPU 3: TITAN V

Nvidia driver version: 460.39
cuDNN version: Probably one of the following:
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn.so.8.0.3
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.0.3
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.0.3
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.0.3
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.0.3
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.0.3
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.0.3
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.5
[pip3] torch==1.9.0
[pip3] torchaudio==0.9.0a0+33b2469
[pip3] torchvision==0.10.0
[conda] blas                      2.113                       mkl    conda-forge
[conda] blas-devel                3.9.0            13_linux64_mkl    conda-forge
[conda] cudatoolkit               11.1.74              h6bb024c_0    nvidia
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] libblas                   3.9.0            13_linux64_mkl    conda-forge
[conda] libcblas                  3.9.0            13_linux64_mkl    conda-forge
[conda] liblapack                 3.9.0            13_linux64_mkl    conda-forge
[conda] liblapacke                3.9.0            13_linux64_mkl    conda-forge
[conda] mkl                       2022.0.1           h8d4b97c_803    conda-forge
[conda] mkl-devel                 2022.0.1           ha770c72_804    conda-forge
[conda] mkl-include               2022.0.1           h8d4b97c_803    conda-forge
[conda] numpy                     1.21.5           py37hf2998dd_0    conda-forge
[conda] pytorch                   1.9.0           py3.7_cuda11.1_cudnn8.0.5_0    pytorch
[conda] torchaudio                0.9.0                      py37    pytorch
[conda] torchvision               0.10.0               py37_cu111    pytorch
```
And this is the environment where loading the extension results in an error:
```
PyTorch version: 1.9.0+cu111
Is debug build: False
CUDA used to build PyTorch: 11.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.2 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.10

Python version: 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:21)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-62-generic-x86_64-with-debian-bullseye-sid
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: GRID V100D-32C
GPU 1: GRID V100D-32C
GPU 2: GRID V100D-32C

Nvidia driver version: 460.32.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.9.0+cu111
[conda] cudatoolkit               11.1.74              h6bb024c_0    nvidia
[conda] numpy                     1.21.6           py37h976b520_0    conda-forge
[conda] torch                     1.9.0+cu111              pypi_0    pypi
```


cc @malfet @zou3519 @jbschlosser