Repository: pytorch
Issue: Pybind11 cpp extensions broken with pytorch v1.5.0
Labels: high priority,module: build,module: docs,module: cpp,triaged,module: regression

## 🐛 Bug

Dear community,

I am a dev from [KeOps project](https://github.com/getkeops/keops/) building optimized operations written in C++/Cuda that are compatible with pytorch (among others scientific languages). 

We successfully build python modules compatible with pyTorch since the pyTorch v0.2... But the last v1.5 broke our modules with a runtime error complaining about [missing symbols](https://github.com/getkeops/keops/issues/59).


To simplify the analysis you may find below a minimal working example that builds a module through pybind11n the spirit of [pytorch doc](https://pytorch.org/tutorials/advanced/cpp_frontend.html#writing-a-basic-application). 

It should output a file `test_module.cpython-38-x86_64-linux-gnu.so` that can be imported from python. This module works well when building with pytorch 1.4 but raised a runtime error when building with pytorch v1.5. 


## To Reproduce

Assuming pyTorch and pybind11 installed (e.g. through conda).  There are 2 files

`module_test.cpp` contains

```cpp
#include <torch/extension.h>
#include <pybind11/pybind11.h>
// Main function
at::Tensor foo(int s) {
     return torch::eye(s);
}    

// PyBind11 entry point 
PYBIND11_MODULE(test_module, m) {
m.def("foo", &foo, "Entry point to test module");
}
```   
and `CMakeList.txt` contains
```
project(test_module LANGUAGES CXX)

find_package(Torch REQUIRED)
find_package(pybind11  REQUIRED)

pybind11_add_module(test_module ${CMAKE_CURRENT_SOURCE_DIR}/test_module.cpp)
target_link_libraries(test_module PUBLIC "${TORCH_LIBRARIES}")
```
it can be compile with

 ```bash
$ mkdir build
$ cd build
$ cmake -DCMAKE_PREFIX_PATH="/home/bcharlier/.conda/envs/keops/lib/python3.8/site-packages/torch/" .. && make
```
and run with (note that `torch` is imported first)
```
$ python -c "import torch; print(torch.__version__); import test_module; print(test_module.foo(3))"

1.5.0
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: /home/bcharlier/src/test_module/build/test_module.cpython-38-x86_64-linux-gnu.so: undefined symbol: _Z16THPVariable_WrapN2at6TensorE
```
The missing symbol is not exactly the same when [building a module with the keops library](https://github.com/getkeops/keops/issues/59)... But I guess this is irrelevant.

## Expected behavior

It works fine when compiling with pytorch v1.4

```bash
$ cmake -DCMAKE_PREFIX_PATH="/home/bcharlier/.conda/envs/keops_torch14/lib/python3.8/site-packages/torch/" .. && make
$ python -c "import torch; print(torch.__version__); import test_module; print(test_module.foo(3))"
1.4.0
tensor([[1., 0., 0.],
        [0., 1., 0.],
        [0., 0., 1.]])
```

## Environment

The bug was reported by various users running on linux. Here is my config:

```
Collecting environment information...
PyTorch version: 1.5.0
Is debug build: No
CUDA used to build PyTorch: 10.2

OS: Arch Linux
GCC version: (Arch Linux 9.3.0-1) 9.3.0
CMake version: version 3.14.0

Python version: 3.8
Is CUDA available: Yes
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: Quadro T2000
Nvidia driver version: 440.82
cuDNN version: /usr/lib/libcudnn.so.7.6.5

Versions of relevant libraries:
[pip3] numpy==1.18.4
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.2.89              hfd86e86_1  
[conda] mkl                       2020.0                      166  
[conda] mkl-service               2.3.0            py38he904b0f_0  
[conda] mkl_fft                   1.0.15           py38ha843d7b_0  
[conda] mkl_random                1.1.0            py38h962f231_0  
[conda] numpy                     1.18.1           py38h4f9e942_0  
[conda] numpy-base                1.18.1           py38hde5b4d6_1  
[conda] pytorch                   1.5.0           py3.8_cuda10.2.89_cudnn7.6.5_0    pytorch

```

cc @ezyang @gchanan @zou3519 @malfet @yf225 @glaringlee