Repository: pytorch
Issue: Segmentation fault when using C++/pybind11 module without also importing torch
Labels: module: cpp-extensions,triaged,module: pybind,module: regression

## 🐛 Bug

Related forum topic: [link](https://discuss.pytorch.org/t/pybind-pytorch-segfault/90910)

If you create a simple C++ pybind11 module and a python script that uses said module but which does not import torch, you will receive a Segmentation Fault.

## To Reproduce

Steps to reproduce the behavior:

1. Create any C++ pybind11 module that uses torch (e.g. `torch::Tensor` or `at::Tensor`), compile in the recommended way
2. Create a python script that imports the module and calls the function defined in 1. Do not include `import torch` in your python script.
3. Observe segmentation fault

Example C++ module:
```
#include <torch/extension.h>

PYBIND11_MODULE(TORCH_EXTENSION_NAME, m)
{
    m.def("zero_tensor", &zero_tensor, "Zero Tensor");
}
```

Example python script:
```
#!/usr/bin/python3
import dummy

dummy.zero_tensor(3, 4, 5)
```

Here, `dummy` is the name of the module.

The above will cause a segfault. I traced it using `valgrind` and got the following output:

```
==355897== Invalid read of size 8
==355897==    at 0x11C6A347: THPVariable_NewWithVar(_typeobject*, at::Tensor, c10::impl::PyInterpreterStatus) (in /home/baskwaadgras/opt/anaconda3/lib/python3.8/site-packages/torch/lib/libtorch_python.so)
==355897==    by 0x11C6AB0E: THPVariable_Wrap(at::Tensor) (in /home/baskwaadgras/opt/anaconda3/lib/python3.8/site-packages/torch/lib/libtorch_python.so)
==355897==    by 0x486FD85: void pybind11::cpp_function::initialize<at::Tensor (*&)(int, int, int), at::Tensor, int, int, int, pybind11::name, pybind11::scope, pybind11::sibling, char [12]>(at::Tensor (*&)(int, int, int), at::Tensor (*)(int, int, int), pybind11::name const&, pybind11::scope const&, pybind11::sibling const&, char const (&) [12])::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) (pybind.h:47)
==355897==    by 0x486D2D9: pybind11::cpp_function::dispatcher(_object*, _object*, _object*) (pybind11.h:767)
==355897==    by 0x25C347: cfunction_call_varargs (call.c:743)
==355897==    by 0x25C347: PyCFunction_Call (call.c:773)
==355897==    by 0x24BDBB: _PyObject_MakeTpCall (call.c:159)
==355897==    by 0x2D7665: _PyObject_Vectorcall (abstract.h:125)
==355897==    by 0x2D7665: call_function (ceval.c:4963)
==355897==    by 0x2D7665: _PyEval_EvalFrameDefault (ceval.c:3469)
==355897==    by 0x2A126F: PyEval_EvalFrameEx (ceval.c:741)
==355897==    by 0x2A126F: _PyEval_EvalCodeWithName (ceval.c:4298)
==355897==    by 0x336542: PyEval_EvalCodeEx (ceval.c:4327)
==355897==    by 0x336542: PyEval_EvalCode (ceval.c:718)
==355897==    by 0x3365E3: run_eval_code_obj (pythonrun.c:1165)
==355897==    by 0x35C853: run_mod (pythonrun.c:1187)
==355897==    by 0x21D38F: pyrun_file (pythonrun.c:1084)
==355897==  Address 0x130 is not stack'd, malloc'd or (recently) free'd
==355897== 
==355897== 
==355897== Process terminating with default action of signal 11 (SIGSEGV)
==355897==  Access not within mapped region at address 0x130
==355897==    at 0x11C6A347: THPVariable_NewWithVar(_typeobject*, at::Tensor, c10::impl::PyInterpreterStatus) (in /home/baskwaadgras/opt/anaconda3/lib/python3.8/site-packages/torch/lib/libtorch_python.so)
==355897==    by 0x11C6AB0E: THPVariable_Wrap(at::Tensor) (in /home/baskwaadgras/opt/anaconda3/lib/python3.8/site-packages/torch/lib/libtorch_python.so)
==355897==    by 0x486FD85: void pybind11::cpp_function::initialize<at::Tensor (*&)(int, int, int), at::Tensor, int, int, int, pybind11::name, pybind11::scope, pybind11::sibling, char [12]>(at::Tensor (*&)(int, int, int), at::Tensor (*)(int, int, int), pybind11::name const&, pybind11::scope const&, pybind11::sibling const&, char const (&) [12])::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) (pybind.h:47)
==355897==    by 0x486D2D9: pybind11::cpp_function::dispatcher(_object*, _object*, _object*) (pybind11.h:767)
==355897==    by 0x25C347: cfunction_call_varargs (call.c:743)
==355897==    by 0x25C347: PyCFunction_Call (call.c:773)
==355897==    by 0x24BDBB: _PyObject_MakeTpCall (call.c:159)
==355897==    by 0x2D7665: _PyObject_Vectorcall (abstract.h:125)
==355897==    by 0x2D7665: call_function (ceval.c:4963)
==355897==    by 0x2D7665: _PyEval_EvalFrameDefault (ceval.c:3469)
==355897==    by 0x2A126F: PyEval_EvalFrameEx (ceval.c:741)
==355897==    by 0x2A126F: _PyEval_EvalCodeWithName (ceval.c:4298)
==355897==    by 0x336542: PyEval_EvalCodeEx (ceval.c:4327)
==355897==    by 0x336542: PyEval_EvalCode (ceval.c:718)
==355897==    by 0x3365E3: run_eval_code_obj (pythonrun.c:1165)
==355897==    by 0x35C853: run_mod (pythonrun.c:1187)
==355897==    by 0x21D38F: pyrun_file (pythonrun.c:1084)
```

It turns out it is caused by libtorch calling a function on the null pointer inside `THPVariable_Wrap`.

In `torch/csrc/autograd/python_variable.cpp` there is a variable called `THPVariableClass`, which is initialised to `nullptr`. The function `THPVariable_Wrap` (which we can see in the valgrind trace above) calls `THPVariable_NewWithVar` with `(PyTypeObject*)THPVariableClass` as the type argument, which is still the null pointer. `THPVariable_NewWithVar` then tries to call `type->tp_alloc(type, 0)` (where type is the `nullptr`), causing the segmentation fault.

Normally, it seems `THPVariableClass` is supposed to be initialised by the function `PyObject* THPAutograd_initExtension(PyObject* _unused, PyObject *unused)` (inside `torch/csrc/autograd/init.cpp`) which, it turns out, is automatically called when you do `import torch`. So adding `import torch` gets rid of the segmentation fault, i.e., the following python script works:
```
#!/usr/bin/python3
import dummy
import torch

dummy.zero_tensor(3,4,5)
```

## Expected behavior

I expected either:

- this to work without segfaulting;
OR
- for libtorch to throw an exception explaining that the user must `import torch` for custom C++ extensions to work.

I think adding a check to `THPVariable_Wrap` or `THPVariable_NewWithVar` to ensure `THPVariableClass` is not null and throwing an exception if it is, should allow the second option. The first option might be more complicated to implement - unless `THPAutograd_initExtension` can be called safely from inside either of the `THPVariable_` functions?

cc @yf225 @glaringlee @zou3519