Repository: pytorch
Issue: Error from PyTorch when finalizing Python embedded in C++
Labels: triaged,module: pybind

## 🐛 Bug

I have a C++ application with an embedded Python session, and I've observed that importing PyTorch causes strange errors when the program ends. From what I can tell, the problem emerges because I finalize Python during the program exit, after PyTorch's internal objects have already been destroyed. PyTorch deallocates the objects' memory, but it does not update the corresponding reference counters. Thus, when Python finalizes and garbage collects, it attempts to deallocate memory that has already been freed and crashes.

If my interpretation is correct, may I suggest a few possibilities:
- Register a cleanup function with `atexit` to make sure internal Python objects are appropriately cleaned up, even if Python has not yet finalized.
- If C++ is an option, wrap the internal objects in classes with appropriate destructors.
- If this edge case is too weird and you decide not to support it, put an error message if the internal objects are destroyed before Python has been finalized.

## To Reproduce

Minimal working example:

<details> <summary> main.cpp </summary>

```c++
#include <Python.h>

/** Singleton class to manage embedded Python session. */
class session {
public:

  /** Get singleton instance.
   *
   *  Initializes an embedded Python session the first time it is
   *  called.
   */
  static session& get() {
    static session instance;
    return instance;
  }

  ~session() {
    if (Py_IsInitialized()) { Py_Finalize(); }
  }

private:

  session() {
    if (!Py_IsInitialized()) { Py_Initialize(); }
  }

  session(const session&) = delete;
  session& operator=(const session&) = delete;

};

int main(int argc, char** argv) {
  session::get(); // Start Python
#ifdef WITH_TORCH
  PyImport_ImportModule("torch");
#endif
  return EXIT_SUCCESS;
}
```

</details>

<details> <summary> Makefile </summary>

```makefile
# System-dependent variables
CXX ?= g++
PYTHON ?= python3
PYTHON_INC_DIR = $(shell $(PYTHON) -c "import sys; from distutils.sysconfig import get_python_inc; sys.stdout.write(get_python_inc())")
PYTHON_LIB = $(shell $(PYTHON) -c "import sys; from os.path import join; from distutils.sysconfig import get_config_var; sys.stdout.write(join(get_config_var('LIBDIR'),get_config_var('INSTSONAME')))")

# Compiler flags
CXX_FLAGS = -I$(PYTHON_INC_DIR)
LD_FLAGS = -Wl,$(PYTHON_LIB)

# ================================================
# Make rules
# ================================================

all: with_torch without_torch
with_torch: main.cpp
	$(CXX) -o $@ $^ $(CXX_FLAGS) $(LD_FLAGS) -DWITH_TORCH
without_torch: main.cpp
	$(CXX) -o $@ $^ $(CXX_FLAGS) $(LD_FLAGS)
%.o: %.cpp
        $(CXX) -c -o $@ $< $(CXX_FLAGS)

.PHONY: clean
clean:
      	rm -f main with_torch without_torch *.o *.core
```

</details>

Running the makefile will generate two executables: `without_torch` and `with_torch`. Both maintain a singleton class that initializes and finalizes an embedded Python session. The only difference is that `with_torch` imports PyTorch. `without_torch` runs fine and exits cleanly. `without_torch` crashes with the following message:

<details> <summary> Error message </summary>

```
*** Error in `/g/g17/moon13/temp/pytorch_bug/with_torch': corrupted double-linked list: 0x0000000000a52a50 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7f5d4)[0x2aaaabb4f5d4]
/lib64/libc.so.6(+0x8151d)[0x2aaaabb5151d]
/usr/tce/packages/python/python-3.7.2/lib/libpython3.7m.so.1.0(+0xb5be7)[0x2aaaaad84be7]
/usr/tce/packages/python/python-3.7.2/lib/libpython3.7m.so.1.0(+0xc9e97)[0x2aaaaad98e97]
/usr/tce/packages/python/python-3.7.2/lib/libpython3.7m.so.1.0(+0xf27f6)[0x2aaaaadc17f6]
/usr/tce/packages/python/python-3.7.2/lib/libpython3.7m.so.1.0(+0x1d7cd4)[0x2aaaaaea6cd4]
/usr/tce/packages/python/python-3.7.2/lib/libpython3.7m.so.1.0(_PyGC_CollectNoFail+0x3d)[0x2aaaaaea7fcd]
/usr/tce/packages/python/python-3.7.2/lib/libpython3.7m.so.1.0(PyImport_Cleanup+0x196)[0x2aaaaae68a66]
/usr/tce/packages/python/python-3.7.2/lib/libpython3.7m.so.1.0(Py_FinalizeEx+0xe0)[0x2aaaaae77b20]
/g/g17/moon13/temp/pytorch_bug/with_torch[0x400a9d]
/lib64/libc.so.6(+0x39b69)[0x2aaaabb09b69]
/lib64/libc.so.6(+0x39bb7)[0x2aaaabb09bb7]
/lib64/libc.so.6(__libc_start_main+0xfc)[0x2aaaabaf23dc]
/g/g17/moon13/temp/pytorch_bug/with_torch[0x400929]
======= Memory map: ========
```

</details>

Running `with_torch` under valgrind (with the [appropriate suppression file](https://svn.python.org/projects/python/trunk/Misc/valgrind-python.supp)) yields many errors similar to:

<details> <summary> Error message </summary>

```
==3436== Invalid read of size 8
==3436==    at 0x500F530: visit_decref (gcmodule.c:271)
==3436==    by 0x4F00C8D: dict_traverse (dictobject.c:2987)
==3436==    by 0x500E7D6: subtract_refs (gcmodule.c:296)
==3436==    by 0x500E7D6: collect (gcmodule.c:853)
==3436==    by 0x500FF24: collect_with_callback (gcmodule.c:1028)
==3436==    by 0x500FF24: PyGC_Collect (gcmodule.c:1573)
==3436==    by 0x4FDFB1A: Py_FinalizeEx (pylifecycle.c:1185)
==3436==    by 0x400A9C: session::~session() (in /g/g17/moon13/temp/pytorch_bug/with_torch)
==3436==    by 0x5C71B68: __run_exit_handlers (exit.c:77)
==3436==    by 0x5C71BB6: exit (exit.c:99)
==3436==    by 0x5C5A3DB: (below main) (libc-start.c:300)
==3436==  Address 0x9331efd8 is 8 bytes inside a block of size 16,128 free'd
==3436==    at 0x4C2B42D: operator delete(void*) (vg_replace_malloc.c:576)
==3436==    by 0x5C71B68: __run_exit_handlers (exit.c:77)
==3436==    by 0x5C71BB6: exit (exit.c:99)
==3436==    by 0x5C5A3DB: (below main) (libc-start.c:300)
==3436==  Block was alloc'd at
==3436==    at 0x4C2A4A3: operator new(unsigned long) (vg_replace_malloc.c:334)
==3436==    by 0xA8CAA33: torch::tensors::initialize_python_bindings() (in /usr/WS1/moon13/python/venv/pascal/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
==3436==    by 0xA4FD82E: THPModule_initExtension(_object*, _object*) (in /usr/WS1/moon13/python/venv/pascal/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
==3436==    by 0x4ED02B8: _PyMethodDef_RawFastCallKeywords (call.c:644)
==3436==    by 0x4ED0304: _PyCFunction_FastCallKeywords (call.c:730)
==3436==    by 0x4EAA613: call_function (ceval.c:4568)
==3436==    by 0x4EAA613: _PyEval_EvalFrameDefault (ceval.c:3093)
==3436==    by 0x4FB671D: _PyEval_EvalCodeWithName (ceval.c:3930)
==3436==    by 0x4FB683D: PyEval_EvalCodeEx (ceval.c:3959)
==3436==    by 0x4FB686A: PyEval_EvalCode (ceval.c:524)
==3436==    by 0x4FB3F34: builtin_exec_impl (bltinmodule.c:1079)
==3436==    by 0x4FB3F34: builtin_exec (bltinmodule.c.h:283)
==3436==    by 0x4ED099B: _PyMethodDef_RawFastCallDict (call.c:530)
==3436==    by 0x4ED0A34: _PyCFunction_FastCallDict (call.c:582)
```

</details>

## Expected behavior

`with_torch` should run and exit cleanly, like `without_torch`.

## Environment

```
~/temp/pytorch_bug $ python collect_env.py
Collecting environment information...
PyTorch version: 1.1.0
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Red Hat Enterprise Linux Server release 7.6 (Maipo)
GCC version: (GCC) 7.3.0
CMake version: version 3.12.1

Python version: 3.7
Is CUDA available: No
CUDA runtime version: 9.2.88
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.16.3
[pip3] torch==1.1.0
[pip3] torchvision==0.2.2.post3
[conda] Could not collect
```

## Additional context

This bug is also discussed in https://github.com/LLNL/lbann/issues/1111.