Repository: cython/cython

Issue:
tp_dealloc generated code is unsafe for weak references
Consider the following class defined in Cython:

```
cdef class Example:
    cdef object __weakref__

    def __dealloc__(self):
        with nogil:
            some_operation()
```

This generates the following code for the `tp_dealloc` slot:

```C++
static void __pyx_tp_dealloc_98_session_Example(PyObject *o) {
  struct __pyx_obj_9_Example *p = (struct __pyx_obj_9_Example *)o;
  #if CYTHON_USE_TP_FINALIZE
  if (unlikely(PyType_HasFeature(Py_TYPE(o), Py_TPFLAGS_HAVE_FINALIZE) && Py_TYPE(o)->tp_finalize) && (!PyType_IS_GC(Py_TYPE(o)) || !_PyGC_FINALIZED(o))) {
    if (PyObject_CallFinalizerFromDealloc(o)) return;
  }
  #endif
  {
    PyObject *etype, *eval, *etb;
    PyErr_Fetch(&etype, &eval, &etb);
    ++Py_REFCNT(o);
    __pyx_pw_98_session_7Example_1__dealloc__(o);
    --Py_REFCNT(o);
    PyErr_Restore(etype, eval, etb);
  }
  if (p->__weakref__) PyObject_ClearWeakRefs(o);
  (*Py_TYPE(o)->tp_free)(o);
}
```

where `__pyx_pw_98_session_7Example_1__dealloc__` corresponds to the `__dealloc__` function defined on the class. The problem is that the interpreter calls this function with a reference count of 0 and this code needs to increment "artificially" the refcount of the object to call the `__pyx_pw_98_session_7Example_1__dealloc__`. This presents a problem as any Python call can release the GIL in `__pyx_pw_98_session_7Example_1__dealloc__` and leave the object with 1 reference but in an inconsistent state (half destroyed). This is particularly harmful if a weak reference (with or without some callback) exists for the object, as this can resurrect the object in an inconsistent state (because for the weakref point of view, the object is alive). In particular, even if that somehow succeeds without crashing something, the interpreter will crash when destruction continues (the call for `PyObject_ClearWeakRefs`) for the object hitting this line (as the weakref has resurrected the object, the refcount is still >0):

https://github.com/python/cpython/blob/master/Objects/weakrefobject.c#L894

I think the easiest way to solve this is to move the call to `PyObject_ClearWeakRefs` before the call to the type's `__dealloc` (`__pyx_pw_98_session_7Example_1__dealloc__`).


Discussion:

Resolving patch:
diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py
--- a/Cython/Compiler/ModuleNode.py
+++ b/Cython/Compiler/ModuleNode.py
@@ -1475,12 +1475,16 @@ def generate_dealloc_function(self, scope, code):
                 UtilityCode.load_cached("PyTrashcan", "ExtensionTypes.c"))
             code.putln("__Pyx_TRASHCAN_BEGIN(o, %s)" % slot_func_cname)
 
-        # call the user's __dealloc__
-        self.generate_usr_dealloc_call(scope, code)
-
         if weakref_slot:
+            # We must clean the weakreferences before calling the user's __dealloc__
+            # because if the __dealloc__ releases the GIL, a weakref can be
+            # dereferenced accessing the object in an inconsistent state or
+            # resurrecting it.
             code.putln("if (p->__weakref__) PyObject_ClearWeakRefs(o);")
 
+        # call the user's __dealloc__
+        self.generate_usr_dealloc_call(scope, code)
+
         if dict_slot:
             code.putln("if (p->__dict__) PyDict_Clear(p->__dict__);")
 


Test patch: