Repository: python/cpython
Pull request #132280: GH-124715: Move trashcan mechanism into `Py_Dealloc`
Author: login: markshannon
Created: 2025-04-08T16:36:58Z | Merged: 2025-04-30T10:37:55Z

<!--
Thanks for your contribution!
Please read this comment in its entirety. It's quite important.

# Pull Request title

It should be in the following format:

```
gh-NNNNN: Summary of the changes made
```

Where: gh-NNNNN refers to the GitHub issue number.

Most PRs will require an issue number. Trivial changes, like fixing a typo, do not need an issue.

# Backport Pull Request title

If this is a backport PR (PR made against branches other than `main`),
please ensure that the PR title is in the following format:

```
[X.Y] <title from the original PR> (GH-NNNN)
```

Where: [X.Y] is the branch name, e.g. [3.6].

GH-NNNN refers to the PR number from `main`.

-->


<!-- gh-issue-number: gh-124715 -->
* Issue: gh-124715
<!-- /gh-issue-number -->



Diff:
diff --git a/Include/cpython/object.h b/Include/cpython/object.h
index e2300aee7a207a..b6c508e6e29649 100644
--- a/Include/cpython/object.h
+++ b/Include/cpython/object.h
@@ -429,81 +429,14 @@ PyAPI_FUNC(void) _Py_NO_RETURN _PyObject_AssertFailed(
     const char *function);
 
 
-/* Trashcan mechanism, thanks to Christian Tismer.
-
-When deallocating a container object, it's possible to trigger an unbounded
-chain of deallocations, as each Py_DECREF in turn drops the refcount on "the
-next" object in the chain to 0.  This can easily lead to stack overflows,
-especially in threads (which typically have less stack space to work with).
-
-A container object can avoid this by bracketing the body of its tp_dealloc
-function with a pair of macros:
-
-static void
-mytype_dealloc(mytype *p)
-{
-    ... declarations go here ...
-
-    PyObject_GC_UnTrack(p);        // must untrack first
-    Py_TRASHCAN_BEGIN(p, mytype_dealloc)
-    ... The body of the deallocator goes here, including all calls ...
-    ... to Py_DECREF on contained objects.                         ...
-    Py_TRASHCAN_END                // there should be no code after this
-}
-
-CAUTION:  Never return from the middle of the body!  If the body needs to
-"get out early", put a label immediately before the Py_TRASHCAN_END
-call, and goto it.  Else the call-depth counter (see below) will stay
-above 0 forever, and the trashcan will never get emptied.
-
-How it works:  The BEGIN macro increments a call-depth counter.  So long
-as this counter is small, the body of the deallocator is run directly without
-further ado.  But if the counter gets large, it instead adds p to a list of
-objects to be deallocated later, skips the body of the deallocator, and
-resumes execution after the END macro.  The tp_dealloc routine then returns
-without deallocating anything (and so unbounded call-stack depth is avoided).
-
-When the call stack finishes unwinding again, code generated by the END macro
-notices this, and calls another routine to deallocate all the objects that
-may have been added to the list of deferred deallocations.  In effect, a
-chain of N deallocations is broken into (N-1)/(Py_TRASHCAN_HEADROOM-1) pieces,
-with the call stack never exceeding a depth of Py_TRASHCAN_HEADROOM.
-
-Since the tp_dealloc of a subclass typically calls the tp_dealloc of the base
-class, we need to ensure that the trashcan is only triggered on the tp_dealloc
-of the actual class being deallocated. Otherwise we might end up with a
-partially-deallocated object. To check this, the tp_dealloc function must be
-passed as second argument to Py_TRASHCAN_BEGIN().
-*/
-
-
 PyAPI_FUNC(void) _PyTrash_thread_deposit_object(PyThreadState *tstate, PyObject *op);
 PyAPI_FUNC(void) _PyTrash_thread_destroy_chain(PyThreadState *tstate);
 
-
-/* Python 3.10 private API, invoked by the Py_TRASHCAN_BEGIN(). */
-
-/* To avoid raising recursion errors during dealloc trigger trashcan before we reach
- * recursion limit. To avoid trashing, we don't attempt to empty the trashcan until
- * we have headroom above the trigger limit */
-#define Py_TRASHCAN_HEADROOM 50
-
-/* Helper function for Py_TRASHCAN_BEGIN */
 PyAPI_FUNC(int) _Py_ReachedRecursionLimitWithMargin(PyThreadState *tstate, int margin_count);
 
-#define Py_TRASHCAN_BEGIN(op, dealloc) \
-do { \
-    PyThreadState *tstate = PyThreadState_Get(); \
-    if (_Py_ReachedRecursionLimitWithMargin(tstate, 2) && Py_TYPE(op)->tp_dealloc == (destructor)dealloc) { \
-        _PyTrash_thread_deposit_object(tstate, (PyObject *)op); \
-        break; \
-    }
-    /* The body of the deallocator is here. */
-#define Py_TRASHCAN_END \
-    if (tstate->delete_later && !_Py_ReachedRecursionLimitWithMargin(tstate, 4)) { \
-        _PyTrash_thread_destroy_chain(tstate); \
-    } \
-} while (0);
+/* For backwards compatibility with the old trashcan mechanism */
+#define Py_TRASHCAN_BEGIN(op, dealloc)
+#define Py_TRASHCAN_END
 
 
 PyAPI_FUNC(void *) PyObject_GetItemData(PyObject *obj);
diff --git a/Include/internal/pycore_ceval.h b/Include/internal/pycore_ceval.h
index 18c8bc0624fea7..942a727339ee4f 100644
--- a/Include/internal/pycore_ceval.h
+++ b/Include/internal/pycore_ceval.h
@@ -196,25 +196,6 @@ extern void _PyEval_DeactivateOpCache(void);
 
 /* --- _Py_EnterRecursiveCall() ----------------------------------------- */
 
-#if !_Py__has_builtin(__builtin_frame_address) && !defined(_MSC_VER)
-static uintptr_t return_pointer_as_int(char* p) {
-    return (uintptr_t)p;
-}
-#endif
-
-static inline uintptr_t
-_Py_get_machine_stack_pointer(void) {
-#if _Py__has_builtin(__builtin_frame_address)
-    return (uintptr_t)__builtin_frame_address(0);
-#elif defined(_MSC_VER)
-    return (uintptr_t)_AddressOfReturnAddress();
-#else
-    char here;
-    /* Avoid compiler warning about returning stack address */
-    return return_pointer_as_int(&here);
-#endif
-}
-
 static inline int _Py_MakeRecCheck(PyThreadState *tstate)  {
     uintptr_t here_addr = _Py_get_machine_stack_pointer();
     _PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate;
@@ -249,12 +230,7 @@ PyAPI_FUNC(void) _Py_InitializeRecursionLimits(PyThreadState *tstate);
 static inline int _Py_ReachedRecursionLimit(PyThreadState *tstate)  {
     uintptr_t here_addr = _Py_get_machine_stack_pointer();
     _PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate;
-    if (here_addr > _tstate->c_stack_soft_limit) {
-        return 0;
-    }
-    if (_tstate->c_stack_hard_limit == 0) {
-        _Py_InitializeRecursionLimits(tstate);
-    }
+    assert(_tstate->c_stack_hard_limit != 0);
     return here_addr <= _tstate->c_stack_soft_limit;
 }
 
diff --git a/Include/internal/pycore_pystate.h b/Include/internal/pycore_pystate.h
index 864e0f5d1db289..81bdc465f0703b 100644
--- a/Include/internal/pycore_pystate.h
+++ b/Include/internal/pycore_pystate.h
@@ -9,6 +9,7 @@ extern "C" {
 #endif
 
 #include "pycore_typedefs.h"      // _PyRuntimeState
+#include "pycore_tstate.h"
 
 
 // Values for PyThreadState.state. A thread must be in the "attached" state
@@ -296,6 +297,34 @@ _Py_AssertHoldsTstateFunc(const char *func)
 #define _Py_AssertHoldsTstate()
 #endif
 
+#if !_Py__has_builtin(__builtin_frame_address) && !defined(_MSC_VER)
+static uintptr_t return_pointer_as_int(char* p) {
+    return (uintptr_t)p;
+}
+#endif
+
+static inline uintptr_t
+_Py_get_machine_stack_pointer(void) {
+#if _Py__has_builtin(__builtin_frame_address)
+    return (uintptr_t)__builtin_frame_address(0);
+#elif defined(_MSC_VER)
+    return (uintptr_t)_AddressOfReturnAddress();
+#else
+    char here;
+    /* Avoid compiler warning about returning stack address */
+    return return_pointer_as_int(&here);
+#endif
+}
+
+static inline intptr_t
+_Py_RecursionLimit_GetMargin(PyThreadState *tstate)
+{
+    _PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate;
+    assert(_tstate->c_stack_hard_limit != 0);
+    intptr_t here_addr = _Py_get_machine_stack_pointer();
+    return Py_ARITHMETIC_RIGHT_SHIFT(intptr_t, here_addr - (intptr_t)_tstate->c_stack_soft_limit, PYOS_STACK_MARGIN_SHIFT);
+}
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/Include/pythonrun.h b/Include/pythonrun.h
index 4d459cb92e36ea..fad2b3c77476e4 100644
--- a/Include/pythonrun.h
+++ b/Include/pythonrun.h
@@ -26,17 +26,25 @@ PyAPI_DATA(int) (*PyOS_InputHook)(void);
  * apart. In practice, that means it must be larger than the C
  * stack consumption of PyEval_EvalDefault */
 #if defined(_Py_ADDRESS_SANITIZER) || defined(_Py_THREAD_SANITIZER)
-#  define PYOS_STACK_MARGIN 4096
+#  define PYOS_LOG2_STACK_MARGIN 12
 #elif defined(Py_DEBUG) && defined(WIN32)
-#  define PYOS_STACK_MARGIN 4096
+#  define PYOS_LOG2_STACK_MARGIN 12
 #elif defined(__wasi__)
    /* Web assembly has two stacks, so this isn't really a size */
-#  define PYOS_STACK_MARGIN 500
+#  define PYOS_LOG2_STACK_MARGIN 9
 #else
-#  define PYOS_STACK_MARGIN 2048
+#  define PYOS_LOG2_STACK_MARGIN 11
 #endif
+#define PYOS_STACK_MARGIN (1 << PYOS_LOG2_STACK_MARGIN)
 #define PYOS_STACK_MARGIN_BYTES (PYOS_STACK_MARGIN * sizeof(void *))
 
+#if SIZEOF_VOID_P == 8
+#define PYOS_STACK_MARGIN_SHIFT (PYOS_LOG2_STACK_MARGIN + 3)
+#else
+#define PYOS_STACK_MARGIN_SHIFT (PYOS_LOG2_STACK_MARGIN + 2)
+#endif
+
+
 #if defined(WIN32)
 #define USE_STACKCHECK
 #endif
diff --git a/Misc/NEWS.d/next/Core_and_Builtins/2025-04-08-17-48-11.gh-issue-124715.xxzQoD.rst b/Misc/NEWS.d/next/Core_and_Builtins/2025-04-08-17-48-11.gh-issue-124715.xxzQoD.rst
new file mode 100644
index 00000000000000..f0e3318560902a
--- /dev/null
+++ b/Misc/NEWS.d/next/Core_and_Builtins/2025-04-08-17-48-11.gh-issue-124715.xxzQoD.rst
@@ -0,0 +1,3 @@
+Prevents against stack overflows when calling :c:func:`Py_DECREF`. Third-party
+extension objects no longer need to use the "trashcan" mechanism, as
+protection is now built into the :c:func:`Py_DECREF` macro.
diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c
index bf221106c31063..eed29dbabf5eef 100644
--- a/Modules/_elementtree.c
+++ b/Modules/_elementtree.c
@@ -689,7 +689,6 @@ element_dealloc(PyObject *op)
 
     /* bpo-31095: UnTrack is needed before calling any callbacks */
     PyObject_GC_UnTrack(self);
-    Py_TRASHCAN_BEGIN(self, element_dealloc)
 
     if (self->weakreflist != NULL)
         PyObject_ClearWeakRefs(op);
@@ -700,7 +699,6 @@ element_dealloc(PyObject *op)
 
     tp->tp_free(self);
     Py_DECREF(tp);
-    Py_TRASHCAN_END
 }
 
 /* -------------------------------------------------------------------- */
diff --git a/Objects/descrobject.c b/Objects/descrobject.c
index 451e07f58e161e..48b52e03ea30c1 100644
--- a/Objects/descrobject.c
+++ b/Objects/descrobject.c
@@ -1311,11 +1311,9 @@ wrapper_dealloc(PyObject *self)
 {
     wrapperobject *wp = (wrapperobject *)self;
     PyObject_GC_UnTrack(wp);
-    Py_TRASHCAN_BEGIN(wp, wrapper_dealloc)
     Py_XDECREF(wp->descr);
     Py_XDECREF(wp->self);
     PyObject_GC_Del(wp);
-    Py_TRASHCAN_END
 }
 
 static PyObject *
diff --git a/Objects/dictobject.c b/Objects/dictobject.c
index 792a34cc569fe8..454b87f1f111ea 100644
--- a/Objects/dictobject.c
+++ b/Objects/dictobject.c
@@ -3262,7 +3262,6 @@ dict_dealloc(PyObject *self)
 
     /* bpo-31095: UnTrack is needed before calling any callbacks */
     PyObject_GC_UnTrack(mp);
-    Py_TRASHCAN_BEGIN(mp, dict_dealloc)
     if (values != NULL) {
         if (values->embedded == 0) {
             for (i = 0, n = values->capacity; i < n; i++) {
@@ -3282,7 +3281,6 @@ dict_dealloc(PyObject *self)
     else {
         Py_TYPE(mp)->tp_free((PyObject *)mp);
     }
-    Py_TRASHCAN_END
 }
 
 
diff --git a/Objects/exceptions.c b/Objects/exceptions.c
index 942b0c630e33f0..6e50c59a4d26d1 100644
--- a/Objects/exceptions.c
+++ b/Objects/exceptions.c
@@ -150,10 +150,8 @@ BaseException_dealloc(PyObject *op)
     // bpo-44348: The trashcan mechanism prevents stack overflow when deleting
     // long chains of exceptions. For example, exceptions can be chained
     // through the __context__ attributes or the __traceback__ attribute.
-    Py_TRASHCAN_BEGIN(self, BaseException_dealloc)
     (void)BaseException_clear(op);
     Py_TYPE(self)->tp_free(self);
-    Py_TRASHCAN_END
 }
 
 static int
diff --git a/Objects/frameobject.c b/Objects/frameobject.c
index db4e4b7e1939de..5e85745f45fa01 100644
--- a/Objects/frameobject.c
+++ b/Objects/frameobject.c
@@ -1916,7 +1916,6 @@ frame_dealloc(PyObject *op)
         _PyObject_GC_UNTRACK(f);
     }
 
-    Py_TRASHCAN_BEGIN(f, frame_dealloc);
     /* GH-106092: If f->f_frame was on the stack and we reached the maximum
      * nesting depth for deallocations, the trashcan may have delayed this
      * deallocation until after f->f_frame is freed. Avoid dereferencing
@@ -1941,7 +1940,6 @@ frame_dealloc(PyObject *op)
     Py_CLEAR(f->f_locals_cache);
     Py_CLEAR(f->f_overwritten_fast_locals);
     PyObject_GC_Del(f);
-    Py_TRASHCAN_END;
 }
 
 static int
diff --git a/Objects/listobject.c b/Objects/listobject.c
index 12d5b33414a92a..efffd61fe0e22e 100644
--- a/Objects/listobject.c
+++ b/Objects/listobject.c
@@ -550,7 +550,6 @@ list_dealloc(PyObject *self)
     PyListObject *op = (PyListObject *)self;
     Py_ssize_t i;
     PyObject_GC_UnTrack(op);
-    Py_TRASHCAN_BEGIN(op, list_dealloc)
     if (op->ob_item != NULL) {
         /* Do it backwards, for Christian Tismer.
            There's a simple test case where somehow this reduces
@@ -569,7 +568,6 @@ list_dealloc(PyObject *self)
     else {
         PyObject_GC_Del(op);
     }
-    Py_TRASHCAN_END
 }
 
 static PyObject *
diff --git a/Objects/methodobject.c b/Objects/methodobject.c
index 1f459dea44192c..6d655f095ce8e0 100644
--- a/Objects/methodobject.c
+++ b/Objects/methodobject.c
@@ -166,10 +166,7 @@ static void
 meth_dealloc(PyObject *self)
 {
     PyCFunctionObject *m = _PyCFunctionObject_CAST(self);
-    // The Py_TRASHCAN mechanism requires that we be able to
-    // call PyObject_GC_UnTrack twice on an object.
     PyObject_GC_UnTrack(m);
-    Py_TRASHCAN_BEGIN(m, meth_dealloc);
     if (m->m_weakreflist != NULL) {
         PyObject_ClearWeakRefs((PyObject*) m);
     }
@@ -186,7 +183,6 @@ meth_dealloc(PyObject *self)
         assert(Py_IS_TYPE(self, &PyCFunction_Type));
         _Py_FREELIST_FREE(pycfunctionobject, m, PyObject_GC_Del);
     }
-    Py_TRASHCAN_END;
 }
 
 static PyObject *
diff --git a/Objects/object.c b/Objects/object.c
index 99bb1d9c0bfad5..df5d7a2af23a56 100644
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -2908,13 +2908,15 @@ Py_ReprLeave(PyObject *obj)
 void
 _PyTrash_thread_deposit_object(PyThreadState *tstate, PyObject *op)
 {
-    _PyObject_ASSERT(op, _PyObject_IS_GC(op));
-    _PyObject_ASSERT(op, !_PyObject_GC_IS_TRACKED(op));
     _PyObject_ASSERT(op, Py_REFCNT(op) == 0);
 #ifdef Py_GIL_DISABLED
     op->ob_tid = (uintptr_t)tstate->delete_later;
 #else
-    _PyGCHead_SET_PREV(_Py_AS_GC(op), (PyGC_Head*)tstate->delete_later);
+    /* Store the delete_later pointer in the refcnt field.
+     * As this object may still be tracked by the GC,
+     * it is important that we never store 0 (NULL). */
+    uintptr_t refcnt = (uintptr_t)tstate->delete_later;
+    *((uintptr_t*)op) = refcnt+1;
 #endif
     tstate->delete_later = op;
 }
@@ -2933,7 +2935,11 @@ _PyTrash_thread_destroy_chain(PyThreadState *tstate)
         op->ob_tid = 0;
         _Py_atomic_store_ssize_relaxed(&op->ob_ref_shared, _Py_REF_MERGED);
 #else
-        tstate->delete_later = (PyObject*) _PyGCHead_PREV(_Py_AS_GC(op));
+        /* Get the delete_later pointer from the refcnt field.
+         * See _PyTrash_thread_deposit_object(). */
+        uintptr_t refcnt = *((uintptr_t*)op);
+        tstate->delete_later = (PyObject *)(refcnt - 1);
+        op->ob_refcnt = 0;
 #endif
 
         /* Call the deallocator directly.  This used to try to
@@ -2998,13 +3004,25 @@ _PyObject_AssertFailed(PyObject *obj, const char *expr, const char *msg,
 }
 
 
+/*
+When deallocating a container object, it's possible to trigger an unbounded
+chain of deallocations, as each Py_DECREF in turn drops the refcount on "the
+next" object in the chain to 0.  This can easily lead to stack overflows.
+To avoid that, if the C stack is nearing its limit, instead of calling
+dealloc on the object, it is added to a queue to be freed later when the
+stack is shallower */
 void
 _Py_Dealloc(PyObject *op)
 {
     PyTypeObject *type = Py_TYPE(op);
     destructor dealloc = type->tp_dealloc;
-#ifdef Py_DEBUG
     PyThreadState *tstate = _PyThreadState_GET();
+    intptr_t margin = _Py_RecursionLimit_GetMargin(tstate);
+    if (margin < 2) {
+        _PyTrash_thread_deposit_object(tstate, (PyObject *)op);
+        return;
+    }
+#ifdef Py_DEBUG
 #if !defined(Py_GIL_DISABLED) && !defined(Py_STACKREF_DEBUG)
     /* This assertion doesn't hold for the free-threading build, as
      * PyStackRef_CLOSE_SPECIALIZED is not implemented */
@@ -3046,6 +3064,9 @@ _Py_Dealloc(PyObject *op)
     Py_XDECREF(old_exc);
     Py_DECREF(type);
 #endif
+    if (tstate->delete_later && margin >= 4) {
+        _PyTrash_thread_destroy_chain(tstate);
+    }
 }
 
 
diff --git a/Objects/odictobject.c b/Objects/odictobject.c
index 1412acb50ac5ff..891f6197401503 100644
--- a/Objects/odictobject.c
+++ b/Objects/odictobject.c
@@ -1389,7 +1389,6 @@ odict_dealloc(PyObject *op)
 {
     PyODictObject *self = _PyODictObject_CAST(op);
     PyObject_GC_UnTrack(self);
-    Py_TRASHCAN_BEGIN(self, odict_dealloc)
 
     Py_XDECREF(self->od_inst_dict);
     if (self->od_weakreflist != NULL)
@@ -1397,8 +1396,6 @@ odict_dealloc(PyObject *op)
 
     _odict_clear_nodes(self);
     PyDict_Type.tp_dealloc((PyObject *)self);
-
-    Py_TRASHCAN_END
 }
 
 /* tp_repr */
diff --git a/Objects/setobject.c b/Objects/setobject.c
index acbb53aafc0a26..2b92249c1fc642 100644
--- a/Objects/setobject.c
+++ b/Objects/setobject.c
@@ -516,7 +516,6 @@ set_dealloc(PyObject *self)
 
     /* bpo-31095: UnTrack is needed before calling any callbacks */
     PyObject_GC_UnTrack(so);
-    Py_TRASHCAN_BEGIN(so, set_dealloc)
     if (so->weakreflist != NULL)
         PyObject_ClearWeakRefs((PyObject *) so);
 
@@ -529,7 +528,6 @@ set_dealloc(PyObject *self)
     if (so->table != so->smalltable)
         PyMem_Free(so->table);
     Py_TYPE(so)->tp_free(so);
-    Py_TRASHCAN_END
 }
 
 static PyObject *
diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c
index 737c4e6d97794a..9b31758485ca5e 100644
--- a/Objects/tupleobject.c
+++ b/Objects/tupleobject.c
@@ -207,7 +207,6 @@ tuple_dealloc(PyObject *self)
     }
 
     PyObject_GC_UnTrack(op);
-    Py_TRASHCAN_BEGIN(op, tuple_dealloc)
 
     Py_ssize_t i = Py_SIZE(op);
     while (--i >= 0) {
@@ -217,8 +216,6 @@ tuple_dealloc(PyObject *self)
     if (!maybe_freelist_push(op)) {
         Py_TYPE(op)->tp_free((PyObject *)op);
     }
-
-    Py_TRASHCAN_END
 }
 
 static PyObject *
diff --git a/Objects/typeobject.c b/Objects/typeobject.c
index b92eaefc90d0af..c46f7be3dae8f5 100644
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -2555,7 +2555,6 @@ subtype_dealloc(PyObject *self)
     /* UnTrack and re-Track around the trashcan macro, alas */
     /* See explanation at end of function for full disclosure */
     PyObject_GC_UnTrack(self);
-    Py_TRASHCAN_BEGIN(self, subtype_dealloc);
 
     /* Find the nearest base with a different tp_dealloc */
     base = type;
@@ -2570,7 +2569,7 @@ subtype_dealloc(PyObject *self)
         _PyObject_GC_TRACK(self);
         if (PyObject_CallFinalizerFromDealloc(self) < 0) {
             /* Resurrected */
-            goto endlabel;
+            return;
         }
         _PyObject_GC_UNTRACK(self);
     }
@@ -2592,7 +2591,7 @@ subtype_dealloc(PyObject *self)
         type->tp_del(self);
         if (Py_REFCNT(self) > 0) {
             /* Resurrected */
-            goto endlabel;
+            return;
         }
         _PyObject_GC_UNTRACK(self);
     }
@@ -2655,46 +2654,6 @@ subtype_dealloc(PyObject *self)
     if (type_needs_decref) {
         _Py_DECREF_TYPE(type);
     }
-
-  endlabel:
-    Py_TRASHCAN_END
-
-    /* Explanation of the weirdness around the trashcan macros:
-
-       Q. What do the trashcan macros do?
-
-       A. Read the comment titled "Trashcan mechanism" in object.h.
-          For one, this explains why there must be a call to GC-untrack
-          before the trashcan begin macro.      Without understanding the
-          trashcan code, the answers to the following questions don't make
-          sense.
-
-       Q. Why do we GC-untrack before the trashcan and then immediately
-          GC-track again afterward?
-
-       A. In the case that the base class is GC-aware, the base class
-          probably GC-untracks the object.      If it does that using the
-          UNTRACK macro, this will crash when the object is already
-          untracked.  Because we don't know what the base class does, the
-          only safe thing is to make sure the object is tracked when we
-          call the base class dealloc.  But...  The trashcan begin macro
-          requires that the object is *untracked* before it is called.  So
-          the dance becomes:
-
-         GC untrack
-         trashcan begin
-         GC track
-
-       Q. Why did the last question say "immediately GC-track again"?
-          It's nowhere near immediately.
-
-       A. Because the code *used* to re-track immediately.      Bad Idea.
-          self has a refcount of 0, and if gc ever gets its hands on it
-          (which can happen if any weakref callback gets invoked), it
-          looks like trash to gc too, and gc also tries to delete self
-          then.  But we're already deleting self.  Double deallocation is
-          a subtle disaster.
-    */
 }
 
 static PyTypeObject *solid_base(PyTypeObject *type);
diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c
index 10e415fa052f64..c6d45b28283ec3 100644
--- a/Python/bltinmodule.c
+++ b/Python/bltinmodule.c
@@ -566,11 +566,9 @@ filter_dealloc(PyObject *self)
 {
     filterobject *lz = _filterobject_CAST(self);
     PyObject_GC_UnTrack(lz);
-    Py_TRASHCAN_BEGIN(lz, filter_dealloc)
     Py_XDECREF(lz->func);
     Py_XDECREF(lz->it);
     Py_TYPE(lz)->tp_free(lz);
-    Py_TRASHCAN_END
 }
 
 static int
diff --git a/Python/ceval.c b/Python/ceval.c
index a59b2b7a16866d..616bb0253f4142 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -476,12 +476,6 @@ _Py_CheckRecursiveCall(PyThreadState *tstate, const char *where)
     _PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate;
     uintptr_t here_addr = _Py_get_machine_stack_pointer();
     assert(_tstate->c_stack_soft_limit != 0);
-    if (_tstate->c_stack_hard_limit == 0) {
-        _Py_InitializeRecursionLimits(tstate);
-    }
-    if (here_addr >= _tstate->c_stack_soft_limit) {
-        return 0;
-    }
     assert(_tstate->c_stack_hard_limit != 0);
     if (here_addr < _tstate->c_stack_hard_limit) {
         /* Overflowing while handling an overflow. Give up. */
diff --git a/Python/gc.c b/Python/gc.c
index dad088e09f872f..f9245d3417a20d 100644
--- a/Python/gc.c
+++ b/Python/gc.c
@@ -2201,9 +2201,8 @@ void
 PyObject_GC_UnTrack(void *op_raw)
 {
     PyObject *op = _PyObject_CAST(op_raw);
-    /* Obscure:  the Py_TRASHCAN mechanism requires that we be able to
-     * call PyObject_GC_UnTrack twice on an object.
-     */
+    /* The code for some objects, such as tuples, is a bit
+     * sloppy about when the object is tracked and untracked. */
     if (_PyObject_GC_IS_TRACKED(op)) {
         _PyObject_GC_UNTRACK(op);
     }
diff --git a/Python/gc_free_threading.c b/Python/gc_free_threading.c
index 4c459b02ce2b8f..692154838b0ef3 100644
--- a/Python/gc_free_threading.c
+++ b/Python/gc_free_threading.c
@@ -2500,9 +2500,8 @@ void
 PyObject_GC_UnTrack(void *op_raw)
 {
     PyObject *op = _PyObject_CAST(op_raw);
-    /* Obscure:  the Py_TRASHCAN mechanism requires that we be able to
-     * call PyObject_GC_UnTrack twice on an object.
-     */
+    /* The code for some objects, such as tuples, is a bit
+     * sloppy about when the object is tracked and untracked. */
     if (_PyObject_GC_IS_TRACKED(op)) {
         _PyObject_GC_UNTRACK(op);
     }
diff --git a/Python/hamt.c b/Python/hamt.c
index e4d1e1663dd573..f9bbf63961d8de 100644
--- a/Python/hamt.c
+++ b/Python/hamt.c
@@ -1118,7 +1118,6 @@ hamt_node_bitmap_dealloc(PyObject *self)
     }
 
     PyObject_GC_UnTrack(self);
-    Py_TRASHCAN_BEGIN(self, hamt_node_bitmap_dealloc)
 
     if (len > 0) {
         i = len;
@@ -1128,7 +1127,6 @@ hamt_node_bitmap_dealloc(PyObject *self)
     }
 
     Py_TYPE(self)->tp_free(self);
-    Py_TRASHCAN_END
 }
 
 #ifdef Py_DEBUG
@@ -1508,7 +1506,6 @@ hamt_node_collision_dealloc(PyObject *self)
     /* Collision's tp_dealloc */
     Py_ssize_t len = Py_SIZE(self);
     PyObject_GC_UnTrack(self);
-    Py_TRASHCAN_BEGIN(self, hamt_node_collision_dealloc)
     if (len > 0) {
         PyHamtNode_Collision *node = _PyHamtNode_Collision_CAST(self);
         while (--len >= 0) {
@@ -1516,7 +1513,6 @@ hamt_node_collision_dealloc(PyObject *self)
         }
     }
     Py_TYPE(self)->tp_free(self);
-    Py_TRASHCAN_END
 }
 
 #ifdef Py_DEBUG
@@ -1878,13 +1874,11 @@ hamt_node_array_dealloc(PyObject *self)
 {
     /* Array's tp_dealloc */
     PyObject_GC_UnTrack(self);
-    Py_TRASHCAN_BEGIN(self, hamt_node_array_dealloc)
     PyHamtNode_Array *obj = _PyHamtNode_Array_CAST(self);
     for (Py_ssize_t i = 0; i < HAMT_ARRAY_NODE_SIZE; i++) {
         Py_XDECREF(obj->a_array[i]);
     }
     Py_TYPE(self)->tp_free(self);
-    Py_TRASHCAN_END
 }
 
 #ifdef Py_DEBUG
diff --git a/Python/instruction_sequence.c b/Python/instruction_sequence.c
index 4ca85eec345d38..3e6c1ffdef346a 100644
--- a/Python/instruction_sequence.c
+++ b/Python/instruction_sequence.c
@@ -403,10 +403,8 @@ inst_seq_dealloc(PyObject *op)
 {
     _PyInstructionSequence *seq = (_PyInstructionSequence *)op;
     PyObject_GC_UnTrack(seq);
-    Py_TRASHCAN_BEGIN(seq, inst_seq_dealloc)
     PyInstructionSequence_Fini(seq);
     PyObject_GC_Del(seq);
-    Py_TRASHCAN_END
 }
 
 static int
diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c
index 934614e73b56f9..9e9d99edf17b42 100644
--- a/Python/pylifecycle.c
+++ b/Python/pylifecycle.c
@@ -848,6 +848,10 @@ pycore_init_builtins(PyThreadState *tstate)
 static PyStatus
 pycore_interp_init(PyThreadState *tstate)
 {
+    _PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate;
+    if (_tstate->c_stack_hard_limit == 0) {
+        _Py_InitializeRecursionLimits(tstate);
+    }
     PyInterpreterState *interp = tstate->interp;
     PyStatus status;
     PyObject *sysmod = NULL;
diff --git a/Python/pystate.c b/Python/pystate.c
index ee35f0fa945f8b..63661815ecd06f 100644
--- a/Python/pystate.c
+++ b/Python/pystate.c
@@ -2132,7 +2132,10 @@ _PyThreadState_Attach(PyThreadState *tstate)
     if (current_fast_get() != NULL) {
         Py_FatalError("non-NULL old thread state");
     }
-
+    _PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate;
+    if (_tstate->c_stack_hard_limit == 0) {
+        _Py_InitializeRecursionLimits(tstate);
+    }
 
     while (1) {
         _PyEval_AcquireLock(tstate);
diff --git a/Python/traceback.c b/Python/traceback.c
index 6a6a3c29859fbc..2d1cb6b1eb7a99 100644
--- a/Python/traceback.c
+++ b/Python/traceback.c
@@ -205,11 +205,9 @@ tb_dealloc(PyObject *op)
 {
     PyTracebackObject *tb = _PyTracebackObject_CAST(op);
     PyObject_GC_UnTrack(tb);
-    Py_TRASHCAN_BEGIN(tb, tb_dealloc)
     Py_XDECREF(tb->tb_next);
     Py_XDECREF(tb->tb_frame);
     PyObject_GC_Del(tb);
-    Py_TRASHCAN_END
 }
 
 static int


Reviews:
user: login: nascheme
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T19:55:10Z
user: login: vstinner
body: The overall design LGTM.

I added a few comments.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-15T14:04:44Z
user: login: iritkatriel
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-30T10:29:37Z

Review comments:
diff_hunk: @@ -2908,13 +2908,11 @@ Py_ReprLeave(PyObject *obj)
 void
 _PyTrash_thread_deposit_object(PyThreadState *tstate, PyObject *op)
 {
-    _PyObject_ASSERT(op, _PyObject_IS_GC(op));
-    _PyObject_ASSERT(op, !_PyObject_GC_IS_TRACKED(op));
     _PyObject_ASSERT(op, Py_REFCNT(op) == 0);
 #ifdef Py_GIL_DISABLED
     op->ob_tid = (uintptr_t)tstate->delete_later;
 #else
-    _PyGCHead_SET_PREV(_Py_AS_GC(op), (PyGC_Head*)tstate->delete_later);
+    *((PyObject**)op) = tstate->delete_later;
path: Objects/object.c
user: login: nascheme
body: Line 2915 is a little obscure.  I think a comment explaining that `ob_refcnt` is being used to create the queue would be helpful.
created_at: 2025-04-08T19:55:10Z
author_association: MEMBER
diff_hunk: @@ -2656,45 +2655,6 @@ subtype_dealloc(PyObject *self)
         _Py_DECREF_TYPE(type);
     }
 
path: Objects/typeobject.c
user: login: vstinner
body: You can now remove this empty line.
created_at: 2025-04-15T13:51:26Z
author_association: MEMBER
diff_hunk: @@ -26,17 +26,25 @@ PyAPI_DATA(int) (*PyOS_InputHook)(void);
  * apart. In practice, that means it must be larger than the C
  * stack consumption of PyEval_EvalDefault */
 #if defined(_Py_ADDRESS_SANITIZER) || defined(_Py_THREAD_SANITIZER)
-#  define PYOS_STACK_MARGIN 4096
+#  define PYOS_LOG_STACK_MARGIN 12
path: Include/pythonrun.h
user: login: vstinner
body: You may rename this constant to `PYOS_LOG2_STACK_MARGIN`. It took me a while to understand that it was the `log(n)/log(2)` function :-) Or just add a a comment explaining what "LOG" stands for here.
created_at: 2025-04-15T13:55:28Z
author_association: MEMBER
diff_hunk: @@ -287,7 +287,9 @@ static void
 capsule_dealloc(PyObject *op)
 {
     PyCapsule *capsule = _PyCapsule_CAST(op);
-    PyObject_GC_UnTrack(op);
+    if (_PyObject_GC_IS_TRACKED(op)) {
+        PyObject_GC_UnTrack(op);
+    }
path: Objects/capsule.c
user: login: vstinner
body: Is this change needed? PyObject_GC_UnTrack() already checks if the object is tracked or not, no?
created_at: 2025-04-15T13:58:05Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,3 @@
+Prevents against stack overflows when calling Py_DECREF. Third-party
+extension objects no longer need to use the "trashcan" mechanism, as
+protection is now built into the ``Py_DECREF`` macro.
path: Misc/NEWS.d/next/Core_and_Builtins/2025-04-08-17-48-11.gh-issue-124715.xxzQoD.rst
user: login: vstinner
body: ```suggestion
Prevents against stack overflows when calling :c:func:`Py_DECREF`. Third-party
extension objects no longer need to use the "trashcan" mechanism, as
protection is now built into the :c:func:`Py_DECREF` function.
```
created_at: 2025-04-15T13:59:32Z
author_association: MEMBER
diff_hunk: @@ -2933,7 +2935,9 @@ _PyTrash_thread_destroy_chain(PyThreadState *tstate)
         op->ob_tid = 0;
         _Py_atomic_store_ssize_relaxed(&op->ob_ref_shared, _Py_REF_MERGED);
 #else
-        tstate->delete_later = (PyObject*) _PyGCHead_PREV(_Py_AS_GC(op));
+        uintptr_t refcnt = *((uintptr_t*)op);
path: Objects/object.c
user: login: vstinner
body: The `-1` operation is a bit magic, I suggest adding this comment:

```suggestion
        /* Get the delete_later pointer from the refcnt field.
         * See _PyTrash_thread_deposit_object(). */
        uintptr_t refcnt = *((uintptr_t*)op);
```
created_at: 2025-04-15T14:02:27Z
author_association: MEMBER
diff_hunk: @@ -2908,13 +2908,15 @@ Py_ReprLeave(PyObject *obj)
 void
 _PyTrash_thread_deposit_object(PyThreadState *tstate, PyObject *op)
 {
-    _PyObject_ASSERT(op, _PyObject_IS_GC(op));
-    _PyObject_ASSERT(op, !_PyObject_GC_IS_TRACKED(op));
     _PyObject_ASSERT(op, Py_REFCNT(op) == 0);
 #ifdef Py_GIL_DISABLED
     op->ob_tid = (uintptr_t)tstate->delete_later;
 #else
-    _PyGCHead_SET_PREV(_Py_AS_GC(op), (PyGC_Head*)tstate->delete_later);
+    /* Store the pointer in the refcnt field.
path: Objects/object.c
user: login: vstinner
body: ```suggestion
    /* Store the delete_later pointer in the refcnt field.
```
created_at: 2025-04-15T14:02:49Z
author_association: MEMBER

PR comments:
user: login: nascheme
created_at: 2025-04-08T19:54:05Z
body: This is a good idea, IMHO.  I had a similar PR but Mark's is more elegant than mine.  
author_association: MEMBER
user: login: bedevere-bot
created_at: 2025-04-09T12:28:52Z
body: :robot: New build scheduled with the buildbot fleet by @markshannon for commit 8aa20f2358de1385c85a5a6d31a9b81ddda23374 :robot:

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F132280%2Fmerge

If you want to schedule another build, you need to add the <kbd>:hammer: test-with-buildbots</kbd> label again.

author_association: NONE
user: login: markshannon
created_at: 2025-04-09T13:55:49Z
body: Initial [performance numbers](https://github.com/faster-cpython/benchmarking-public/tree/main/results/bm-20250409-3.14.0a7+-8aa20f2) are a bit down. 0.2% - 1.6% slower.

I'm testing another branch that adds `PyThreadState *` as a parameter to `_PyDealloc`, which might reduce the impact.

author_association: MEMBER
user: login: markshannon
created_at: 2025-04-10T08:58:39Z
body: With faster recursion checks, [performance](https://github.com/faster-cpython/benchmarking-public/tree/main/results/bm-20250409-3.14.0a7%2B-3514aa1) is mixed but no worse overall:

* Linux ARM: 0.4% slower
* Linux x86: 1.5% slower
* Windows x86: 0.9% faster
* Mac (ARM): 3.8% faster

Although the 1.5% slowdown on linux x86 is concerning.

I also tried passing in the thread state as a parameter to `_Py_Dealloc` which is a much larger change. [Results](https://github.com/faster-cpython/benchmarking-public/tree/main/results/bm-20250409-3.14.0a7%2B-49ac4ce) were a bit worse than this PR:

* Linux ARM: no change
* Linux x86: 1.3% slower
* Windows x86: 1.3% slower
* Mac (ARM): 3.9% faster


author_association: MEMBER
user: login: iritkatriel
created_at: 2025-04-10T10:42:48Z
body: Maybe perf will be better with just one additional if for a decref that is not a container. So you have an implementation of decref for containers, and do one if to decide whether to apply it or stay with the current one.
author_association: MEMBER
user: login: markshannon
created_at: 2025-04-11T14:29:45Z
body: > Maybe perf will be better with just one additional if for a decref that is not a container. So you have an implementation of decref for containers, and do one if to decide whether to apply it or stay with the current one.

Do you mean a test in `Py_DECREF` or in `Py_Dealloc`?
We really want to avoid adding more branches to `Py_DECREF` as `Py_DECREF` is inlined in so many places.

Adding a branch in `Py_Dealloc` would save the stack checks, but those are cheap. Getting the "margin" is a load, subtract and shift.

Testing for collections is more expensive. `PyObject_IS_GC` does three loads, two of which are dependent on the first.


author_association: MEMBER
user: login: iritkatriel
created_at: 2025-04-11T15:18:27Z
body: > > Maybe perf will be better with just one additional if for a decref that is not a container. So you have an implementation of decref for containers, and do one if to decide whether to apply it or stay with the current one.
> 
> Do you mean a test in `Py_DECREF` or in `Py_Dealloc`? 

Sorry, yeah, dealloc. Maybe not worth it.
author_association: MEMBER
user: login: bedevere-bot
created_at: 2025-04-30T10:43:39Z
body: :warning::warning::warning: Buildbot failure :warning::warning::warning:
------------------------------------------------------------------------

Hi! The buildbot **s390x RHEL8 3.x** (tier-3) has failed when building commit 44e4c479fbf2c28605bd39303b1ce484753f6177.

What do you need to do:

1. Don't panic.
2. Check [the buildbot page in the devguide](https://devguide.python.org/buildbots/) if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/509/builds/9127) and take a look at the build logs.
4. Check if the failure is related to this commit (44e4c479fbf2c28605bd39303b1ce484753f6177) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/509/builds/9127

Failed tests:

- test_functools

Failed subtests:

- test_async_global_awaited_by - test.test_external_inspection.TestGetStackTrace.test_async_global_awaited_by


Summary of the results of the build (if available):

== 

<details>
<summary>Click to see traceback logs</summary>

```python-traceback
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x/build/Lib/test/test_external_inspection.py", line 531, in test_async_global_awaited_by
    self.assertEqual([[['echo_client_spam'], 'echo client spam', [[['main'], 'Task-1', []]]]], entries[-1][1])
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Lists differ: [[['echo_client_spam'], 'echo client spam', [[['main'], 'Task-1', []]]]] != []

```

</details>

author_association: NONE
user: login: vstinner
created_at: 2025-04-30T10:50:02Z
body: > Hi! The buildbot s390x RHEL8 3.x (tier-3) has failed when building commit https://github.com/python/cpython/commit/44e4c479fbf2c28605bd39303b1ce484753f6177.

```
test_recursive_pickle (test.test_functools.TestPartialPy.test_recursive_pickle) ...

Objects/classobject.c:247: _PyObject_GC_UNTRACK: Assertion "_PyObject_GC_IS_TRACKED(((PyObject*)(op)))" failed: object not tracked by the garbage collector
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x3ffae27c8f0
object refcount : 0
object type     : 0x1669768
object type name: method
object repr     : <refcnt 0 at 0x3ffae27c8f0>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0x000003ffb6177270 [python] (most recent call first):
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x/build/Lib/test/test_functools.py", line 420 in test_recursive_pickle
  (...)
```
author_association: MEMBER
user: login: vstinner
created_at: 2025-04-30T11:04:16Z
body: I wrote https://github.com/python/cpython/pull/133199 to fix test_functools.
author_association: MEMBER