Repository: pybind/pybind11
Pull request #5666: feat: Embeded sub-interpreters
Author: login: b-pass
Created: 2025-05-17T00:57:06Z | Merged: 2025-05-20T22:17:49Z

<!--
Title (above): please place [branch_name] at the beginning if you are targeting a branch other than master. *Do not target stable*.
It is recommended to use conventional commit format, see conventionalcommits.org, but not required.
-->
## Description

<!-- Include relevant issues or PRs here, describe what changed and why -->
This PR adds a header file, `subinterpreter.h` which provides utility classes for embedded Python to create, use, and destroy sub-interpreters.  The PR includes documentation rewrite of the "Embedded Sub-interpreter support" section.  The PR also splits out the multiple interpreter tests from `test_embed/test_interpreter.cpp` into their own file in that directory, updates them to use the new API, and adds new tests to cover the API and some interesting aspects of subinterpreters.

### API

Everything is in `subinterpreter.h` because this feature requires `PYBIND11_SUBINTERPRETER_SUPPORT`, which is 3.12+ (and some other caveats).  Including this file in non-supported versions produces a `#error`. There are no notable API outside of this new file, except for a change to embedded modules which was split out into its own PR: #5665 .  This PR currently includes the `embed.h` changes from that PR because this PR would not compile without them, and so this PR depends on that one being merged first (and when it is, I will fix this one with a rebase).

Class `subinterpreter` manages the lifetime of a sub-interpreter. Default constructing a `subinterpreter` does not create a `subinterpreter` it creates an empty wrapper, this is done to make the objects easier to move around and to prevent accidentally creating them (as creating them has to acquire and release the GIL).

To create a sub-interpreter, one does: `py::subinterpreter si = subinterpreter::create()`, similar to `py::initialize_interpreter()` for the main interpreter.  I didn't provide a `py::subinterpreter::destroy(...)` since that doesn't seem to make a lot of sense (you instead destruct the `subinterpreter` object returned by `create`) but if we want one to mirror `py::finalize_interpreter()` then I could add one easily.

`subinterpreter` also includes some useful utilities: `current()` to get a non-owning `subinterpreter` for the currently active interpreter (even the main one), `main()`  to get a non-owning `subinterpreter` for the main interpreter (even if it is not the current one), `id()` to get the ID number, and `state_dict()` to get the interpreter state dict.

Class `subinterpreter_scoped_activate` is the RAII wrapper, similar to `gil_scoped_acquire` which acquires the sub-interpreter's GIL and makes it the active interpreter.  The similarity to what the gil RAII classes do is also why it is named that.

Class `scoped_subinterpreter` is also provided in order to mirror the main interpreter's `scoped_interpreter`.

## Suggested changelog entry:

<!-- Fill in the below block with the expected RestructuredText entry. Delete if no entry needed;
     but do not delete header or rst block if an entry is needed! Will be collected via a script. -->

* Added API in `pybind11/subinterpreter.h` for embedding sub-intepreters (requires Python 3.12+).


<!-- readthedocs-preview pybind11 start -->
----
📚 Documentation preview 📚: https://pybind11--5666.org.readthedocs.build/

<!-- readthedocs-preview pybind11 end -->


Diff:
diff --git a/.github/workflows/format.yml b/.github/workflows/format.yml
index 3d6d1e39fd..bc2db70ea2 100644
--- a/.github/workflows/format.yml
+++ b/.github/workflows/format.yml
@@ -55,3 +55,6 @@ jobs:
 
     - name: Build
       run: cmake --build build -j 2 -- --keep-going
+
+    - name: Embedded
+      run: cmake --build build -t cpptest -j 2 -- --keep-going
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 676fc4b66c..347f816354 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -223,6 +223,7 @@ set(PYBIND11_HEADERS
     include/pybind11/operators.h
     include/pybind11/pybind11.h
     include/pybind11/pytypes.h
+    include/pybind11/subinterpreter.h
     include/pybind11/stl.h
     include/pybind11/stl_bind.h
     include/pybind11/stl/filesystem.h
diff --git a/docs/advanced/embedding.rst b/docs/advanced/embedding.rst
index dec767aac9..3ac0579385 100644
--- a/docs/advanced/embedding.rst
+++ b/docs/advanced/embedding.rst
@@ -237,31 +237,259 @@ global data. All the details can be found in the CPython documentation.
 
     Creating two concurrent ``scoped_interpreter`` guards is a fatal error. So is
     calling ``initialize_interpreter`` for a second time after the interpreter
-    has already been initialized.
+    has already been initialized. Use :class:`scoped_subinterpreter` to create
+    a sub-interpreter.  See :ref:`subinterp` for important details on sub-interpreters.
 
     Do not use the raw CPython API functions ``Py_Initialize`` and
     ``Py_Finalize`` as these do not properly handle the lifetime of
     pybind11's internal data.
 
 
-Sub-interpreter support
-=======================
+.. _subinterp:
+
+Embedding Sub-interpreters
+==========================
+
+A sub-interpreter is a separate interpreter instance which provides a
+separate, isolated interpreter environment within the same process as the main
+interpreter.  Sub-interpreters are created and managed with a separate API from
+the main interpreter. Beginning in Python 3.12, sub-interpreters each have
+their own Global Interpreter Lock (GIL), which means that running a
+sub-interpreter in a separate thread from the main interpreter can achieve true
+concurrency.
+
+pybind11's sub-interpreter API can be found in ``pybind11/subinterpreter.h``.
+
+pybind11 :class:`subinterpreter` instances can be safely moved and shared between
+threads as needed. However, managing multiple threads and the lifetimes of multiple
+interpreters and their GILs can be challenging.
+Proceed with caution (and lots of testing)!
+
+The main interpreter must be initialized before creating a sub-interpreter, and
+the main interpreter must outlive all sub-interpreters. Sub-interpreters are
+managed through a different API than the main interpreter.
+
+The :class:`subinterpreter` class manages the lifetime of sub-interpreters.
+Instances are movable, but not copyable. Default constructing this class does
+*not* create a sub-interpreter (it creates an empty holder).  To create a
+sub-interpreter, call :func:`subinterpreter::create()`.
+
+.. warning::
+
+    Sub-interpreter creation acquires (and subsequently releases) the main
+    interpreter GIL. If another thread holds the main GIL, the function will
+    block until the main GIL can be acquired.
+
+    Sub-interpreter destruction temporarily activates the sub-interpreter. The
+    sub-interpreter must not be active (on any threads) at the time the
+    :class:`subinterpreter` destructor is called.
+
+    Both actions will re-acquire any interpreter's GIL that was held prior to
+    the call before returning (or return to no active interpreter if none was
+    active at the time of the call).
+
+Each sub-interpreter will import a separate copy of each ``PYBIND11_EMBEDDED_MODULE``
+when those modules specify a ``multiple_interpreters`` tag. If a module does not
+specify a ``multiple_interpreters`` tag, then Python will report an ``ImportError``
+if it is imported in a sub-interpreter.
+
+pybind11 also has a :class:`scoped_subinterpreter` class, which creates and
+activates a sub-interpreter when it is constructed, and deactivates and deletes
+it when it goes out of scope.
+
+Activating a Sub-interpreter
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Once a sub-interpreter is created, you can "activate" it on a thread (and
+acquire its GIL) by creating a :class:`subinterpreter_scoped_activate`
+instance and passing it the sub-intepreter to be activated.  The function
+will acquire the sub-interpreter's GIL and make the sub-interpreter the
+current active interpreter on the current thread for the lifetime of the
+instance. When the :class:`subinterpreter_scoped_activate` instance goes out
+of scope, the sub-interpreter GIL is released and the prior interpreter that
+was active on the thread (if any) is reactivated and it's GIL is re-acquired.
+
+When using ``subinterpreter_scoped_activate``:
+
+1. If the thread holds any interpreter's GIL:
+   - That GIL is released
+2. The new sub-interpreter's GIL is acquired
+3. The new sub-interpreter is made active.
+4. When the scope ends:
+    - The sub-interpreter's GIL is released
+    - If there was a previous interpreter:
+        - The old interpreter's GIL is re-acquired
+        - The old interpreter is made active
+    - Otherwise, no interpreter is currently active and no GIL is held.
+
+Example:
+
+.. code-block:: cpp
+
+    py::initialize_interpreter();
+    // Main GIL is held
+    {
+        py::subinterpreter sub = py::subinterpreter::create();
+        // Main interpreter is still active, main GIL re-acquired
+        {
+            py::subinterpreter_scoped_activate guard(sub);
+            // Sub-interpreter active, thread holds sub's GIL
+            {
+                py::subinterpreter_scoped_activate main_guard(py);
+                // Sub's GIL was automatically released
+                // Main interpreter active, thread holds main's GIL
+            }
+            // Back to sub-interpreter, thread holds sub's GIL again
+        }
+        // Main interpreter is active, main's GIL is held
+    }
+
+
+GIL API for sub-interpreters
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+:class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be used to
+manage the GIL of a sub-interpreter just as they do for the main interpreter.
+They both manage the GIL of the currently active interpreter, without the
+programmer having to do anything special or different. There is one important
+caveat:
+
+.. note::
+
+    When no interpreter is active through a
+    :class:`subinterpreter_scoped_activate` instance (such as on a new thread),
+    :class:`gil_scoped_acquire` will acquire the **main** GIL and
+    activate the **main** interpreter.
+
+
+Full Sub-interpreter example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Here is an example showing how to create and activate sub-interpreters:
+
+.. code-block:: cpp
+
+    #include <iostream>
+    #include <pybind11/embed.h>
+    #include <pybind11/subinterpreter.h>
+
+    namespace py = pybind11;
+
+    PYBIND11_EMBEDDED_MODULE(printer, m, py::multiple_interpreters::per_interpreter_gil()) {
+        m.def("which", [](const std::string& when) {
+            std::cout << when << "; Current Interpreter is "
+                    << py::subinterpreter::current().id()
+                    << std::endl;
+        });
+    }
+
+    int main() {
+        py::scoped_interpreter main_interp;
+
+        py::module_::import("printer").attr("which")("First init");
+
+        {
+            py::subinterpreter sub = py::subinterpreter::create();
+
+            py::module_::import("printer").attr("which")("Created sub");
+
+            {
+                py::subinterpreter_scoped_activate guard(sub);
+                try {
+                    py::module_::import("printer").attr("which")("Activated sub");
+                }
+                catch (py::error_already_set &e) {
+                    std::cerr << "EXCEPTION " << e.what() << std::endl;
+                    return 1;
+                }
+            }
+
+            py::module_::import("printer").attr("which")("Deactivated sub");
+
+            {
+                py::gil_scoped_release nogil;
+                {
+                    py::subinterpreter_scoped_activate guard(sub);
+                    try {
+                        {
+                            py::subinterpreter_scoped_activate main_guard(py::subinterpreter::main());
+                            try {
+                                py::module_::import("printer").attr("which")("Main within sub");
+                            }
+                            catch (py::error_already_set &e) {
+                                std::cerr << "EXCEPTION " << e.what() << std::endl;
+                                return 1;
+                            }
+                        }
+                        py::module_::import("printer").attr("which")("After Main, still within sub");
+                    }
+                    catch (py::error_already_set &e) {
+                        std::cerr << "EXCEPTION " << e.what() << std::endl;
+                        return 1;
+                    }
+                }
+            }
+        }
+
+        py::module_::import("printer").attr("which")("At end");
+
+        return 0;
+    }
+
+Expected output:
+
+.. code-block:: text
+
+    First init; Current Interpreter is 0
+    Created sub; Current Interpreter is 0
+    Activated sub; Current Interpreter is 1
+    Deactivated sub; Current Interpreter is 0
+    Main within sub; Current Interpreter is 0
+    After Main, still within sub; Current Interpreter is 1
+    At end; Current Interpreter is 0
+
+.. warning::
+
+    In Python 3.12 sub-interpreters must be destroyed in the same OS thread
+    that created them.  Failure to follow this rule may result in deadlocks
+    or crashes when destroying the sub-interpreter on the wrong thread.
+
+    This constraint is not present in Python 3.13+.
+
+
+Best Practices for sub-interpreter safety
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+- Never share Python objects across different interpreters.
+
+- :class:`error_already_set` objects contain a reference to the Python exception type,
+  and :func:`error_already_set::what()` acquires the GIL. So Python exceptions must
+  **never** be allowed to propagate past the enclosing
+  :class:`subinterpreter_scoped_activate` instance!
+  (So your try/catch should be *just inside* the scope covered by the
+  :class:`subinterpreter_scoped_activate`.)
+
+- Avoid global/static state whenever possible. Instead, keep state within each interpreter,
+  such as within the interpreter state dict, which can be accessed via
+  ``subinterpreter::current().state_dict()``, or within instance members and tied to
+  Python objects.
+
+- Avoid trying to "cache" Python objects in C++ variables across function calls (this is an easy
+  way to accidentally introduce sub-interpreter bugs). In the code example above, note that we
+  did not save the result of :func:`module_::import`, in order to avoid accidentally using the
+  resulting Python object when the wrong interpreter was active.
 
-Creating multiple copies of ``scoped_interpreter`` is not possible because it
-represents the main Python interpreter. Sub-interpreters are something different
-and they do permit the existence of multiple interpreters. This is an advanced
-feature of the CPython API and should be handled with care. pybind11 does not
-currently offer a C++ interface for sub-interpreters, so refer to the CPython
-documentation for all the details regarding this feature.
+- Avoid moving or disarming RAII objects managing GIL and sub-interpreter lifetimes. Doing so can
+  lead to confusion about lifetimes.  (For example, accidentally extending a
+  :class:`subinterpreter_scoped_activate` past the lifetime of it's :class:`subinterpreter`.)
 
-We'll just mention a couple of caveats the sub-interpreters support in pybind11:
+- While sub-interpreters each have their own GIL, there can now be multiple independent GILs in one
+  program so you need to consider the possibility of deadlocks caused by multiple GILs and/or the
+  interactions of the GIL(s) and your C++ code's own locking.
 
- 1. Sub-interpreters will not receive independent copies of embedded modules.
-    Instead, these are shared and modifications in one interpreter may be
-    reflected in another.
+- When using multiple threads to run independent sub-interpreters, the independent GILs allow
+  concurrent calls from different interpreters into the same C++ code from different threads.
+  So you must still consider the thread safety of your C++ code.  Remember, in Python 3.12
+  sub-interpreters must be destroyed on the same thread that they were created on.
 
- 2. Managing multiple threads, multiple interpreters and the GIL can be
-    challenging and there are several caveats here, even within the pure
-    CPython API (please refer to the Python docs for details). As for
-    pybind11, keep in mind that ``gil_scoped_release`` and ``gil_scoped_acquire``
-    do not take sub-interpreters into account.
+- Familiarize yourself with :ref:`misc_concurrency`.
diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst
index 7d2a279585..b8cb1923e9 100644
--- a/docs/advanced/misc.rst
+++ b/docs/advanced/misc.rst
@@ -228,6 +228,8 @@ You can explicitly disable sub-interpreter support in your module by using the
 :func:`multiple_interpreters::not_supported()` tag. This is the default behavior if you do not
 specify a multiple_interpreters tag.
 
+.. _misc_concurrency:
+
 Concurrency and Parallelism in Python with pybind11
 ===================================================
 
diff --git a/include/pybind11/gil.h b/include/pybind11/gil.h
index 1a9bfeaddf..e90ea41528 100644
--- a/include/pybind11/gil.h
+++ b/include/pybind11/gil.h
@@ -130,7 +130,7 @@ class gil_scoped_acquire {
     }
 
     /// This method will disable the PyThreadState_DeleteCurrent call and the
-    /// GIL won't be acquired. This method should be used if the interpreter
+    /// GIL won't be released. This method should be used if the interpreter
     /// could be shutting down when this is called, as thread deletion is not
     /// allowed during shutdown. Check _Py_IsFinalizing() on Python 3.7+, and
     /// protect subsequent code.
diff --git a/include/pybind11/subinterpreter.h b/include/pybind11/subinterpreter.h
new file mode 100644
index 0000000000..306e47bc6a
--- /dev/null
+++ b/include/pybind11/subinterpreter.h
@@ -0,0 +1,320 @@
+/*
+    pybind11/subinterpreter.h: Support for creating and using subinterpreters
+
+    Copyright (c) 2025 The Pybind Development Team.
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "detail/common.h"
+#include "detail/internals.h"
+#include "gil.h"
+
+#include <stdexcept>
+
+#if !defined(PYBIND11_SUBINTERPRETER_SUPPORT)
+#    error "This platform does not support subinterpreters, do not include this file."
+#endif
+
+PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+PYBIND11_NAMESPACE_BEGIN(detail)
+PyInterpreterState *get_interpreter_state_unchecked() {
+    auto cur_tstate = get_thread_state_unchecked();
+    if (cur_tstate)
+        return cur_tstate->interp;
+    else
+        return nullptr;
+}
+PYBIND11_NAMESPACE_END(detail)
+
+class subinterpreter;
+
+/// Activate the subinterpreter and acquire its GIL, while also releasing any GIL and interpreter
+/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and its
+/// associated GIL are restored to their state as they were before the scope was entered.
+class subinterpreter_scoped_activate {
+public:
+    explicit subinterpreter_scoped_activate(subinterpreter const &si);
+    ~subinterpreter_scoped_activate();
+
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate &&) = delete;
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate const &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate const &) = delete;
+
+private:
+    PyThreadState *old_tstate_ = nullptr;
+    PyThreadState *tstate_ = nullptr;
+    PyGILState_STATE gil_state_;
+    bool simple_gil_ = false;
+};
+
+/// Holds a Python subinterpreter instance
+class subinterpreter {
+public:
+    /// empty/unusable, but move-assignable.  use create() to create a subinterpreter.
+    subinterpreter() = default;
+
+    subinterpreter(subinterpreter const &copy) = delete;
+    subinterpreter &operator=(subinterpreter const &copy) = delete;
+
+    subinterpreter(subinterpreter &&old) noexcept
+        : istate_(old.istate_), creation_tstate_(old.creation_tstate_) {
+        old.istate_ = nullptr;
+        old.creation_tstate_ = nullptr;
+    }
+
+    subinterpreter &operator=(subinterpreter &&old) noexcept {
+        std::swap(old.istate_, istate_);
+        std::swap(old.creation_tstate_, creation_tstate_);
+        return *this;
+    }
+
+    /// Create a new subinterpreter with the specified configuration
+    /// @note This function acquires (and then releases) the main interpreter GIL, but the main
+    /// interpreter and its GIL are not required to be held prior to calling this function.
+    static inline subinterpreter create(PyInterpreterConfig const &cfg) {
+        error_scope err_scope;
+        subinterpreter result;
+        {
+            // we must hold the main GIL in order to create a subinterpreter
+            subinterpreter_scoped_activate main_guard(main());
+
+            auto prev_tstate = PyThreadState_Get();
+
+            auto status = Py_NewInterpreterFromConfig(&result.creation_tstate_, &cfg);
+
+            // this doesn't raise a normal Python exception, it provides an exit() status code.
+            if (PyStatus_Exception(status)) {
+                pybind11_fail("failed to create new sub-interpreter");
+            }
+
+            // upon success, the new interpreter is activated in this thread
+            result.istate_ = result.creation_tstate_->interp;
+            detail::get_num_interpreters_seen() += 1; // there are now many interpreters
+            detail::get_internals(); // initialize internals.tstate, amongst other things...
+
+            // In 3.13+ this state should be deleted right away, and the memory will be reused for
+            // the next threadstate on this interpreter. However, on 3.12 we cannot do that, we
+            // must keep it around (but not use it) ... see destructor.
+#if PY_VERSION_HEX >= 0x030D0000
+            PyThreadState_Clear(result.creation_tstate_);
+            PyThreadState_DeleteCurrent();
+#endif
+
+            // we have to switch back to main, and then the scopes will handle cleanup
+            PyThreadState_Swap(prev_tstate);
+        }
+        return result;
+    }
+
+    /// Calls create() with a default configuration of an isolated interpreter that disallows fork,
+    /// exec, and Python threads.
+    static inline subinterpreter create() {
+        // same as the default config in the python docs
+        PyInterpreterConfig cfg;
+        std::memset(&cfg, 0, sizeof(cfg));
+        cfg.check_multi_interp_extensions = 1;
+        cfg.gil = PyInterpreterConfig_OWN_GIL;
+        return create(cfg);
+    }
+
+    ~subinterpreter() {
+        if (!creation_tstate_) {
+            // non-owning wrapper, do nothing.
+            return;
+        }
+
+        PyThreadState *destroy_tstate;
+        PyThreadState *old_tstate;
+
+        // Python 3.12 requires us to keep the original PyThreadState alive until we are ready to
+        // destroy the interpreter.  We prefer to use that to destroy the interpreter.
+#if PY_VERSION_HEX < 0x030D0000
+        // The tstate passed to Py_EndInterpreter MUST have been created on the current OS thread.
+        bool same_thread = false;
+#    ifdef PY_HAVE_THREAD_NATIVE_ID
+        same_thread = PyThread_get_thread_native_id() == creation_tstate_->native_thread_id;
+#    endif
+        if (same_thread) {
+            // OK it is safe to use the creation state here
+            destroy_tstate = creation_tstate_;
+            old_tstate = PyThreadState_Swap(destroy_tstate);
+        } else {
+            // We have to make a new tstate on this thread and use that.
+            destroy_tstate = PyThreadState_New(istate_);
+            old_tstate = PyThreadState_Swap(destroy_tstate);
+
+            // We can use the one we just created, so we must delete the creation state.
+            PyThreadState_Clear(creation_tstate_);
+            PyThreadState_Delete(creation_tstate_);
+        }
+#else
+        destroy_tstate = PyThreadState_New(istate_);
+        old_tstate = PyThreadState_Swap(destroy_tstate);
+#endif
+
+        bool switch_back = old_tstate && old_tstate->interp != istate_;
+
+        // Get the internals pointer (without creating it if it doesn't exist).  It's possible
+        // for the internals to be created during Py_EndInterpreter() (e.g. if a py::capsule
+        // calls `get_internals()` during destruction), so we get the pointer-pointer here and
+        // check it after.
+        auto *&internals_ptr_ptr = detail::get_internals_pp<detail::internals>();
+        auto *&local_internals_ptr_ptr = detail::get_internals_pp<detail::local_internals>();
+        {
+            dict sd = state_dict();
+            internals_ptr_ptr
+                = detail::get_internals_pp_from_capsule_in_state_dict<detail::internals>(
+                    sd, PYBIND11_INTERNALS_ID);
+            local_internals_ptr_ptr
+                = detail::get_internals_pp_from_capsule_in_state_dict<detail::local_internals>(
+                    sd, detail::get_local_internals_id());
+        }
+
+        // End it
+        Py_EndInterpreter(destroy_tstate);
+
+        // do NOT decrease detail::get_num_interpreters_seen, because it can never decrease
+        // while other threads are running...
+
+        if (internals_ptr_ptr) {
+            internals_ptr_ptr->reset();
+        }
+        if (local_internals_ptr_ptr) {
+            local_internals_ptr_ptr->reset();
+        }
+
+        // switch back to the old tstate and old GIL (if there was one)
+        if (switch_back)
+            PyThreadState_Swap(old_tstate);
+    }
+
+    /// Get a handle to the main interpreter that can be used with subinterpreter_scoped_activate
+    /// Note that destructing the handle is a noop, the main interpreter can only be ended by
+    /// py::finalize_interpreter()
+    static subinterpreter main() {
+        subinterpreter m;
+        m.istate_ = PyInterpreterState_Main();
+        m.disarm(); // make destruct a noop
+        return m;
+    }
+
+    /// Get a non-owning wrapper of the currently active interpreter (if any)
+    static subinterpreter current() {
+        subinterpreter c;
+        c.istate_ = detail::get_interpreter_state_unchecked();
+        c.disarm(); // make destruct a noop, we don't own this...
+        return c;
+    }
+
+    /// Get the numerical identifier for the sub-interpreter
+    int64_t id() const {
+        if (istate_ != nullptr)
+            return PyInterpreterState_GetID(istate_);
+        else
+            return -1; // CPython uses one-up numbers from 0, so negative should be safe to return
+                       // here.
+    }
+
+    /// Get the interpreter's state dict.  This interpreter's GIL must be held before calling!
+    dict state_dict() { return reinterpret_borrow<dict>(PyInterpreterState_GetDict(istate_)); }
+
+    /// abandon cleanup of this subinterpreter (leak it). this might be needed during
+    /// finalization...
+    void disarm() { creation_tstate_ = nullptr; }
+
+    /// An empty wrapper cannot be activated
+    bool empty() const { return istate_ == nullptr; }
+
+    /// Is this wrapper non-empty
+    explicit operator bool() const { return !empty(); }
+
+private:
+    friend class subinterpreter_scoped_activate;
+    PyInterpreterState *istate_ = nullptr;
+    PyThreadState *creation_tstate_ = nullptr;
+};
+
+class scoped_subinterpreter {
+public:
+    scoped_subinterpreter() : si_(subinterpreter::create()), scope_(si_) {}
+
+    explicit scoped_subinterpreter(PyInterpreterConfig const &cfg)
+        : si_(subinterpreter::create(cfg)), scope_(si_) {}
+
+private:
+    subinterpreter si_;
+    subinterpreter_scoped_activate scope_;
+};
+
+inline subinterpreter_scoped_activate::subinterpreter_scoped_activate(subinterpreter const &si) {
+    if (!si.istate_) {
+        pybind11_fail("null subinterpreter");
+    }
+
+    if (detail::get_interpreter_state_unchecked() == si.istate_) {
+        // we are already on this interpreter, make sure we hold the GIL
+        simple_gil_ = true;
+        gil_state_ = PyGILState_Ensure();
+        return;
+    }
+
+    // we can't really interact with the interpreter at all until we switch to it
+    // not even to, for example, look in its state dict or touch its internals
+    tstate_ = PyThreadState_New(si.istate_);
+
+    // make the interpreter active and acquire the GIL
+    old_tstate_ = PyThreadState_Swap(tstate_);
+
+    // save this in internals for scoped_gil calls
+    PYBIND11_TLS_REPLACE_VALUE(detail::get_internals().tstate, tstate_);
+}
+
+inline subinterpreter_scoped_activate::~subinterpreter_scoped_activate() {
+    if (simple_gil_) {
+        // We were on this interpreter already, so just make sure the GIL goes back as it was
+        PyGILState_Release(gil_state_);
+    } else {
+#if defined(PYBIND11_DETAILED_ERROR_MESSAGES)
+        bool has_active_exception;
+#    if defined(__cpp_lib_uncaught_exceptions)
+        has_active_exception = std::uncaught_exceptions() > 0;
+#    else
+        // removed in C++20, replaced with uncaught_exceptions
+        has_active_exception = std::uncaught_exception();
+#    endif
+        if (has_active_exception) {
+            try {
+                std::rethrow_exception(std::current_exception());
+            } catch (error_already_set &) {
+                // Because error_already_set holds python objects and what() acquires the GIL, it
+                // is basically never OK to let these exceptions propagate outside the current
+                // active interpreter.
+                pybind11_fail("~subinterpreter_scoped_activate: cannot propagate Python "
+                              "exceptions outside of their owning interpreter");
+            } catch (...) {
+            }
+        }
+#endif
+
+        if (tstate_) {
+#if defined(PYBIND11_DETAILED_ERROR_MESSAGES)
+            if (detail::get_thread_state_unchecked() != tstate_) {
+                pybind11_fail("~subinterpreter_scoped_activate: thread state must be current!");
+            }
+#endif
+            PYBIND11_TLS_DELETE_VALUE(detail::get_internals().tstate);
+            PyThreadState_Clear(tstate_);
+            PyThreadState_DeleteCurrent();
+        }
+
+        // Go back the previous interpreter (if any) and acquire THAT gil
+        PyThreadState_Swap(old_tstate_);
+    }
+}
+
+PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE)
diff --git a/tests/extra_python_package/test_files.py b/tests/extra_python_package/test_files.py
index 2919cf3ea3..9725bedae1 100644
--- a/tests/extra_python_package/test_files.py
+++ b/tests/extra_python_package/test_files.py
@@ -58,6 +58,7 @@
     "include/pybind11/options.h",
     "include/pybind11/pybind11.h",
     "include/pybind11/pytypes.h",
+    "include/pybind11/subinterpreter.h",
     "include/pybind11/stl.h",
     "include/pybind11/stl_bind.h",
     "include/pybind11/trampoline_self_life_support.h",
diff --git a/tests/test_embed/CMakeLists.txt b/tests/test_embed/CMakeLists.txt
index 1a537a6580..af3c848ad4 100644
--- a/tests/test_embed/CMakeLists.txt
+++ b/tests/test_embed/CMakeLists.txt
@@ -28,7 +28,7 @@ endif()
 
 find_package(Threads REQUIRED)
 
-add_executable(test_embed catch.cpp test_interpreter.cpp)
+add_executable(test_embed catch.cpp test_interpreter.cpp test_subinterpreter.cpp)
 pybind11_enable_warnings(test_embed)
 
 target_link_libraries(test_embed PRIVATE pybind11::embed Catch2::Catch2 Threads::Threads)
diff --git a/tests/test_embed/test_interpreter.cpp b/tests/test_embed/test_interpreter.cpp
index 6e4be7378a..e555c0d70c 100644
--- a/tests/test_embed/test_interpreter.cpp
+++ b/tests/test_embed/test_interpreter.cpp
@@ -19,6 +19,21 @@ size_t get_sys_path_size() {
     return py::len(sys_path);
 }
 
+bool has_state_dict_internals_obj() {
+    py::dict state = py::detail::get_python_state_dict();
+    return state.contains(PYBIND11_INTERNALS_ID);
+}
+
+bool has_pybind11_internals_static() {
+    auto *&ipp = py::detail::get_internals_pp<py::detail::internals>();
+    return (ipp != nullptr) && *ipp;
+}
+
+uintptr_t get_details_as_uintptr() {
+    return reinterpret_cast<uintptr_t>(
+        py::detail::get_internals_pp<py::detail::internals>()->get());
+}
+
 class Widget {
 public:
     explicit Widget(std::string message) : message(std::move(message)) {}
@@ -258,21 +273,6 @@ TEST_CASE("Add program dir to path using PyConfig") {
 }
 #endif
 
-bool has_state_dict_internals_obj() {
-    py::dict state = py::detail::get_python_state_dict();
-    return state.contains(PYBIND11_INTERNALS_ID);
-}
-
-bool has_pybind11_internals_static() {
-    auto *&ipp = py::detail::get_internals_pp<py::detail::internals>();
-    return (ipp != nullptr) && *ipp;
-}
-
-uintptr_t get_details_as_uintptr() {
-    return reinterpret_cast<uintptr_t>(
-        py::detail::get_internals_pp<py::detail::internals>()->get());
-}
-
 TEST_CASE("Restart the interpreter") {
     // Verify pre-restart state.
     REQUIRE(py::module_::import("widget_module").attr("add")(1, 2).cast<int>() == 3);
@@ -336,279 +336,6 @@ TEST_CASE("Restart the interpreter") {
     REQUIRE(py_widget.attr("the_message").cast<std::string>() == "Hello after restart");
 }
 
-#if defined(PYBIND11_SUBINTERPRETER_SUPPORT)
-TEST_CASE("Subinterpreter") {
-    py::module_::import("external_module"); // in the main interpreter
-
-    // Add tags to the modules in the main interpreter and test the basics.
-    py::module_::import("__main__").attr("main_tag") = "main interpreter";
-    {
-        auto m = py::module_::import("widget_module");
-        m.attr("extension_module_tag") = "added to module in main interpreter";
-
-        REQUIRE(m.attr("add")(1, 2).cast<int>() == 3);
-    }
-
-    auto main_int
-        = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
-
-    REQUIRE(has_state_dict_internals_obj());
-    REQUIRE(has_pybind11_internals_static());
-
-    /// Create and switch to a subinterpreter.
-    auto *main_tstate = PyThreadState_Get();
-    auto *sub_tstate = Py_NewInterpreter();
-
-    py::detail::get_num_interpreters_seen()++;
-
-    // Subinterpreters get their own copy of builtins.
-    REQUIRE_FALSE(has_state_dict_internals_obj());
-
-    // internals hasn't been populated yet, but will be different for the subinterpreter
-    REQUIRE_FALSE(has_pybind11_internals_static());
-
-    py::list(py::module_::import("sys").attr("path")).append(py::str("."));
-
-    auto ext_int = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
-    py::detail::get_internals();
-    REQUIRE(has_pybind11_internals_static());
-    REQUIRE(get_details_as_uintptr() == ext_int);
-    REQUIRE(main_int != ext_int);
-
-    // Modules tags should be gone.
-    REQUIRE_FALSE(py::hasattr(py::module_::import("__main__"), "tag"));
-    {
-        REQUIRE_NOTHROW(py::module_::import("widget_module"));
-        auto m = py::module_::import("widget_module");
-        REQUIRE_FALSE(py::hasattr(m, "extension_module_tag"));
-
-        // Function bindings should still work.
-        REQUIRE(m.attr("add")(1, 2).cast<int>() == 3);
-    }
-
-    // The subinterpreter now has internals populated since we imported a pybind11 module
-    REQUIRE(has_pybind11_internals_static());
-
-    // Restore main interpreter.
-    Py_EndInterpreter(sub_tstate);
-    py::detail::get_num_interpreters_seen() = 1;
-    PyThreadState_Swap(main_tstate);
-
-    REQUIRE(py::hasattr(py::module_::import("__main__"), "main_tag"));
-    REQUIRE(py::hasattr(py::module_::import("widget_module"), "extension_module_tag"));
-    REQUIRE(has_state_dict_internals_obj());
-}
-
-TEST_CASE("Multiple Subinterpreters") {
-    // Make sure the module is in the main interpreter and save its pointer
-    auto *main_ext = py::module_::import("external_module").ptr();
-    auto main_int
-        = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
-    py::module_::import("external_module").attr("multi_interp") = "1";
-
-    auto *main_tstate = PyThreadState_Get();
-
-    /// Create and switch to a subinterpreter.
-    auto *sub1_tstate = Py_NewInterpreter();
-    py::detail::get_num_interpreters_seen()++;
-
-    py::list(py::module_::import("sys").attr("path")).append(py::str("."));
-
-    // The subinterpreter has its own copy of this module which is completely separate from main
-    auto *sub1_ext = py::module_::import("external_module").ptr();
-    REQUIRE(sub1_ext != main_ext);
-    REQUIRE_FALSE(py::hasattr(py::module_::import("external_module"), "multi_interp"));
-    py::module_::import("external_module").attr("multi_interp") = "2";
-    // The subinterpreter also has its own internals
-    auto sub1_int
-        = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
-    REQUIRE(sub1_int != main_int);
-
-    // Create another interpreter
-    auto *sub2_tstate = Py_NewInterpreter();
-    py::detail::get_num_interpreters_seen()++;
-
-    py::list(py::module_::import("sys").attr("path")).append(py::str("."));
-
-    // The second subinterpreter is separate from both main and the other subinterpreter
-    auto *sub2_ext = py::module_::import("external_module").ptr();
-    REQUIRE(sub2_ext != main_ext);
-    REQUIRE(sub2_ext != sub1_ext);
-    REQUIRE_FALSE(py::hasattr(py::module_::import("external_module"), "multi_interp"));
-    py::module_::import("external_module").attr("multi_interp") = "3";
-    // The subinterpreter also has its own internals
-    auto sub2_int
-        = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
-    REQUIRE(sub2_int != main_int);
-    REQUIRE(sub2_int != sub1_int);
-
-    PyThreadState_Swap(sub1_tstate); // go back to sub1
-
-    REQUIRE(py::cast<std::string>(py::module_::import("external_module").attr("multi_interp"))
-            == "2");
-
-    PyThreadState_Swap(main_tstate); // go back to main
-
-    auto post_int
-        = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
-    // Make sure internals went back the way it was before
-    REQUIRE(main_int == post_int);
-
-    REQUIRE(py::cast<std::string>(py::module_::import("external_module").attr("multi_interp"))
-            == "1");
-
-    PyThreadState_Swap(sub1_tstate);
-    Py_EndInterpreter(sub1_tstate);
-    PyThreadState_Swap(sub2_tstate);
-    Py_EndInterpreter(sub2_tstate);
-
-    py::detail::get_num_interpreters_seen() = 1;
-    PyThreadState_Swap(main_tstate);
-}
-#endif
-
-#if defined(Py_MOD_PER_INTERPRETER_GIL_SUPPORTED) && defined(PYBIND11_SUBINTERPRETER_SUPPORT)
-TEST_CASE("Per-Subinterpreter GIL") {
-    auto main_int
-        = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
-
-    std::atomic<int> started, sync, failure;
-    started = 0;
-    sync = 0;
-    failure = 0;
-
-// REQUIRE throws on failure, so we can't use it within the thread
-#    define T_REQUIRE(status)                                                                     \
-        do {                                                                                      \
-            assert(status);                                                                       \
-            if (!(status))                                                                        \
-                ++failure;                                                                        \
-        } while (0)
-
-    auto &&thread_main = [&](int num) {
-        while (started == 0)
-            std::this_thread::sleep_for(std::chrono::microseconds(1));
-        ++started;
-
-        py::gil_scoped_acquire gil;
-        auto main_tstate = PyThreadState_Get();
-
-        // we have the GIL, we can access the main interpreter
-        auto t_int
-            = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
-        T_REQUIRE(t_int == main_int);
-        py::module_::import("external_module").attr("multi_interp") = "1";
-
-        PyThreadState *sub = nullptr;
-        PyInterpreterConfig cfg;
-        memset(&cfg, 0, sizeof(cfg));
-        cfg.check_multi_interp_extensions = 1;
-        cfg.gil = PyInterpreterConfig_OWN_GIL;
-        auto status = Py_NewInterpreterFromConfig(&sub, &cfg);
-        T_REQUIRE(!PyStatus_IsError(status));
-
-        py::detail::get_num_interpreters_seen()++;
-
-        py::list(py::module_::import("sys").attr("path")).append(py::str("."));
-
-        // we have switched to the new interpreter and released the main gil
-
-        // trampoline_module did not provide the per_interpreter_gil tag, so it cannot be
-        // imported
-        bool caught = false;
-        try {
-            py::module_::import("trampoline_module");
-        } catch (pybind11::error_already_set &pe) {
-            T_REQUIRE(pe.matches(PyExc_ImportError));
-            std::string msg(pe.what());
-            T_REQUIRE(msg.find("does not support loading in subinterpreters")
-                      != std::string::npos);
-            caught = true;
-        }
-        T_REQUIRE(caught);
-
-        // widget_module did provide the per_interpreter_gil tag, so it this does not throw
-        py::module_::import("widget_module");
-
-        T_REQUIRE(!py::hasattr(py::module_::import("external_module"), "multi_interp"));
-        py::module_::import("external_module").attr("multi_interp") = std::to_string(num);
-
-        // wait for something to set sync to our thread number
-        // we are holding our subinterpreter's GIL
-        while (sync != num)
-            std::this_thread::sleep_for(std::chrono::microseconds(1));
-
-        // now change it so the next thread can mvoe on
-        ++sync;
-
-        // but keep holding the GIL until after the next thread moves on as well
-        while (sync == num + 1)
-            std::this_thread::sleep_for(std::chrono::microseconds(1));
-
-        // one last check before quitting the thread, the internals should be different
-        auto sub_int
-            = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
-        T_REQUIRE(sub_int != main_int);
-
-        Py_EndInterpreter(sub);
-
-        // switch back so the scoped_acquire can release the GIL properly
-        PyThreadState_Swap(main_tstate);
-    };
-
-    std::thread t1(thread_main, 1);
-    std::thread t2(thread_main, 2);
-
-    // we spawned two threads, at this point they are both waiting for started to increase
-    ++started;
-
-    // ok now wait for the threads to start
-    while (started != 3)
-        std::this_thread::sleep_for(std::chrono::microseconds(1));
-
-    // we still hold the main GIL, at this point both threads are waiting on the main GIL
-    // IN THE CASE of free threading, the threads are waiting on sync (because there is no GIL)
-
-    // IF the below code hangs in one of the wait loops, then the child thread GIL behavior did not
-    // function as expected.
-    {
-        // release the GIL and allow the threads to run
-        py::gil_scoped_release nogil;
-
-        // the threads are now waiting on the sync
-        REQUIRE(sync == 0);
-
-        // this will trigger thread 1 and then advance and trigger 2 and then advance
-        sync = 1;
-
-        // wait for thread 2 to advance
-        while (sync != 3)
-            std::this_thread::sleep_for(std::chrono::microseconds(1));
-
-        // we know now that thread 1 has run and may be finishing
-        // and thread 2 is waiting for permission to advance
-
-        // so we move sync so that thread 2 can finish executing
-        ++sync;
-
-        // now wait for both threads to complete
-        t1.join();
-        t2.join();
-    }
-
-    // now we have the gil again, sanity check
-    REQUIRE(py::cast<std::string>(py::module_::import("external_module").attr("multi_interp"))
-            == "1");
-
-    // the threads are stopped. we can now lower this for the rest of the test
-    py::detail::get_num_interpreters_seen() = 1;
-
-    // make sure nothing unexpected happened inside the threads, now that they are completed
-    REQUIRE(failure == 0);
-#    undef T_REQUIRE
-}
-#endif
-
 TEST_CASE("Execution frame") {
     // When the interpreter is embedded, there is no execution frame, but `py::exec`
     // should still function by using reasonable globals: `__main__.__dict__`.
diff --git a/tests/test_embed/test_subinterpreter.cpp b/tests/test_embed/test_subinterpreter.cpp
new file mode 100644
index 0000000000..9d7d88b8a0
--- /dev/null
+++ b/tests/test_embed/test_subinterpreter.cpp
@@ -0,0 +1,431 @@
+#include <pybind11/embed.h>
+#ifdef PYBIND11_SUBINTERPRETER_SUPPORT
+#    include <pybind11/subinterpreter.h>
+
+// Silence MSVC C++17 deprecation warning from Catch regarding std::uncaught_exceptions (up to
+// catch 2.0.1; this should be fixed in the next catch release after 2.0.1).
+PYBIND11_WARNING_DISABLE_MSVC(4996)
+
+#    include <catch.hpp>
+#    include <cstdlib>
+#    include <fstream>
+#    include <functional>
+#    include <thread>
+#    include <utility>
+
+namespace py = pybind11;
+using namespace py::literals;
+
+bool has_state_dict_internals_obj();
+bool has_pybind11_internals_static();
+uintptr_t get_details_as_uintptr();
+
+void unsafe_reset_internals_for_single_interpreter() {
+    // unsafe normally, but for subsequent tests, put this back.. we know there are no threads
+    // running and only 1 interpreter
+    py::detail::get_num_interpreters_seen() = 1;
+    py::detail::get_internals_pp<py::detail::internals>() = nullptr;
+    py::detail::get_internals();
+    py::detail::get_internals_pp<py::detail::local_internals>() = nullptr;
+    py::detail::get_local_internals();
+}
+
+TEST_CASE("Single Subinterpreter") {
+    py::module_::import("external_module"); // in the main interpreter
+
+    // Add tags to the modules in the main interpreter and test the basics.
+    py::module_::import("__main__").attr("main_tag") = "main interpreter";
+    {
+        auto m = py::module_::import("widget_module");
+        m.attr("extension_module_tag") = "added to module in main interpreter";
+
+        REQUIRE(m.attr("add")(1, 2).cast<int>() == 3);
+    }
+    REQUIRE(has_state_dict_internals_obj());
+    REQUIRE(has_pybind11_internals_static());
+
+    auto main_int
+        = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
+
+    /// Create and switch to a subinterpreter.
+    {
+        py::scoped_subinterpreter ssi;
+
+        // The subinterpreter has internals populated
+        REQUIRE(has_pybind11_internals_static());
+
+        py::list(py::module_::import("sys").attr("path")).append(py::str("."));
+
+        auto ext_int
+            = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
+        py::detail::get_internals();
+        REQUIRE(has_pybind11_internals_static());
+        REQUIRE(get_details_as_uintptr() == ext_int);
+        REQUIRE(ext_int != main_int);
+
+        // Modules tags should be gone.
+        REQUIRE_FALSE(py::hasattr(py::module_::import("__main__"), "tag"));
+        {
+            auto m = py::module_::import("widget_module");
+            REQUIRE_FALSE(py::hasattr(m, "extension_module_tag"));
+
+            // Function bindings should still work.
+            REQUIRE(m.attr("add")(1, 2).cast<int>() == 3);
+        }
+    }
+
+    REQUIRE(py::hasattr(py::module_::import("__main__"), "main_tag"));
+    REQUIRE(py::hasattr(py::module_::import("widget_module"), "extension_module_tag"));
+    REQUIRE(has_state_dict_internals_obj());
+
+    unsafe_reset_internals_for_single_interpreter();
+}
+
+#    if PY_VERSION_HEX >= 0x030D0000
+TEST_CASE("Move Subinterpreter") {
+    std::unique_ptr<py::subinterpreter> sub(new py::subinterpreter(py::subinterpreter::create()));
+
+    // on this thread, use the subinterpreter and import some non-trivial junk
+    {
+        py::subinterpreter_scoped_activate activate(*sub);
+
+        py::list(py::module_::import("sys").attr("path")).append(py::str("."));
+        py::module_::import("datetime");
+        py::module_::import("threading");
+        py::module_::import("external_module");
+    }
+
+    std::thread([&]() {
+        // Use it again
+        {
+            py::subinterpreter_scoped_activate activate(*sub);
+            py::module_::import("external_module");
+        }
+        sub.reset();
+    }).join();
+
+    REQUIRE(!sub);
+
+    unsafe_reset_internals_for_single_interpreter();
+}
+#    endif
+
+TEST_CASE("GIL Subinterpreter") {
+
+    PyInterpreterState *main_interp = PyInterpreterState_Get();
+
+    {
+        auto sub = py::subinterpreter::create();
+
+        REQUIRE(main_interp == PyInterpreterState_Get());
+
+        PyInterpreterState *sub_interp = nullptr;
+
+        {
+            py::subinterpreter_scoped_activate activate(sub);
+
+            sub_interp = PyInterpreterState_Get();
+            REQUIRE(sub_interp != main_interp);
+
+            py::list(py::module_::import("sys").attr("path")).append(py::str("."));
+            py::module_::import("datetime");
+            py::module_::import("threading");
+            py::module_::import("external_module");
+
+            {
+                py::subinterpreter_scoped_activate main(py::subinterpreter::main());
+                REQUIRE(PyInterpreterState_Get() == main_interp);
+
+                {
+                    py::gil_scoped_release nogil{};
+                    {
+                        py::gil_scoped_acquire yesgil{};
+                        REQUIRE(PyInterpreterState_Get() == main_interp);
+                    }
+                }
+
+                REQUIRE(PyInterpreterState_Get() == main_interp);
+            }
+
+            REQUIRE(PyInterpreterState_Get() == sub_interp);
+
+            {
+                py::gil_scoped_release nogil{};
+                {
+                    py::gil_scoped_acquire yesgil{};
+                    REQUIRE(PyInterpreterState_Get() == sub_interp);
+                }
+            }
+
+            REQUIRE(PyInterpreterState_Get() == sub_interp);
+        }
+
+        REQUIRE(PyInterpreterState_Get() == main_interp);
+
+        {
+            py::gil_scoped_release nogil{};
+            {
+                py::gil_scoped_acquire yesgil{};
+                REQUIRE(PyInterpreterState_Get() == main_interp);
+            }
+        }
+
+        REQUIRE(PyInterpreterState_Get() == main_interp);
+
+        bool thread_result;
+
+        {
+            thread_result = false;
+            py::gil_scoped_release nogil{};
+            std::thread([&]() {
+                {
+                    py::subinterpreter_scoped_activate ssa{sub};
+                }
+                {
+                    py::gil_scoped_acquire gil{};
+                    thread_result = (PyInterpreterState_Get() == main_interp);
+                }
+            }).join();
+        }
+        REQUIRE(thread_result);
+
+        {
+            thread_result = false;
+            py::gil_scoped_release nogil{};
+            std::thread([&]() {
+                py::gil_scoped_acquire gil{};
+                thread_result = (PyInterpreterState_Get() == main_interp);
+            }).join();
+        }
+        REQUIRE(thread_result);
+    }
+
+    REQUIRE(PyInterpreterState_Get() == main_interp);
+    unsafe_reset_internals_for_single_interpreter();
+}
+
+TEST_CASE("Multiple Subinterpreters") {
+    // Make sure the module is in the main interpreter and save its pointer
+    auto *main_ext = py::module_::import("external_module").ptr();
+    auto main_int
+        = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
+    py::module_::import("external_module").attr("multi_interp") = "1";
+
+    {
+        py::subinterpreter si1 = py::subinterpreter::create();
+        std::unique_ptr<py::subinterpreter> psi2;
+
+        PyObject *sub1_ext = nullptr;
+        PyObject *sub2_ext = nullptr;
+        uintptr_t sub1_int = 0;
+        uintptr_t sub2_int = 0;
+
+        {
+            py::subinterpreter_scoped_activate scoped(si1);
+            py::list(py::module_::import("sys").attr("path")).append(py::str("."));
+
+            // The subinterpreter has its own copy of this module which is completely separate from
+            // main
+            sub1_ext = py::module_::import("external_module").ptr();
+            REQUIRE(sub1_ext != main_ext);
+            REQUIRE_FALSE(py::hasattr(py::module_::import("external_module"), "multi_interp"));
+            py::module_::import("external_module").attr("multi_interp") = "2";
+            // The subinterpreter also has its own internals
+            sub1_int
+                = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
+            REQUIRE(sub1_int != main_int);
+
+            // while the old one is active, create a new one
+            psi2.reset(new py::subinterpreter(py::subinterpreter::create()));
+        }
+
+        {
+            py::subinterpreter_scoped_activate scoped(*psi2);
+            py::list(py::module_::import("sys").attr("path")).append(py::str("."));
+
+            // The second subinterpreter is separate from both main and the other subinterpreter
+            sub2_ext = py::module_::import("external_module").ptr();
+            REQUIRE(sub2_ext != main_ext);
+            REQUIRE(sub2_ext != sub1_ext);
+            REQUIRE_FALSE(py::hasattr(py::module_::import("external_module"), "multi_interp"));
+            py::module_::import("external_module").attr("multi_interp") = "3";
+            // The subinterpreter also has its own internals
+            sub2_int
+                = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
+            REQUIRE(sub2_int != main_int);
+            REQUIRE(sub2_int != sub1_int);
+        }
+
+        {
+            py::subinterpreter_scoped_activate scoped(si1);
+            REQUIRE(
+                py::cast<std::string>(py::module_::import("external_module").attr("multi_interp"))
+                == "2");
+        }
+
+        // out here we should be in the main interpreter, with the GIL, with the other 2 still
+        // alive
+
+        auto post_int
+            = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
+        // Make sure internals went back the way it was before
+        REQUIRE(main_int == post_int);
+
+        REQUIRE(py::cast<std::string>(py::module_::import("external_module").attr("multi_interp"))
+                == "1");
+    }
+
+    // now back to just main
+
+    auto post_int
+        = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
+    // Make sure internals went back the way it was before
+    REQUIRE(main_int == post_int);
+
+    REQUIRE(py::cast<std::string>(py::module_::import("external_module").attr("multi_interp"))
+            == "1");
+
+    unsafe_reset_internals_for_single_interpreter();
+}
+
+#    ifdef Py_MOD_PER_INTERPRETER_GIL_SUPPORTED
+TEST_CASE("Per-Subinterpreter GIL") {
+    auto main_int
+        = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
+
+    std::atomic<int> started, sync, failure;
+    started = 0;
+    sync = 0;
+    failure = 0;
+
+// REQUIRE throws on failure, so we can't use it within the thread
+#        define T_REQUIRE(status)                                                                 \
+            do {                                                                                  \
+                assert(status);                                                                   \
+                if (!(status))                                                                    \
+                    ++failure;                                                                    \
+            } while (0)
+
+    auto &&thread_main = [&](int num) {
+        while (started == 0)
+            std::this_thread::sleep_for(std::chrono::microseconds(1));
+        ++started;
+
+        py::gil_scoped_acquire gil;
+
+        // we have the GIL, we can access the main interpreter
+        auto t_int
+            = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
+        T_REQUIRE(t_int == main_int);
+        py::module_::import("external_module").attr("multi_interp") = "1";
+
+        auto sub = py::subinterpreter::create();
+
+        {
+            py::subinterpreter_scoped_activate sguard{sub};
+
+            py::list(py::module_::import("sys").attr("path")).append(py::str("."));
+
+            // we have switched to the new interpreter and released the main gil
+
+            // trampoline_module did not provide the per_interpreter_gil tag, so it cannot be
+            // imported
+            bool caught = false;
+            try {
+                py::module_::import("trampoline_module");
+            } catch (pybind11::error_already_set &pe) {
+                T_REQUIRE(pe.matches(PyExc_ImportError));
+                std::string msg(pe.what());
+                T_REQUIRE(msg.find("does not support loading in subinterpreters")
+                          != std::string::npos);
+                caught = true;
+            }
+            T_REQUIRE(caught);
+
+            // widget_module did provide the per_interpreter_gil tag, so it this does not throw
+            try {
+                py::module_::import("widget_module");
+                caught = false;
+            } catch (pybind11::error_already_set &) {
+                caught = true;
+            }
+            T_REQUIRE(!caught);
+
+            // widget_module did provide the per_interpreter_gil tag, so it this does not throw
+            py::module_::import("widget_module");
+
+            T_REQUIRE(!py::hasattr(py::module_::import("external_module"), "multi_interp"));
+            py::module_::import("external_module").attr("multi_interp") = std::to_string(num);
+
+            // wait for something to set sync to our thread number
+            // we are holding our subinterpreter's GIL
+            while (sync != num)
+                std::this_thread::sleep_for(std::chrono::microseconds(1));
+
+            // now change it so the next thread can move on
+            ++sync;
+
+            // but keep holding the GIL until after the next thread moves on as well
+            while (sync == num + 1)
+                std::this_thread::sleep_for(std::chrono::microseconds(1));
+
+            // one last check before quitting the thread, the internals should be different
+            auto sub_int
+                = py::module_::import("external_module").attr("internals_at")().cast<uintptr_t>();
+            T_REQUIRE(sub_int != main_int);
+        }
+    };
+#        undef T_REQUIRE
+
+    std::thread t1(thread_main, 1);
+    std::thread t2(thread_main, 2);
+
+    // we spawned two threads, at this point they are both waiting for started to increase
+    ++started;
+
+    // ok now wait for the threads to start
+    while (started != 3)
+        std::this_thread::sleep_for(std::chrono::microseconds(1));
+
+    // we still hold the main GIL, at this point both threads are waiting on the main GIL
+    // IN THE CASE of free threading, the threads are waiting on sync (because there is no GIL)
+
+    // IF the below code hangs in one of the wait loops, then the child thread GIL behavior did not
+    // function as expected.
+    {
+        // release the GIL and allow the threads to run
+        py::gil_scoped_release nogil;
+
+        // the threads are now waiting on the sync
+        REQUIRE(sync == 0);
+
+        // this will trigger thread 1 and then advance and trigger 2 and then advance
+        sync = 1;
+
+        // wait for thread 2 to advance
+        while (sync != 3)
+            std::this_thread::sleep_for(std::chrono::microseconds(1));
+
+        // we know now that thread 1 has run and may be finishing
+        // and thread 2 is waiting for permission to advance
+
+        // so we move sync so that thread 2 can finish executing
+        ++sync;
+
+        // now wait for both threads to complete
+        t1.join();
+        t2.join();
+    }
+
+    // now we have the gil again, sanity check
+    REQUIRE(py::cast<std::string>(py::module_::import("external_module").attr("multi_interp"))
+            == "1");
+
+    unsafe_reset_internals_for_single_interpreter();
+
+    // make sure nothing unexpected happened inside the threads, now that they are completed
+    REQUIRE(failure == 0);
+}
+#    endif // Py_MOD_PER_INTERPRETER_GIL_SUPPORTED
+
+#endif // PYBIND11_SUBINTERPRETER_SUPPORT


Reviews:
user: login: copilot-pull-request-reviewer[bot]
body: ## Pull Request Overview

This PR introduces support for Python sub-interpreters in pybind11 by adding a dedicated API, updating documentation, and refactoring tests to exercise the new functionality.

- Added a new `pybind11/subinterpreter.h` header with RAII wrappers and utilities for sub-interpreters.
- Updated embedding documentation (`docs/advanced/embedding.rst`) to describe sub-interpreter API and best practices.
- Refactored existing interpreter tests: extracted sub-interpreter tests into `test_subinterpreter.cpp` and updated build configuration.

### Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

<details>
<summary>Show a summary per file</summary>

| File                                    | Description                                                      |
| --------------------------------------- | ---------------------------------------------------------------- |
| tests/test_embed/test_interpreter.cpp   | Removed embedded sub-interpreter tests and helper functions.     |
| tests/test_embed/CMakeLists.txt         | Added `test_subinterpreter.cpp` to the test executable.          |
| tests/extra_python_package/test_files.py| Included `include/pybind11/subinterpreter.h` in test package.    |
| include/pybind11/subinterpreter.h       | New API for creating, using, and destroying Python sub-interpreters. |
| include/pybind11/gil.h                  | Clarified shutdown comment in `gil_scoped_acquire`.              |
| docs/advanced/misc.rst                  | Added an anchor for concurrency section.                         |
| docs/advanced/embedding.rst             | Rewrote embedding guide to cover sub-interpreter usage and safety. |
| CMakeLists.txt                          | Installed `subinterpreter.h` alongside other headers.            |
</details>



<details>
<summary>Comments suppressed due to low confidence (1)</summary>

**docs/advanced/embedding.rst:484**
* Use “its” instead of “it's” in documentation.
```
:class:`subinterpreter_scoped_activate` past the lifetime of it's :class:`subinterpreter`)
```
</details>


state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T03:21:26Z
user: login: Skylion007
body: Some comments
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-20T16:05:20Z
user: login: henryiii
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-20T16:22:45Z
user: login: henryiii
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-20T16:30:18Z

Review comments:
diff_hunk: @@ -0,0 +1,320 @@
+/*
+    pybind11/subinterpreter.h: Support for creating and using subinterpreters
+
+    Copyright (c) 2025 The Pybind Development Team.
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "detail/common.h"
+#include "detail/internals.h"
+#include "gil.h"
+
+#include <stdexcept>
+
+#if !defined(PYBIND11_SUBINTERPRETER_SUPPORT)
+#    error "This platform does not support subinterpreters, do not include this file."
+#endif
+
+PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+PYBIND11_NAMESPACE_BEGIN(detail)
+PyInterpreterState *get_interpreter_state_unchecked() {
+    auto cur_tstate = get_thread_state_unchecked();
+    if (cur_tstate)
+        return cur_tstate->interp;
+    else
+        return nullptr;
+}
+PYBIND11_NAMESPACE_END(detail)
+
+class subinterpreter;
+
+/// Activate the subinterpreter and acquire it's GIL, while also releasing any GIL and interpreter
+/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and it's
path: include/pybind11/subinterpreter.h
user: login: Copilot
body: Replace “it's” with the correct possessive “its” in the comment.
```suggestion
/// Activate the subinterpreter and acquire its GIL, while also releasing any GIL and interpreter
/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and its
```
created_at: 2025-05-20T03:21:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,320 @@
+/*
+    pybind11/subinterpreter.h: Support for creating and using subinterpreters
+
+    Copyright (c) 2025 The Pybind Development Team.
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "detail/common.h"
+#include "detail/internals.h"
+#include "gil.h"
+
+#include <stdexcept>
+
+#if !defined(PYBIND11_SUBINTERPRETER_SUPPORT)
+#    error "This platform does not support subinterpreters, do not include this file."
+#endif
+
+PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+PYBIND11_NAMESPACE_BEGIN(detail)
+PyInterpreterState *get_interpreter_state_unchecked() {
+    auto cur_tstate = get_thread_state_unchecked();
+    if (cur_tstate)
+        return cur_tstate->interp;
+    else
+        return nullptr;
+}
+PYBIND11_NAMESPACE_END(detail)
+
+class subinterpreter;
+
+/// Activate the subinterpreter and acquire it's GIL, while also releasing any GIL and interpreter
+/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and it's
+/// associated GIL are restored to their state as they were before the scope was entered.
+class subinterpreter_scoped_activate {
+public:
+    explicit subinterpreter_scoped_activate(subinterpreter const &si);
+    ~subinterpreter_scoped_activate();
+
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate &&) = delete;
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate const &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate const &) = delete;
+
+private:
+    PyThreadState *old_tstate_ = nullptr;
+    PyThreadState *tstate_ = nullptr;
+    PyGILState_STATE gil_state_;
+    bool simple_gil_ = false;
+};
+
+/// Holds a Python subinterpreter instance
+class subinterpreter {
+public:
+    /// empty/unusable, but move-assignable.  use create() to create a subinterpreter.
+    subinterpreter() = default;
+
+    subinterpreter(subinterpreter const &copy) = delete;
+    subinterpreter &operator=(subinterpreter const &copy) = delete;
+
+    subinterpreter(subinterpreter &&old)
+        : istate_(old.istate_), creation_tstate_(old.creation_tstate_) {
+        old.istate_ = nullptr;
+        old.creation_tstate_ = nullptr;
+    }
+
+    subinterpreter &operator=(subinterpreter &&old) {
+        std::swap(old.istate_, istate_);
+        std::swap(old.creation_tstate_, creation_tstate_);
+        return *this;
+    }
+
+    /// Create a new subinterpreter with the specified configuration
+    /// @note This function acquires (and then releases) the main interpreter GIL, but the main
+    /// interpreter and its GIL are not required to be held prior to calling this function.
+    static inline subinterpreter create(PyInterpreterConfig const &cfg) {
+        error_scope err_scope;
+        subinterpreter result;
+        {
+            // we must hold the main GIL in order to create a subinterpreter
+            subinterpreter_scoped_activate main_guard(main());
+
+            auto prev_tstate = PyThreadState_Get();
+
+            auto status = Py_NewInterpreterFromConfig(&result.creation_tstate_, &cfg);
+
+            // this doesn't raise a normal Python exception, it provides an exit() status code.
+            if (PyStatus_Exception(status)) {
+                pybind11_fail("failed to create new sub-interpreter");
+            }
+
+            // upon success, the new interpreter is activated in this thread
+            result.istate_ = result.creation_tstate_->interp;
+            detail::get_num_interpreters_seen() += 1; // there are now many interpreters
+            detail::get_internals(); // initialize internals.tstate, amongst other things...
+
+            // In 3.13+ this state should be deleted right away, and the memory will be reused for
+            // the next threadstate on this interpreter. However, on 3.12 we cannot do that, we
+            // must keep it around (but not use it) ... see destructor.
+#if PY_VERSION_HEX >= 0x030D0000
+            PyThreadState_Clear(result.creation_tstate_);
+            PyThreadState_DeleteCurrent();
+#endif
+
+            // we have to switch back to main, and then the scopes will handle cleanup
+            PyThreadState_Swap(prev_tstate);
+        }
+        return result;
+    }
+
+    /// Calls create() with a default configuration of an isolated interpreter that disallows fork,
+    /// exec, and Python threads.
+    static inline subinterpreter create() {
+        // same as the default config in the python docs
+        PyInterpreterConfig cfg;
+        memset(&cfg, 0, sizeof(cfg));
+        cfg.check_multi_interp_extensions = 1;
+        cfg.gil = PyInterpreterConfig_OWN_GIL;
+        return create(cfg);
+    }
+
+    ~subinterpreter() {
+        if (!creation_tstate_) {
+            // non-owning wrapper, do nothing.
+            return;
+        }
+
+        PyThreadState *destroy_tstate;
+        PyThreadState *old_tstate;
+
+        // Python 3.12 requires us to keep the original PyThreadState alive until we are ready to
+        // destroy the interpreter.  We prefer to use that to destroy the interpreter.
+#if PY_VERSION_HEX < 0x030D0000
+        // The tstate passed to Py_EndInterpreter MUST have been created on the current OS thread.
+        bool same_thread = false;
+#    ifdef PY_HAVE_THREAD_NATIVE_ID
+        same_thread = PyThread_get_thread_native_id() == creation_tstate_->native_thread_id;
+#    endif
+        if (same_thread) {
+            // OK it is safe to use the creation state here
+            destroy_tstate = creation_tstate_;
+            old_tstate = PyThreadState_Swap(destroy_tstate);
+        } else {
+            // We have to make a new tstate on this thread and use that.
+            destroy_tstate = PyThreadState_New(istate_);
+            old_tstate = PyThreadState_Swap(destroy_tstate);
+
+            // We can use the one we just created, so we must delete the creation state.
+            PyThreadState_Clear(creation_tstate_);
+            PyThreadState_Delete(creation_tstate_);
+        }
+#else
+        destroy_tstate = PyThreadState_New(istate_);
+        old_tstate = PyThreadState_Swap(destroy_tstate);
+#endif
+
+        bool switch_back = old_tstate && old_tstate->interp != istate_;
+
+        // Get the internals pointer (without creating it if it doesn't exist).  It's possible
+        // for the internals to be created during Py_EndInterpreter() (e.g. if a py::capsule
+        // calls `get_internals()` during destruction), so we get the pointer-pointer here and
+        // check it after.
+        auto *&internals_ptr_ptr = detail::get_internals_pp<detail::internals>();
+        auto *&local_internals_ptr_ptr = detail::get_internals_pp<detail::local_internals>();
+        {
+            dict sd = state_dict();
+            internals_ptr_ptr
+                = detail::get_internals_pp_from_capsule_in_state_dict<detail::internals>(
+                    sd, PYBIND11_INTERNALS_ID);
+            local_internals_ptr_ptr
+                = detail::get_internals_pp_from_capsule_in_state_dict<detail::local_internals>(
+                    sd, detail::get_local_internals_id());
+        }
+
+        // End it
+        Py_EndInterpreter(destroy_tstate);
+
+        // do NOT decrease detail::get_num_interpreters_seen, because it can never decrease
+        // while other threads are running...
+
+        if (internals_ptr_ptr) {
+            internals_ptr_ptr->reset();
+        }
+        if (local_internals_ptr_ptr) {
+            local_internals_ptr_ptr->reset();
+        }
+
+        // switch back to the old tstate and old GIL (if there was one)
+        if (switch_back)
+            PyThreadState_Swap(old_tstate);
+    }
+
+    /// Get a handle to the main interpreter that can be used with subinterpreter_scoped_activate
+    /// Note that destructing the handle is a noop, the main interpreter can only be ended by
+    /// py::finalize_interpreter()
+    static subinterpreter main() {
+        subinterpreter m;
+        m.istate_ = PyInterpreterState_Main();
+        m.disarm(); // make destruct a noop
+        return m;
+    }
+
+    /// Get a non-owning wrapper of the currently active interpreter (if any)
+    static subinterpreter current() {
+        subinterpreter c;
+        c.istate_ = detail::get_interpreter_state_unchecked();
+        c.disarm(); // make destruct a noop, we don't own this...
+        return c;
+    }
+
+    /// Get the numerical identifier for the sub-interpreter
+    int64_t id() const {
+        if (istate_ != nullptr)
+            return PyInterpreterState_GetID(istate_);
+        else
+            return -1; // CPython uses one-up numbers from 0, so negative should be safe to return
+                       // here.
+    }
+
+    /// Get the interpreter's state dict.  This interpreter's GIL must be held before calling!
+    dict state_dict() { return reinterpret_borrow<dict>(PyInterpreterState_GetDict(istate_)); }
+
+    /// abandon cleanup of this subinterpreter (leak it). this might be needed during
+    /// finalization...
+    void disarm() { creation_tstate_ = nullptr; }
+
+    /// An empty wrapper cannot be activated
+    bool empty() const { return istate_ == nullptr; }
+
+    /// Is this wrapper non-empty
+    explicit operator bool() const { return !empty(); }
+
+private:
+    friend class subinterpreter_scoped_activate;
+    PyInterpreterState *istate_ = nullptr;
+    PyThreadState *creation_tstate_ = nullptr;
+};
+
+class scoped_subinterpreter {
+public:
+    scoped_subinterpreter() : si_(subinterpreter::create()), scope_(si_) {}
+
+    explicit scoped_subinterpreter(PyInterpreterConfig const &cfg)
+        : si_(subinterpreter::create(cfg)), scope_(si_) {}
+
+private:
+    subinterpreter si_;
+    subinterpreter_scoped_activate scope_;
+};
+
+inline subinterpreter_scoped_activate::subinterpreter_scoped_activate(subinterpreter const &si) {
+    if (!si.istate_) {
+        pybind11_fail("null subinterpreter");
+    }
+
+    if (detail::get_interpreter_state_unchecked() == si.istate_) {
+        // we are already on this interpreter, make sure we hold the GIL
+        simple_gil_ = true;
+        gil_state_ = PyGILState_Ensure();
+        return;
+    }
+
+    // we can't really innteract with the interpreter at all until we switch to it
path: include/pybind11/subinterpreter.h
user: login: Copilot
body: Correct typo: change “innteract” to “interact” in this comment.
```suggestion
    // we can't really interact with the interpreter at all until we switch to it
```
created_at: 2025-05-20T03:21:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,320 @@
+/*
+    pybind11/subinterpreter.h: Support for creating and using subinterpreters
+
+    Copyright (c) 2025 The Pybind Development Team.
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "detail/common.h"
+#include "detail/internals.h"
+#include "gil.h"
+
+#include <stdexcept>
+
+#if !defined(PYBIND11_SUBINTERPRETER_SUPPORT)
+#    error "This platform does not support subinterpreters, do not include this file."
+#endif
+
+PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+PYBIND11_NAMESPACE_BEGIN(detail)
+PyInterpreterState *get_interpreter_state_unchecked() {
+    auto cur_tstate = get_thread_state_unchecked();
+    if (cur_tstate)
+        return cur_tstate->interp;
+    else
+        return nullptr;
+}
+PYBIND11_NAMESPACE_END(detail)
+
+class subinterpreter;
+
+/// Activate the subinterpreter and acquire it's GIL, while also releasing any GIL and interpreter
+/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and it's
+/// associated GIL are restored to their state as they were before the scope was entered.
+class subinterpreter_scoped_activate {
+public:
+    explicit subinterpreter_scoped_activate(subinterpreter const &si);
+    ~subinterpreter_scoped_activate();
+
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate &&) = delete;
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate const &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate const &) = delete;
+
+private:
+    PyThreadState *old_tstate_ = nullptr;
+    PyThreadState *tstate_ = nullptr;
+    PyGILState_STATE gil_state_;
+    bool simple_gil_ = false;
+};
+
+/// Holds a Python subinterpreter instance
+class subinterpreter {
+public:
+    /// empty/unusable, but move-assignable.  use create() to create a subinterpreter.
+    subinterpreter() = default;
+
+    subinterpreter(subinterpreter const &copy) = delete;
+    subinterpreter &operator=(subinterpreter const &copy) = delete;
+
+    subinterpreter(subinterpreter &&old)
+        : istate_(old.istate_), creation_tstate_(old.creation_tstate_) {
+        old.istate_ = nullptr;
+        old.creation_tstate_ = nullptr;
+    }
+
+    subinterpreter &operator=(subinterpreter &&old) {
+        std::swap(old.istate_, istate_);
+        std::swap(old.creation_tstate_, creation_tstate_);
+        return *this;
+    }
+
+    /// Create a new subinterpreter with the specified configuration
+    /// @note This function acquires (and then releases) the main interpreter GIL, but the main
+    /// interpreter and its GIL are not required to be held prior to calling this function.
+    static inline subinterpreter create(PyInterpreterConfig const &cfg) {
+        error_scope err_scope;
+        subinterpreter result;
+        {
+            // we must hold the main GIL in order to create a subinterpreter
+            subinterpreter_scoped_activate main_guard(main());
+
+            auto prev_tstate = PyThreadState_Get();
+
+            auto status = Py_NewInterpreterFromConfig(&result.creation_tstate_, &cfg);
+
+            // this doesn't raise a normal Python exception, it provides an exit() status code.
+            if (PyStatus_Exception(status)) {
+                pybind11_fail("failed to create new sub-interpreter");
+            }
+
+            // upon success, the new interpreter is activated in this thread
+            result.istate_ = result.creation_tstate_->interp;
+            detail::get_num_interpreters_seen() += 1; // there are now many interpreters
+            detail::get_internals(); // initialize internals.tstate, amongst other things...
+
+            // In 3.13+ this state should be deleted right away, and the memory will be reused for
+            // the next threadstate on this interpreter. However, on 3.12 we cannot do that, we
+            // must keep it around (but not use it) ... see destructor.
+#if PY_VERSION_HEX >= 0x030D0000
+            PyThreadState_Clear(result.creation_tstate_);
+            PyThreadState_DeleteCurrent();
+#endif
+
+            // we have to switch back to main, and then the scopes will handle cleanup
+            PyThreadState_Swap(prev_tstate);
+        }
+        return result;
+    }
+
+    /// Calls create() with a default configuration of an isolated interpreter that disallows fork,
+    /// exec, and Python threads.
+    static inline subinterpreter create() {
+        // same as the default config in the python docs
+        PyInterpreterConfig cfg;
+        memset(&cfg, 0, sizeof(cfg));
+        cfg.check_multi_interp_extensions = 1;
+        cfg.gil = PyInterpreterConfig_OWN_GIL;
+        return create(cfg);
+    }
+
+    ~subinterpreter() {
+        if (!creation_tstate_) {
+            // non-owning wrapper, do nothing.
+            return;
+        }
+
+        PyThreadState *destroy_tstate;
+        PyThreadState *old_tstate;
+
+        // Python 3.12 requires us to keep the original PyThreadState alive until we are ready to
+        // destroy the interpreter.  We prefer to use that to destroy the interpreter.
+#if PY_VERSION_HEX < 0x030D0000
+        // The tstate passed to Py_EndInterpreter MUST have been created on the current OS thread.
+        bool same_thread = false;
+#    ifdef PY_HAVE_THREAD_NATIVE_ID
+        same_thread = PyThread_get_thread_native_id() == creation_tstate_->native_thread_id;
+#    endif
+        if (same_thread) {
+            // OK it is safe to use the creation state here
+            destroy_tstate = creation_tstate_;
+            old_tstate = PyThreadState_Swap(destroy_tstate);
+        } else {
+            // We have to make a new tstate on this thread and use that.
+            destroy_tstate = PyThreadState_New(istate_);
+            old_tstate = PyThreadState_Swap(destroy_tstate);
+
+            // We can use the one we just created, so we must delete the creation state.
+            PyThreadState_Clear(creation_tstate_);
+            PyThreadState_Delete(creation_tstate_);
+        }
+#else
+        destroy_tstate = PyThreadState_New(istate_);
+        old_tstate = PyThreadState_Swap(destroy_tstate);
+#endif
+
+        bool switch_back = old_tstate && old_tstate->interp != istate_;
+
+        // Get the internals pointer (without creating it if it doesn't exist).  It's possible
+        // for the internals to be created during Py_EndInterpreter() (e.g. if a py::capsule
+        // calls `get_internals()` during destruction), so we get the pointer-pointer here and
+        // check it after.
+        auto *&internals_ptr_ptr = detail::get_internals_pp<detail::internals>();
+        auto *&local_internals_ptr_ptr = detail::get_internals_pp<detail::local_internals>();
+        {
+            dict sd = state_dict();
+            internals_ptr_ptr
+                = detail::get_internals_pp_from_capsule_in_state_dict<detail::internals>(
+                    sd, PYBIND11_INTERNALS_ID);
+            local_internals_ptr_ptr
+                = detail::get_internals_pp_from_capsule_in_state_dict<detail::local_internals>(
+                    sd, detail::get_local_internals_id());
+        }
+
+        // End it
+        Py_EndInterpreter(destroy_tstate);
+
+        // do NOT decrease detail::get_num_interpreters_seen, because it can never decrease
+        // while other threads are running...
+
+        if (internals_ptr_ptr) {
+            internals_ptr_ptr->reset();
+        }
+        if (local_internals_ptr_ptr) {
+            local_internals_ptr_ptr->reset();
+        }
+
+        // switch back to the old tstate and old GIL (if there was one)
+        if (switch_back)
+            PyThreadState_Swap(old_tstate);
+    }
+
+    /// Get a handle to the main interpreter that can be used with subinterpreter_scoped_activate
+    /// Note that destructing the handle is a noop, the main interpreter can only be ended by
+    /// py::finalize_interpreter()
+    static subinterpreter main() {
+        subinterpreter m;
+        m.istate_ = PyInterpreterState_Main();
+        m.disarm(); // make destruct a noop
+        return m;
+    }
+
+    /// Get a non-owning wrapper of the currently active interpreter (if any)
+    static subinterpreter current() {
+        subinterpreter c;
+        c.istate_ = detail::get_interpreter_state_unchecked();
+        c.disarm(); // make destruct a noop, we don't own this...
+        return c;
+    }
+
+    /// Get the numerical identifier for the sub-interpreter
+    int64_t id() const {
+        if (istate_ != nullptr)
+            return PyInterpreterState_GetID(istate_);
+        else
+            return -1; // CPython uses one-up numbers from 0, so negative should be safe to return
+                       // here.
+    }
+
+    /// Get the interpreter's state dict.  This interpreter's GIL must be held before calling!
+    dict state_dict() { return reinterpret_borrow<dict>(PyInterpreterState_GetDict(istate_)); }
+
+    /// abandon cleanup of this subinterpreter (leak it). this might be needed during
+    /// finalization...
+    void disarm() { creation_tstate_ = nullptr; }
+
+    /// An empty wrapper cannot be activated
+    bool empty() const { return istate_ == nullptr; }
+
+    /// Is this wrapper non-empty
+    explicit operator bool() const { return !empty(); }
+
+private:
+    friend class subinterpreter_scoped_activate;
+    PyInterpreterState *istate_ = nullptr;
+    PyThreadState *creation_tstate_ = nullptr;
+};
+
+class scoped_subinterpreter {
+public:
+    scoped_subinterpreter() : si_(subinterpreter::create()), scope_(si_) {}
+
+    explicit scoped_subinterpreter(PyInterpreterConfig const &cfg)
+        : si_(subinterpreter::create(cfg)), scope_(si_) {}
+
+private:
+    subinterpreter si_;
+    subinterpreter_scoped_activate scope_;
+};
+
+inline subinterpreter_scoped_activate::subinterpreter_scoped_activate(subinterpreter const &si) {
+    if (!si.istate_) {
+        pybind11_fail("null subinterpreter");
+    }
+
+    if (detail::get_interpreter_state_unchecked() == si.istate_) {
+        // we are already on this interpreter, make sure we hold the GIL
+        simple_gil_ = true;
+        gil_state_ = PyGILState_Ensure();
+        return;
+    }
+
+    // we can't really innteract with the interpreter at all until we switch to it
+    // not even to, for example, look in it's state dict or touch its internals
path: include/pybind11/subinterpreter.h
user: login: Copilot
body: Use the possessive “its” instead of “it's” in the comment.
```suggestion
    // not even to, for example, look in its state dict or touch its internals
```
created_at: 2025-05-20T03:21:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -130,7 +130,7 @@ class gil_scoped_acquire {
     }
 
     /// This method will disable the PyThreadState_DeleteCurrent call and the
-    /// GIL won't be acquired. This method should be used if the interpreter
+    /// GIL won't be released. This method should be used if the interpreter
     /// could be shutting down when this is called, as thread deletion is not
     /// allowed during shutdown. Check _Py_IsFinalizing() on Python 3.7+, and
     /// protect subsequent code.
path: include/pybind11/gil.h
user: login: Copilot
body: [nitpick] Clarify whether the intention is that the GIL is not released or not acquired; reword to accurately describe the behavior.
created_at: 2025-05-20T03:21:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -237,31 +237,259 @@ global data. All the details can be found in the CPython documentation.
 
     Creating two concurrent ``scoped_interpreter`` guards is a fatal error. So is
     calling ``initialize_interpreter`` for a second time after the interpreter
-    has already been initialized.
+    has already been initialized. Use :class:`scoped_subinterpreter` to create
+    a sub-interpreter.  See :ref:`subinterp` for important details on sub-interpreters.
 
     Do not use the raw CPython API functions ``Py_Initialize`` and
     ``Py_Finalize`` as these do not properly handle the lifetime of
     pybind11's internal data.
 
 
-Sub-interpreter support
-=======================
+.. _subinterp:
+
+Embedding Sub-interpreters
+==========================
+
+A sub-interpreter is a separate interpreter instance which provides a
+separate, isolated interpreter environment within the same process as the main
+interpreter.  Sub-interpreters are created and managed with a separate API from
+the main interpreter. Beginning in Python 3.12, sub-interpreters each have
+their own Global Interpreter Lock (GIL), which means that running a
+sub-interpreter in a separate thread from the main interpreter can achieve true
+concurrency.
+
+pybind11's sub-interpreter API can be found in ``pybind11/subinterpreter.h``.
+
+pybind11 :class:`subinterpreter` instances can be safely moved and shared between
+threads as needed. However, managing multiple threads and the lifetimes of multiple
+interpreters and their GILs can be challenging.
+Proceed with caution (and lots of testing)!
+
+The main interpreter must be initialized before creating a sub-interpreter, and
+the main interpreter must outlive all sub-interpreters. Sub-interpreters are
+managed through a different API than the main interpreter.
+
+The :class:`subinterpreter` class manages the lifetime of sub-interpreters.
+Instances are movable, but not copyable. Default constructing this class does
+*not* create a sub-interpreter (it creates an empty holder).  To create a
+sub-interpreter, call :func:`subinterpreter::create()`.
+
+.. warning::
+
+    Sub-interpreter creation acquires (and subsequently releases) the main
+    interpreter GIL. If another thread holds the main GIL, the function will
+    block until the main GIL can be acquired.
+
+    Sub-interpreter destruction temporarily activates the sub-interpreter. The
+    sub-interpreter must not be active (on any threads) at the time the
+    :class:`subinterpreter` destructor is called.
+
+    Both actions will re-acquire any interpreter's GIL that was held prior to
+    the call before returning (or return to no active interpreter if none was
+    active at the time of the call).
+
+Each sub-interpreter will import a separate copy of each ``PYBIND11_EMBEDDED_MODULE``
+when those modules specify a ``multiple_interpreters`` tag. If a module does not
+specify a ``multiple_interpreters`` tag, then Python will report an ``ImportError``
+if it is imported in a sub-interpreter.
+
+pybind11 also has a :class:`scoped_subinterpreter` class, which creates and
+activates a sub-interpreter when it is constructed, and deactivates and deletes
+it when it goes out of scope.
+
+Activating a Sub-interpreter
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Once a sub-interpreter is created, you can "activate" it on a thread (and
+acquire it's GIL) by creating a :class:`subinterpreter_scoped_activate`
path: docs/advanced/embedding.rst
user: login: Copilot
body: Use “its” instead of “it's” in documentation.
```suggestion
acquire its GIL) by creating a :class:`subinterpreter_scoped_activate`
```
created_at: 2025-05-20T03:21:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,320 @@
+/*
+    pybind11/subinterpreter.h: Support for creating and using subinterpreters
+
+    Copyright (c) 2025 The Pybind Development Team.
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "detail/common.h"
+#include "detail/internals.h"
+#include "gil.h"
+
+#include <stdexcept>
+
+#if !defined(PYBIND11_SUBINTERPRETER_SUPPORT)
+#    error "This platform does not support subinterpreters, do not include this file."
+#endif
+
+PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+PYBIND11_NAMESPACE_BEGIN(detail)
+PyInterpreterState *get_interpreter_state_unchecked() {
+    auto cur_tstate = get_thread_state_unchecked();
+    if (cur_tstate)
+        return cur_tstate->interp;
+    else
+        return nullptr;
+}
+PYBIND11_NAMESPACE_END(detail)
+
+class subinterpreter;
+
+/// Activate the subinterpreter and acquire its GIL, while also releasing any GIL and interpreter
+/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and its
+/// associated GIL are restored to their state as they were before the scope was entered.
+class subinterpreter_scoped_activate {
+public:
+    explicit subinterpreter_scoped_activate(subinterpreter const &si);
+    ~subinterpreter_scoped_activate();
+
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate &&) = delete;
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate const &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate const &) = delete;
+
+private:
+    PyThreadState *old_tstate_ = nullptr;
+    PyThreadState *tstate_ = nullptr;
+    PyGILState_STATE gil_state_;
+    bool simple_gil_ = false;
+};
+
+/// Holds a Python subinterpreter instance
+class subinterpreter {
+public:
+    /// empty/unusable, but move-assignable.  use create() to create a subinterpreter.
+    subinterpreter() = default;
+
+    subinterpreter(subinterpreter const &copy) = delete;
+    subinterpreter &operator=(subinterpreter const &copy) = delete;
+
+    subinterpreter(subinterpreter &&old)
+        : istate_(old.istate_), creation_tstate_(old.creation_tstate_) {
+        old.istate_ = nullptr;
+        old.creation_tstate_ = nullptr;
+    }
+
+    subinterpreter &operator=(subinterpreter &&old) {
path: include/pybind11/subinterpreter.h
user: login: Skylion007
body: ```suggestion
    subinterpreter &operator=(subinterpreter &&old) noexcept {
```
I'm surprised clang-tidy didn't catch this...
created_at: 2025-05-20T16:01:14Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,320 @@
+/*
+    pybind11/subinterpreter.h: Support for creating and using subinterpreters
+
+    Copyright (c) 2025 The Pybind Development Team.
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "detail/common.h"
+#include "detail/internals.h"
+#include "gil.h"
+
+#include <stdexcept>
+
+#if !defined(PYBIND11_SUBINTERPRETER_SUPPORT)
+#    error "This platform does not support subinterpreters, do not include this file."
+#endif
+
+PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+PYBIND11_NAMESPACE_BEGIN(detail)
+PyInterpreterState *get_interpreter_state_unchecked() {
+    auto cur_tstate = get_thread_state_unchecked();
+    if (cur_tstate)
+        return cur_tstate->interp;
+    else
+        return nullptr;
+}
+PYBIND11_NAMESPACE_END(detail)
+
+class subinterpreter;
+
+/// Activate the subinterpreter and acquire its GIL, while also releasing any GIL and interpreter
+/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and its
+/// associated GIL are restored to their state as they were before the scope was entered.
+class subinterpreter_scoped_activate {
+public:
+    explicit subinterpreter_scoped_activate(subinterpreter const &si);
+    ~subinterpreter_scoped_activate();
+
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate &&) = delete;
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate const &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate const &) = delete;
+
+private:
+    PyThreadState *old_tstate_ = nullptr;
+    PyThreadState *tstate_ = nullptr;
+    PyGILState_STATE gil_state_;
+    bool simple_gil_ = false;
+};
+
+/// Holds a Python subinterpreter instance
+class subinterpreter {
+public:
+    /// empty/unusable, but move-assignable.  use create() to create a subinterpreter.
+    subinterpreter() = default;
+
+    subinterpreter(subinterpreter const &copy) = delete;
+    subinterpreter &operator=(subinterpreter const &copy) = delete;
+
+    subinterpreter(subinterpreter &&old)
path: include/pybind11/subinterpreter.h
user: login: Skylion007
body: Also should be noexcept?
created_at: 2025-05-20T16:01:45Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,320 @@
+/*
+    pybind11/subinterpreter.h: Support for creating and using subinterpreters
+
+    Copyright (c) 2025 The Pybind Development Team.
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "detail/common.h"
+#include "detail/internals.h"
+#include "gil.h"
+
+#include <stdexcept>
+
+#if !defined(PYBIND11_SUBINTERPRETER_SUPPORT)
+#    error "This platform does not support subinterpreters, do not include this file."
+#endif
+
+PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+PYBIND11_NAMESPACE_BEGIN(detail)
+PyInterpreterState *get_interpreter_state_unchecked() {
+    auto cur_tstate = get_thread_state_unchecked();
+    if (cur_tstate)
+        return cur_tstate->interp;
+    else
+        return nullptr;
+}
+PYBIND11_NAMESPACE_END(detail)
+
+class subinterpreter;
+
+/// Activate the subinterpreter and acquire its GIL, while also releasing any GIL and interpreter
+/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and its
+/// associated GIL are restored to their state as they were before the scope was entered.
+class subinterpreter_scoped_activate {
+public:
+    explicit subinterpreter_scoped_activate(subinterpreter const &si);
+    ~subinterpreter_scoped_activate();
+
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate &&) = delete;
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate const &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate const &) = delete;
+
+private:
+    PyThreadState *old_tstate_ = nullptr;
+    PyThreadState *tstate_ = nullptr;
+    PyGILState_STATE gil_state_;
+    bool simple_gil_ = false;
+};
+
+/// Holds a Python subinterpreter instance
+class subinterpreter {
+public:
+    /// empty/unusable, but move-assignable.  use create() to create a subinterpreter.
+    subinterpreter() = default;
+
+    subinterpreter(subinterpreter const &copy) = delete;
+    subinterpreter &operator=(subinterpreter const &copy) = delete;
+
+    subinterpreter(subinterpreter &&old)
+        : istate_(old.istate_), creation_tstate_(old.creation_tstate_) {
+        old.istate_ = nullptr;
+        old.creation_tstate_ = nullptr;
+    }
+
+    subinterpreter &operator=(subinterpreter &&old) {
+        std::swap(old.istate_, istate_);
+        std::swap(old.creation_tstate_, creation_tstate_);
+        return *this;
+    }
+
+    /// Create a new subinterpreter with the specified configuration
+    /// @note This function acquires (and then releases) the main interpreter GIL, but the main
+    /// interpreter and its GIL are not required to be held prior to calling this function.
+    static inline subinterpreter create(PyInterpreterConfig const &cfg) {
+        error_scope err_scope;
+        subinterpreter result;
+        {
+            // we must hold the main GIL in order to create a subinterpreter
+            subinterpreter_scoped_activate main_guard(main());
+
+            auto prev_tstate = PyThreadState_Get();
+
+            auto status = Py_NewInterpreterFromConfig(&result.creation_tstate_, &cfg);
+
+            // this doesn't raise a normal Python exception, it provides an exit() status code.
+            if (PyStatus_Exception(status)) {
+                pybind11_fail("failed to create new sub-interpreter");
+            }
+
+            // upon success, the new interpreter is activated in this thread
+            result.istate_ = result.creation_tstate_->interp;
+            detail::get_num_interpreters_seen() += 1; // there are now many interpreters
+            detail::get_internals(); // initialize internals.tstate, amongst other things...
+
+            // In 3.13+ this state should be deleted right away, and the memory will be reused for
+            // the next threadstate on this interpreter. However, on 3.12 we cannot do that, we
+            // must keep it around (but not use it) ... see destructor.
+#if PY_VERSION_HEX >= 0x030D0000
+            PyThreadState_Clear(result.creation_tstate_);
+            PyThreadState_DeleteCurrent();
+#endif
+
+            // we have to switch back to main, and then the scopes will handle cleanup
+            PyThreadState_Swap(prev_tstate);
+        }
+        return result;
+    }
+
+    /// Calls create() with a default configuration of an isolated interpreter that disallows fork,
+    /// exec, and Python threads.
+    static inline subinterpreter create() {
+        // same as the default config in the python docs
+        PyInterpreterConfig cfg;
+        memset(&cfg, 0, sizeof(cfg));
path: include/pybind11/subinterpreter.h
user: login: Skylion007
body: ```suggestion
        std::memset(&cfg, 0, sizeof(cfg));
```
created_at: 2025-05-20T16:02:21Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,320 @@
+/*
+    pybind11/subinterpreter.h: Support for creating and using subinterpreters
+
+    Copyright (c) 2025 The Pybind Development Team.
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "detail/common.h"
+#include "detail/internals.h"
+#include "gil.h"
+
+#include <stdexcept>
+
+#if !defined(PYBIND11_SUBINTERPRETER_SUPPORT)
+#    error "This platform does not support subinterpreters, do not include this file."
+#endif
+
+PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+PYBIND11_NAMESPACE_BEGIN(detail)
+PyInterpreterState *get_interpreter_state_unchecked() {
+    auto cur_tstate = get_thread_state_unchecked();
+    if (cur_tstate)
+        return cur_tstate->interp;
+    else
+        return nullptr;
+}
+PYBIND11_NAMESPACE_END(detail)
+
+class subinterpreter;
+
+/// Activate the subinterpreter and acquire its GIL, while also releasing any GIL and interpreter
+/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and its
+/// associated GIL are restored to their state as they were before the scope was entered.
+class subinterpreter_scoped_activate {
+public:
+    explicit subinterpreter_scoped_activate(subinterpreter const &si);
+    ~subinterpreter_scoped_activate();
+
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate &&) = delete;
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate const &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate const &) = delete;
+
+private:
+    PyThreadState *old_tstate_ = nullptr;
+    PyThreadState *tstate_ = nullptr;
+    PyGILState_STATE gil_state_;
+    bool simple_gil_ = false;
+};
+
+/// Holds a Python subinterpreter instance
+class subinterpreter {
+public:
+    /// empty/unusable, but move-assignable.  use create() to create a subinterpreter.
+    subinterpreter() = default;
+
+    subinterpreter(subinterpreter const &copy) = delete;
+    subinterpreter &operator=(subinterpreter const &copy) = delete;
+
+    subinterpreter(subinterpreter &&old)
+        : istate_(old.istate_), creation_tstate_(old.creation_tstate_) {
+        old.istate_ = nullptr;
+        old.creation_tstate_ = nullptr;
+    }
+
+    subinterpreter &operator=(subinterpreter &&old) {
+        std::swap(old.istate_, istate_);
+        std::swap(old.creation_tstate_, creation_tstate_);
+        return *this;
+    }
+
+    /// Create a new subinterpreter with the specified configuration
+    /// @note This function acquires (and then releases) the main interpreter GIL, but the main
+    /// interpreter and its GIL are not required to be held prior to calling this function.
+    static inline subinterpreter create(PyInterpreterConfig const &cfg) {
+        error_scope err_scope;
+        subinterpreter result;
+        {
+            // we must hold the main GIL in order to create a subinterpreter
+            subinterpreter_scoped_activate main_guard(main());
+
+            auto prev_tstate = PyThreadState_Get();
+
+            auto status = Py_NewInterpreterFromConfig(&result.creation_tstate_, &cfg);
+
+            // this doesn't raise a normal Python exception, it provides an exit() status code.
+            if (PyStatus_Exception(status)) {
+                pybind11_fail("failed to create new sub-interpreter");
+            }
+
+            // upon success, the new interpreter is activated in this thread
+            result.istate_ = result.creation_tstate_->interp;
+            detail::get_num_interpreters_seen() += 1; // there are now many interpreters
+            detail::get_internals(); // initialize internals.tstate, amongst other things...
+
+            // In 3.13+ this state should be deleted right away, and the memory will be reused for
+            // the next threadstate on this interpreter. However, on 3.12 we cannot do that, we
+            // must keep it around (but not use it) ... see destructor.
+#if PY_VERSION_HEX >= 0x030D0000
+            PyThreadState_Clear(result.creation_tstate_);
+            PyThreadState_DeleteCurrent();
+#endif
+
+            // we have to switch back to main, and then the scopes will handle cleanup
+            PyThreadState_Swap(prev_tstate);
+        }
+        return result;
+    }
+
+    /// Calls create() with a default configuration of an isolated interpreter that disallows fork,
+    /// exec, and Python threads.
+    static inline subinterpreter create() {
+        // same as the default config in the python docs
+        PyInterpreterConfig cfg;
+        memset(&cfg, 0, sizeof(cfg));
+        cfg.check_multi_interp_extensions = 1;
+        cfg.gil = PyInterpreterConfig_OWN_GIL;
+        return create(cfg);
+    }
+
+    ~subinterpreter() {
+        if (!creation_tstate_) {
+            // non-owning wrapper, do nothing.
+            return;
+        }
+
+        PyThreadState *destroy_tstate;
+        PyThreadState *old_tstate;
+
+        // Python 3.12 requires us to keep the original PyThreadState alive until we are ready to
+        // destroy the interpreter.  We prefer to use that to destroy the interpreter.
+#if PY_VERSION_HEX < 0x030D0000
+        // The tstate passed to Py_EndInterpreter MUST have been created on the current OS thread.
+        bool same_thread = false;
+#    ifdef PY_HAVE_THREAD_NATIVE_ID
+        same_thread = PyThread_get_thread_native_id() == creation_tstate_->native_thread_id;
+#    endif
+        if (same_thread) {
+            // OK it is safe to use the creation state here
+            destroy_tstate = creation_tstate_;
+            old_tstate = PyThreadState_Swap(destroy_tstate);
+        } else {
+            // We have to make a new tstate on this thread and use that.
+            destroy_tstate = PyThreadState_New(istate_);
+            old_tstate = PyThreadState_Swap(destroy_tstate);
+
+            // We can use the one we just created, so we must delete the creation state.
+            PyThreadState_Clear(creation_tstate_);
+            PyThreadState_Delete(creation_tstate_);
+        }
+#else
+        destroy_tstate = PyThreadState_New(istate_);
+        old_tstate = PyThreadState_Swap(destroy_tstate);
+#endif
+
+        bool switch_back = old_tstate && old_tstate->interp != istate_;
+
+        // Get the internals pointer (without creating it if it doesn't exist).  It's possible
+        // for the internals to be created during Py_EndInterpreter() (e.g. if a py::capsule
+        // calls `get_internals()` during destruction), so we get the pointer-pointer here and
+        // check it after.
+        auto *&internals_ptr_ptr = detail::get_internals_pp<detail::internals>();
+        auto *&local_internals_ptr_ptr = detail::get_internals_pp<detail::local_internals>();
+        {
+            dict sd = state_dict();
+            internals_ptr_ptr
+                = detail::get_internals_pp_from_capsule_in_state_dict<detail::internals>(
+                    sd, PYBIND11_INTERNALS_ID);
+            local_internals_ptr_ptr
+                = detail::get_internals_pp_from_capsule_in_state_dict<detail::local_internals>(
+                    sd, detail::get_local_internals_id());
+        }
+
+        // End it
+        Py_EndInterpreter(destroy_tstate);
+
+        // do NOT decrease detail::get_num_interpreters_seen, because it can never decrease
+        // while other threads are running...
+
+        if (internals_ptr_ptr) {
+            internals_ptr_ptr->reset();
+        }
+        if (local_internals_ptr_ptr) {
+            local_internals_ptr_ptr->reset();
+        }
+
+        // switch back to the old tstate and old GIL (if there was one)
+        if (switch_back)
+            PyThreadState_Swap(old_tstate);
+    }
+
+    /// Get a handle to the main interpreter that can be used with subinterpreter_scoped_activate
+    /// Note that destructing the handle is a noop, the main interpreter can only be ended by
+    /// py::finalize_interpreter()
+    static subinterpreter main() {
+        subinterpreter m;
+        m.istate_ = PyInterpreterState_Main();
+        m.disarm(); // make destruct a noop
+        return m;
+    }
+
+    /// Get a non-owning wrapper of the currently active interpreter (if any)
+    static subinterpreter current() {
+        subinterpreter c;
+        c.istate_ = detail::get_interpreter_state_unchecked();
+        c.disarm(); // make destruct a noop, we don't own this...
+        return c;
+    }
+
+    /// Get the numerical identifier for the sub-interpreter
+    int64_t id() const {
+        if (istate_ != nullptr)
+            return PyInterpreterState_GetID(istate_);
+        else
+            return -1; // CPython uses one-up numbers from 0, so negative should be safe to return
+                       // here.
+    }
+
+    /// Get the interpreter's state dict.  This interpreter's GIL must be held before calling!
+    dict state_dict() { return reinterpret_borrow<dict>(PyInterpreterState_GetDict(istate_)); }
+
+    /// abandon cleanup of this subinterpreter (leak it). this might be needed during
+    /// finalization...
+    void disarm() { creation_tstate_ = nullptr; }
+
+    /// An empty wrapper cannot be activated
+    bool empty() const { return istate_ == nullptr; }
path: include/pybind11/subinterpreter.h
user: login: Skylion007
body: I forget, do we have a macro for nodiscard?
created_at: 2025-05-20T16:02:47Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,320 @@
+/*
+    pybind11/subinterpreter.h: Support for creating and using subinterpreters
+
+    Copyright (c) 2025 The Pybind Development Team.
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "detail/common.h"
+#include "detail/internals.h"
+#include "gil.h"
+
+#include <stdexcept>
+
+#if !defined(PYBIND11_SUBINTERPRETER_SUPPORT)
+#    error "This platform does not support subinterpreters, do not include this file."
+#endif
+
+PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+PYBIND11_NAMESPACE_BEGIN(detail)
+PyInterpreterState *get_interpreter_state_unchecked() {
+    auto cur_tstate = get_thread_state_unchecked();
+    if (cur_tstate)
+        return cur_tstate->interp;
+    else
+        return nullptr;
+}
+PYBIND11_NAMESPACE_END(detail)
+
+class subinterpreter;
+
+/// Activate the subinterpreter and acquire its GIL, while also releasing any GIL and interpreter
+/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and its
+/// associated GIL are restored to their state as they were before the scope was entered.
+class subinterpreter_scoped_activate {
+public:
+    explicit subinterpreter_scoped_activate(subinterpreter const &si);
+    ~subinterpreter_scoped_activate();
+
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate &&) = delete;
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate const &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate const &) = delete;
+
+private:
+    PyThreadState *old_tstate_ = nullptr;
+    PyThreadState *tstate_ = nullptr;
+    PyGILState_STATE gil_state_;
+    bool simple_gil_ = false;
+};
+
+/// Holds a Python subinterpreter instance
+class subinterpreter {
+public:
+    /// empty/unusable, but move-assignable.  use create() to create a subinterpreter.
+    subinterpreter() = default;
+
+    subinterpreter(subinterpreter const &copy) = delete;
+    subinterpreter &operator=(subinterpreter const &copy) = delete;
+
+    subinterpreter(subinterpreter &&old)
+        : istate_(old.istate_), creation_tstate_(old.creation_tstate_) {
+        old.istate_ = nullptr;
+        old.creation_tstate_ = nullptr;
+    }
+
+    subinterpreter &operator=(subinterpreter &&old) {
path: include/pybind11/subinterpreter.h
user: login: Skylion007
body: Hmm, is this not being included in the cmake for some reason?
created_at: 2025-05-20T16:05:00Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,320 @@
+/*
+    pybind11/subinterpreter.h: Support for creating and using subinterpreters
+
+    Copyright (c) 2025 The Pybind Development Team.
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "detail/common.h"
+#include "detail/internals.h"
+#include "gil.h"
+
+#include <stdexcept>
+
+#if !defined(PYBIND11_SUBINTERPRETER_SUPPORT)
+#    error "This platform does not support subinterpreters, do not include this file."
+#endif
+
+PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+PYBIND11_NAMESPACE_BEGIN(detail)
+PyInterpreterState *get_interpreter_state_unchecked() {
+    auto cur_tstate = get_thread_state_unchecked();
+    if (cur_tstate)
+        return cur_tstate->interp;
+    else
+        return nullptr;
+}
+PYBIND11_NAMESPACE_END(detail)
+
+class subinterpreter;
+
+/// Activate the subinterpreter and acquire its GIL, while also releasing any GIL and interpreter
+/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and its
+/// associated GIL are restored to their state as they were before the scope was entered.
+class subinterpreter_scoped_activate {
+public:
+    explicit subinterpreter_scoped_activate(subinterpreter const &si);
+    ~subinterpreter_scoped_activate();
+
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate &&) = delete;
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate const &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate const &) = delete;
+
+private:
+    PyThreadState *old_tstate_ = nullptr;
+    PyThreadState *tstate_ = nullptr;
+    PyGILState_STATE gil_state_;
+    bool simple_gil_ = false;
+};
+
+/// Holds a Python subinterpreter instance
+class subinterpreter {
+public:
+    /// empty/unusable, but move-assignable.  use create() to create a subinterpreter.
+    subinterpreter() = default;
+
+    subinterpreter(subinterpreter const &copy) = delete;
+    subinterpreter &operator=(subinterpreter const &copy) = delete;
+
+    subinterpreter(subinterpreter &&old)
+        : istate_(old.istate_), creation_tstate_(old.creation_tstate_) {
+        old.istate_ = nullptr;
+        old.creation_tstate_ = nullptr;
+    }
+
+    subinterpreter &operator=(subinterpreter &&old) {
path: include/pybind11/subinterpreter.h
user: login: henryiii
body: I don't think our embedded tests are. The header is included, but it doesn't get checked unless it's also compiled somewhere, I think.
created_at: 2025-05-20T16:22:45Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,320 @@
+/*
+    pybind11/subinterpreter.h: Support for creating and using subinterpreters
+
+    Copyright (c) 2025 The Pybind Development Team.
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "detail/common.h"
+#include "detail/internals.h"
+#include "gil.h"
+
+#include <stdexcept>
+
+#if !defined(PYBIND11_SUBINTERPRETER_SUPPORT)
+#    error "This platform does not support subinterpreters, do not include this file."
+#endif
+
+PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+PYBIND11_NAMESPACE_BEGIN(detail)
+PyInterpreterState *get_interpreter_state_unchecked() {
+    auto cur_tstate = get_thread_state_unchecked();
+    if (cur_tstate)
+        return cur_tstate->interp;
+    else
+        return nullptr;
+}
+PYBIND11_NAMESPACE_END(detail)
+
+class subinterpreter;
+
+/// Activate the subinterpreter and acquire its GIL, while also releasing any GIL and interpreter
+/// currently held. Upon exiting the scope, the previous subinterpreter (if any) and its
+/// associated GIL are restored to their state as they were before the scope was entered.
+class subinterpreter_scoped_activate {
+public:
+    explicit subinterpreter_scoped_activate(subinterpreter const &si);
+    ~subinterpreter_scoped_activate();
+
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate &&) = delete;
+    subinterpreter_scoped_activate(subinterpreter_scoped_activate const &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate &) = delete;
+    subinterpreter_scoped_activate &operator=(subinterpreter_scoped_activate const &) = delete;
+
+private:
+    PyThreadState *old_tstate_ = nullptr;
+    PyThreadState *tstate_ = nullptr;
+    PyGILState_STATE gil_state_;
+    bool simple_gil_ = false;
+};
+
+/// Holds a Python subinterpreter instance
+class subinterpreter {
+public:
+    /// empty/unusable, but move-assignable.  use create() to create a subinterpreter.
+    subinterpreter() = default;
+
+    subinterpreter(subinterpreter const &copy) = delete;
+    subinterpreter &operator=(subinterpreter const &copy) = delete;
+
+    subinterpreter(subinterpreter &&old)
+        : istate_(old.istate_), creation_tstate_(old.creation_tstate_) {
+        old.istate_ = nullptr;
+        old.creation_tstate_ = nullptr;
+    }
+
+    subinterpreter &operator=(subinterpreter &&old) {
+        std::swap(old.istate_, istate_);
+        std::swap(old.creation_tstate_, creation_tstate_);
+        return *this;
+    }
+
+    /// Create a new subinterpreter with the specified configuration
+    /// @note This function acquires (and then releases) the main interpreter GIL, but the main
+    /// interpreter and its GIL are not required to be held prior to calling this function.
+    static inline subinterpreter create(PyInterpreterConfig const &cfg) {
+        error_scope err_scope;
+        subinterpreter result;
+        {
+            // we must hold the main GIL in order to create a subinterpreter
+            subinterpreter_scoped_activate main_guard(main());
+
+            auto prev_tstate = PyThreadState_Get();
+
+            auto status = Py_NewInterpreterFromConfig(&result.creation_tstate_, &cfg);
+
+            // this doesn't raise a normal Python exception, it provides an exit() status code.
+            if (PyStatus_Exception(status)) {
+                pybind11_fail("failed to create new sub-interpreter");
+            }
+
+            // upon success, the new interpreter is activated in this thread
+            result.istate_ = result.creation_tstate_->interp;
+            detail::get_num_interpreters_seen() += 1; // there are now many interpreters
+            detail::get_internals(); // initialize internals.tstate, amongst other things...
+
+            // In 3.13+ this state should be deleted right away, and the memory will be reused for
+            // the next threadstate on this interpreter. However, on 3.12 we cannot do that, we
+            // must keep it around (but not use it) ... see destructor.
+#if PY_VERSION_HEX >= 0x030D0000
+            PyThreadState_Clear(result.creation_tstate_);
+            PyThreadState_DeleteCurrent();
+#endif
+
+            // we have to switch back to main, and then the scopes will handle cleanup
+            PyThreadState_Swap(prev_tstate);
+        }
+        return result;
+    }
+
+    /// Calls create() with a default configuration of an isolated interpreter that disallows fork,
+    /// exec, and Python threads.
+    static inline subinterpreter create() {
+        // same as the default config in the python docs
+        PyInterpreterConfig cfg;
+        memset(&cfg, 0, sizeof(cfg));
+        cfg.check_multi_interp_extensions = 1;
+        cfg.gil = PyInterpreterConfig_OWN_GIL;
+        return create(cfg);
+    }
+
+    ~subinterpreter() {
+        if (!creation_tstate_) {
+            // non-owning wrapper, do nothing.
+            return;
+        }
+
+        PyThreadState *destroy_tstate;
+        PyThreadState *old_tstate;
+
+        // Python 3.12 requires us to keep the original PyThreadState alive until we are ready to
+        // destroy the interpreter.  We prefer to use that to destroy the interpreter.
+#if PY_VERSION_HEX < 0x030D0000
+        // The tstate passed to Py_EndInterpreter MUST have been created on the current OS thread.
+        bool same_thread = false;
+#    ifdef PY_HAVE_THREAD_NATIVE_ID
+        same_thread = PyThread_get_thread_native_id() == creation_tstate_->native_thread_id;
+#    endif
+        if (same_thread) {
+            // OK it is safe to use the creation state here
+            destroy_tstate = creation_tstate_;
+            old_tstate = PyThreadState_Swap(destroy_tstate);
+        } else {
+            // We have to make a new tstate on this thread and use that.
+            destroy_tstate = PyThreadState_New(istate_);
+            old_tstate = PyThreadState_Swap(destroy_tstate);
+
+            // We can use the one we just created, so we must delete the creation state.
+            PyThreadState_Clear(creation_tstate_);
+            PyThreadState_Delete(creation_tstate_);
+        }
+#else
+        destroy_tstate = PyThreadState_New(istate_);
+        old_tstate = PyThreadState_Swap(destroy_tstate);
+#endif
+
+        bool switch_back = old_tstate && old_tstate->interp != istate_;
+
+        // Get the internals pointer (without creating it if it doesn't exist).  It's possible
+        // for the internals to be created during Py_EndInterpreter() (e.g. if a py::capsule
+        // calls `get_internals()` during destruction), so we get the pointer-pointer here and
+        // check it after.
+        auto *&internals_ptr_ptr = detail::get_internals_pp<detail::internals>();
+        auto *&local_internals_ptr_ptr = detail::get_internals_pp<detail::local_internals>();
+        {
+            dict sd = state_dict();
+            internals_ptr_ptr
+                = detail::get_internals_pp_from_capsule_in_state_dict<detail::internals>(
+                    sd, PYBIND11_INTERNALS_ID);
+            local_internals_ptr_ptr
+                = detail::get_internals_pp_from_capsule_in_state_dict<detail::local_internals>(
+                    sd, detail::get_local_internals_id());
+        }
+
+        // End it
+        Py_EndInterpreter(destroy_tstate);
+
+        // do NOT decrease detail::get_num_interpreters_seen, because it can never decrease
+        // while other threads are running...
+
+        if (internals_ptr_ptr) {
+            internals_ptr_ptr->reset();
+        }
+        if (local_internals_ptr_ptr) {
+            local_internals_ptr_ptr->reset();
+        }
+
+        // switch back to the old tstate and old GIL (if there was one)
+        if (switch_back)
+            PyThreadState_Swap(old_tstate);
+    }
+
+    /// Get a handle to the main interpreter that can be used with subinterpreter_scoped_activate
+    /// Note that destructing the handle is a noop, the main interpreter can only be ended by
+    /// py::finalize_interpreter()
+    static subinterpreter main() {
+        subinterpreter m;
+        m.istate_ = PyInterpreterState_Main();
+        m.disarm(); // make destruct a noop
+        return m;
+    }
+
+    /// Get a non-owning wrapper of the currently active interpreter (if any)
+    static subinterpreter current() {
+        subinterpreter c;
+        c.istate_ = detail::get_interpreter_state_unchecked();
+        c.disarm(); // make destruct a noop, we don't own this...
+        return c;
+    }
+
+    /// Get the numerical identifier for the sub-interpreter
+    int64_t id() const {
+        if (istate_ != nullptr)
+            return PyInterpreterState_GetID(istate_);
+        else
+            return -1; // CPython uses one-up numbers from 0, so negative should be safe to return
+                       // here.
+    }
+
+    /// Get the interpreter's state dict.  This interpreter's GIL must be held before calling!
+    dict state_dict() { return reinterpret_borrow<dict>(PyInterpreterState_GetDict(istate_)); }
+
+    /// abandon cleanup of this subinterpreter (leak it). this might be needed during
+    /// finalization...
+    void disarm() { creation_tstate_ = nullptr; }
+
+    /// An empty wrapper cannot be activated
+    bool empty() const { return istate_ == nullptr; }
path: include/pybind11/subinterpreter.h
user: login: henryiii
body: `git grep nodiscard` doesn't reveal anything. That should be in the macro def, so no.
created_at: 2025-05-20T16:30:18Z
author_association: COLLABORATOR

PR comments:
user: login: henryiii
created_at: 2025-05-17T01:06:31Z
body: I think #5646 is about to go in, then I'll rebase this and add 3.13t and 3.14t in. (Edit: I think I meant to put this on the previous PR, didn't realize there were two)
author_association: COLLABORATOR
user: login: b-pass
created_at: 2025-05-17T14:33:02Z
body: It looks to me like the 3.14 crashes are actually a CPython bug, python/cpython#134144.
author_association: COLLABORATOR
user: login: henryiii
created_at: 2025-05-19T01:42:20Z
body: Do you know why mingw is failing?
author_association: COLLABORATOR
user: login: b-pass
created_at: 2025-05-19T02:09:12Z
body: > Do you know why mingw is failing?

Not sure.  My guess is the failure is destroying the sub-interpreter on a different OS thread than the one that created it.  You can do that in 3.13, but in 3.12 it seems even though the feature is "stable" there are some issue with PyThreadState objects on sub-interpreters in that version.
author_association: COLLABORATOR
user: login: rwgk
created_at: 2025-05-19T04:42:37Z
body: @henryiii Maybe it's a good time to drop AppVeyor? I haven't seen anything resulting uniquely from the AppVeyor job for years.
author_association: COLLABORATOR
user: login: henryiii
created_at: 2025-05-20T02:58:31Z
body: I don't think the failure is due to this PR, pasting it here and rerunning:

```
=================================== FAILURES ===================================
_ test_run_in_process_multiple_threads_parallel[test_cross_module_gil_nested_pybind11_released] _

test_fn = <function test_cross_module_gil_nested_pybind11_released at 0x2ebaf8a7500>

    @pytest.mark.skipif(sys.platform.startswith("emscripten"), reason="Requires threads")
    @pytest.mark.parametrize("test_fn", ALL_BASIC_TESTS_PLUS_INTENTIONAL_DEADLOCK)
    @pytest.mark.skipif(
        "env.GRAALPY",
        reason="GraalPy transiently complains about unfinished threads at process exit",
    )
    def test_run_in_process_multiple_threads_parallel(test_fn):
        """Makes sure there is no GIL deadlock when running in a thread multiple times in parallel.
    
        It runs in a separate process to be able to stop and assert if it deadlocks.
        """
>       assert _run_in_process(_run_in_threads, test_fn, num_threads=8, parallel=True) == 0
E       assert -11 == 0
E        +  where -11 = _run_in_process(_run_in_threads, <function test_cross_module_gil_nested_pybind11_released at 0x2ebaf8a7500>, num_threads=8, parallel=True)

test_fn    = <function test_cross_module_gil_nested_pybind11_released at 0x2ebaf8a7500>

../../tests/test_gil_scoped.py:241: AssertionError
=============================== warnings summary ===============================
<frozen importlib._bootstrap>:491
  <frozen importlib._bootstrap>:491: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'exo_planet_c_api', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.
```
author_association: COLLABORATOR
user: login: henryiii
created_at: 2025-05-20T03:14:37Z
body: Yes, it's a flake, and unrelated. We'll need to see how often it shows up, and in which jobs (3.14t only, or both).
author_association: COLLABORATOR
user: login: henryiii
created_at: 2025-05-20T05:36:05Z
body: @ericsnowcurrently You might be interested in this PR. ;)
author_association: COLLABORATOR
user: login: henryiii
created_at: 2025-05-20T05:42:19Z
body: I'm dropping the flaky 3.14t free-threading logs into https://github.com/pybind/pybind11/issues/5674. I've seen two of these now. They have not been in the subintepreters part (at least yet).
author_association: COLLABORATOR
user: login: ericsnowcurrently
created_at: 2025-05-20T13:59:26Z
body: I started looking at the PR a few days ago and got distracted by other stuff.  I do plan on getting back to it, but don't wait for me.

@henryiii (or any other pybind11 folks at PyCon), if you have time at the sprints I wouldn't mind walking through this with you.
author_association: NONE
user: login: henryiii
created_at: 2025-05-20T14:31:07Z
body: I left yesterday afternoon. @virtuald, are you still at the sprints?

The good news is the API is mostly separate from pybind11. What you do once you have a subinterpreter, of course, then you use pybind11 to interact with it, but the launching of subintepreters is isolated to this PR. (The embedding of Python is also from pybind11.)

I'm tempted to drop this into the 3.0 RC, since I'm still finishing preparations, and then we'll have ~1 week to change if anything pops up; this is new so last RC changes are fine, IMO.

author_association: COLLABORATOR
user: login: henryiii
created_at: 2025-05-20T20:33:46Z
body: I added cpptest to the clang-tidy job, but it didn't pick up on anything.
author_association: COLLABORATOR
user: login: henryiii
created_at: 2025-05-20T20:36:20Z
body: That's the first time I've seen that test flake on 3.13t, so it can happen on both. Noted in #5674.
author_association: COLLABORATOR