Repository: getsentry/sentry-python

Issue:
Python 3.12 support - RuntimeError
### Problem Statement

Tried using sentry with python 3.12. Ran into newly implemented RuntimeError tests. Want to give a heads up for once there is 3.12 (hopefuly after 3.11 https://github.com/getsentry/sentry-python/issues/1950 🤞 ) support that this is something that will probably need fixing(?)

Example traceback ([source](https://gitlab.com/cki-project/infrastructure/-/issues/235)): 
```
Exception ignored in atexit callback: <function AtexitIntegration.setup_once.<locals>._shutdown at 0x7fa77fa24e00>
Traceback (most recent call last):
  File "/builds/4704997395/software/venv/lib64/python3.12/site-packages/sentry_sdk/integrations/atexit.py", line 61, in _shutdown
    client.close(callback=integration.callback)
  File "/builds/4704997395/software/venv/lib64/python3.12/site-packages/sentry_sdk/client.py", line 580, in close
    self.flush(timeout=timeout, callback=callback)
  File "/builds/4704997395/software/venv/lib64/python3.12/site-packages/sentry_sdk/client.py", line 604, in flush
    self.transport.flush(timeout=timeout, callback=callback)
  File "/builds/4704997395/software/venv/lib64/python3.12/site-packages/sentry_sdk/transport.py", line 525, in flush
    self._worker.submit(lambda: self._flush_client_reports(force=True))
  File "/builds/4704997395/software/venv/lib64/python3.12/site-packages/sentry_sdk/worker.py", line 117, in submit
    self._ensure_thread()
  File "/builds/4704997395/software/venv/lib64/python3.12/site-packages/sentry_sdk/worker.py", line 42, in _ensure_thread
    self.start()
  File "/builds/4704997395/software/venv/lib64/python3.12/site-packages/sentry_sdk/worker.py", line 70, in start
    self._thread.start()
  File "/builds/4704997395/software/venv/lib64/python3.12/site-packages/sentry_sdk/integrations/threading.py", line 56, in sentry_start
    return old_start(self, *a, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/threading.py", line 971, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't create new thread at interpreter shutdown
```

cpython commit adding this "detection": https://github.com/python/cpython/commit/ce558e69d4087dd3653207de78345fbb8a2c7835

### Solution Brainstorm

Some sort of "graceful shutdown" on exit perhaps? 


Discussion:
We got to work around this by closing sentry manually at the end of our programs 🤷 
Thanks for the heads up @Toaster192!
Would be great to have this fixed now that Python 3.12 is officially released.
Agreed @jnoordsij. We're currently figuring out what we need to fix for 3.12, this is definitely on the list.
Looking at #1950, is there some hope hat it will not take month(s) for 3.12 support? Even more so considering that 3.11 was just about (non-technical) metadata, while there's an actual issue with 3.12 right now which is not necessarily a blocker but at least a bit annoying when an application has CLI tools for example where this exception gets raised.
We have a PR open for supporting 3.12, but we can not give you an estimate on when it will be merged/released:

https://github.com/getsentry/sentry-python/pull/2408
I'm also seeing this issue when terminating a Sanic server

```
^C[2023-10-14 15:38:55 -0400] [18739] [INFO] Received signal SIGINT. Shutting down.
[2023-10-14 15:38:55 -0400] [18739] [INFO] Server Stopped
Exception ignored in atexit callback: <function AtexitIntegration.setup_once.<locals>._shutdown at 0x103eb0c20>
Traceback (most recent call last):
  File "/Users/robd/code/allfactors-sanic/env/lib/python3.12/site-packages/sentry_sdk/integrations/atexit.py", line 61, in _shutdown
    client.close(callback=integration.callback)
  File "/Users/robd/code/allfactors-sanic/env/lib/python3.12/site-packages/sentry_sdk/client.py", line 618, in close
    self.flush(timeout=timeout, callback=callback)
  File "/Users/robd/code/allfactors-sanic/env/lib/python3.12/site-packages/sentry_sdk/client.py", line 646, in flush
    self.transport.flush(timeout=timeout, callback=callback)
  File "/Users/robd/code/allfactors-sanic/env/lib/python3.12/site-packages/sentry_sdk/transport.py", line 549, in flush
    self._worker.submit(lambda: self._flush_client_reports(force=True))
  File "/Users/robd/code/allfactors-sanic/env/lib/python3.12/site-packages/sentry_sdk/worker.py", line 117, in submit
    self._ensure_thread()
  File "/Users/robd/code/allfactors-sanic/env/lib/python3.12/site-packages/sentry_sdk/worker.py", line 42, in _ensure_thread
    self.start()
  File "/Users/robd/code/allfactors-sanic/env/lib/python3.12/site-packages/sentry_sdk/worker.py", line 70, in start
    self._thread.start()
  File "/Users/robd/code/allfactors-sanic/env/lib/python3.12/site-packages/sentry_sdk/integrations/threading.py", line 56, in sentry_start
    return old_start(self, *a, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.0/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 971, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't create new thread at interpreter shutdown
Traceback (most recent call last):
  File "/Users/robd/code/allfactors-sanic/src/af_events.py", line 1455, in <module>
    app.run(host='0.0.0.0', port=8081, workers=2, access_log=False, debug=False)
  File "/Users/robd/code/allfactors-sanic/env/lib/python3.12/site-packages/sanic/mixins/startup.py", line 209, in run
    serve(primary=self)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^
  File "/Users/robd/code/allfactors-sanic/env/lib/python3.12/site-packages/sanic/mixins/startup.py", line 880, in serve
    sock.shutdown(SHUT_RDWR)
OSError: [Errno 57] Socket is not connected
Sentry is attempting to send 2 pending events
```
This also occurs when running migrations in Django.

```
python .\src\manage.py migrate
2023-10-20 09:44:42,475 DEBUG __init__: couldn't import psycopg 'c' implementation: No module named 'psycopg_c'
Operations to perform:
  Apply all migrations: <bunch of apps>
Running migrations:
  No migrations to apply.
Exception ignored in atexit callback: <function AtexitIntegration.setup_once.<locals>._shutdown at 0x000001F382D9B380>
Traceback (most recent call last):
  File "C:\Users\user\.virtualenvs\sad-django\Lib\site-packages\sentry_sdk\integrations\atexit.py", line 61, in _shutdown
    client.close(callback=integration.callback)
  File "C:\Users\user\.virtualenvs\sad-django\Lib\site-packages\sentry_sdk\client.py", line 618, in close
    self.flush(timeout=timeout, callback=callback)
  File "C:\Users\user\.virtualenvs\sad-django\Lib\site-packages\sentry_sdk\client.py", line 646, in flush
    self.transport.flush(timeout=timeout, callback=callback)
  File "C:\Users\user\.virtualenvs\sad-django\Lib\site-packages\sentry_sdk\transport.py", line 549, in flush
    self._worker.submit(lambda: self._flush_client_reports(force=True))
  File "C:\Users\user\.virtualenvs\sad-django\Lib\site-packages\sentry_sdk\worker.py", line 117, in submit
    self._ensure_thread()
  File "C:\Users\user\.virtualenvs\sad-django\Lib\site-packages\sentry_sdk\worker.py", line 42, in _ensure_thread
    self.start()
  File "C:\Users\user\.virtualenvs\sad-django\Lib\site-packages\sentry_sdk\worker.py", line 70, in start
    self._thread.start()
  File "C:\Users\user\.virtualenvs\sad-django\Lib\site-packages\sentry_sdk\integrations\threading.py", line 56, in sentry_start
    return old_start(self, *a, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\users\user\.pyenv\pyenv-win\versions\3.12.0\Lib\threading.py", line 971, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't create new thread at interpreter shutdown
```
Yes, I think it always happens when the Python interpreter exits while the sentry SDK is 'active'...
Thanks for all the input! We will try to tackle this soon!

Resolving patch:
diff --git a/sentry_sdk/metrics.py b/sentry_sdk/metrics.py
--- a/sentry_sdk/metrics.py
+++ b/sentry_sdk/metrics.py
@@ -332,18 +332,27 @@ def __init__(
         self._ensure_thread()
 
     def _ensure_thread(self):
-        # type: (...) -> None
+        # type: (...) -> bool
         """For forking processes we might need to restart this thread.
         This ensures that our process actually has that thread running.
         """
+        if not self._running:
+            return False
         pid = os.getpid()
         if self._flusher_pid == pid:
-            return
+            return True
         with self._lock:
             self._flusher_pid = pid
             self._flusher = Thread(target=self._flush_loop)
             self._flusher.daemon = True
-            self._flusher.start()
+            try:
+                self._flusher.start()
+            except RuntimeError:
+                # Unfortunately at this point the interpreter is in a start that no
+                # longer allows us to spawn a thread and we have to bail.
+                self._running = False
+                return False
+        return True
 
     def _flush_loop(self):
         # type: (...) -> None
@@ -400,9 +409,7 @@ def add(
         timestamp=None,  # type: Optional[float]
     ):
         # type: (...) -> None
-        self._ensure_thread()
-
-        if self._flusher is None:
+        if not self._ensure_thread() or self._flusher is None:
             return
 
         if timestamp is None:
diff --git a/sentry_sdk/worker.py b/sentry_sdk/worker.py
--- a/sentry_sdk/worker.py
+++ b/sentry_sdk/worker.py
@@ -67,8 +67,14 @@ def start(self):
                     target=self._target, name="raven-sentry.BackgroundWorker"
                 )
                 self._thread.daemon = True
-                self._thread.start()
-                self._thread_for_pid = os.getpid()
+                try:
+                    self._thread.start()
+                    self._thread_for_pid = os.getpid()
+                except RuntimeError:
+                    # At this point we can no longer start because the interpreter
+                    # is already shutting down.  Sadly at this point we can no longer
+                    # send out events.
+                    self._thread = None
 
     def kill(self):
         # type: () -> None


Test patch: