Repository: pytest-dev/pytest-qt
Pull request #596: Fix and clean up timer handling in signal/callback blockers
Author: login: The-Compiler
Created: 2025-03-22T19:22:54Z | Merged: 2025-03-25T14:17:16Z

- Always create the timeout `QTimer`, even if it's not going to be used.
- Connect the signal once in `_AbstractSignalBlocker`/`CallbackBlocker.__init__` and never disconnect it.
- When checking whether a timeout was set, simply check `self._timeout` instead of `self._timer`.

Before this change, we conditionally created a `QTimer` and deleted it again when cleaning up. In addition to coming with hard to follow complexity, this also caused multiple issues around this timer:

Warnings about disconnecting signal
===================================

In `AbstractSignalBlocker._cleanup`, we attempted to disconnect `self._timer.timeout()` from `self._quit_loop_by_timeout`, under the condition that a timer was created in `__init__` (i.e. `self.timeout` was not `0` or `None`).

However, the signal only got connected in `AbstractSignalBlocker.wait()`. Thus, if `AbstractSignalBlocker` is used as a context manager with a timeout and the signal is emitted inside it:

- `self._timer` is present
- The signal calls `self._quit_loop_by_signal()`
- `self._cleanup()` gets called from there
- That tries to disconnect `self._timer.timeout()` from `self._quit_loop_by_timeout()`
- Since `self.wait()` was never called, the signal was never connected
- Which then results in either an exception (PyQt), an internal SystemError (older PySide6) or a warning (newer PySide6).

In 560f565c062abb60fbba44719649e50e56f49cca and later 81b317c5e2170b135076958a730ef62bdb448317 this was fixed by ignoring `TypeError`/`RuntimeError`, but that turned out to not be sufficient for newer PySide versions: #552, #558.

The proper fix for this is to not attempt to disconnect a signal that was never connected, which makes all the PySide6 warnings go away (and thus we can run our testsuite with `-Werror` now).

As a drive-by fix, this also removes the old `filterwarnings` mark definition, which was introduced in 95fee8b20ccd20cde728dc4e05a488797fd19b93 (perhaps as a stop-gap for older pytest versions?) but shouldn't be needed anymore (and is unused since e33880994f85bdb336f943764ffb6ad0dacc366e).

# AttributeError when a signal/callback is emitted from a different thread 

If a signal is emitted from a thread, `_AbstractSignalBlocker._cleanup()` also gets called from the non-main thread (Qt will complain: "QObject::killTimer: Timers cannot be stopped from another thread").

If the signal emission just happened to happen between the None-check and calling `.start()` in `wait()`, it failed with an `AttributeError`:

Main thread in `AbstractSignalBlocker.wait()`:

```python
if self._timer is not None:  # <--- this was true...
    self._timer.timeout.connect(self._quit_loop_by_timeout)
    # <--- ...now here, but self._timer is None now!
    self._timer.start()
````

Emitting thread in `AbstractSignalBlocker._cleanup()` via `._quit_loop_by_signal()`:

```python
if self._timer is not None:
    ...
    self._timer = None  # <--- here
```

In SignalBlocker.connect, we used:

```python
actual_signal.connect(self._quit_loop_by_signal)
```

which by default is supposed to use a `QueuedConnection` if the signal gets emitted in a different thread:

    https://doc.qt.io/qt-6/qt.html#ConnectionType-enum
    (Default) If the receiver lives in the thread that emits the signal,
    `Qt::DirectConnection` is used. Otherwise, `Qt::QueuedConnection` is used. The
    connection type is determined when the signal is emitted.

though then that page says:

    https://doc.qt.io/qt-6/qobject.html#thread-affinity
    Note: If a `QObject` has no thread affinity (that is, if `thread()` returns
    zero), or if it lives in a thread that has no running event loop, then it
    cannot receive queued signals or posted events.

Which means `AbstractSignalBlocker` needs to be a `QObject` for this to work.

However, that means we need to use `qt_api.QtCore.QObject` as subclass, i.e. at import time of `wait_signal.py`. Yet, `qt_api` only gets initialized in `pytest_configure` so that it's configurable via a pytest setting.

Unfortunately, doing that is tricky, as it means we can't import `wait_signal.py` at all during import time, and people might do so for e.g. type annotations.

With this refactoring, the `AttributeError` is out of the way, though there are other subtle failures with multi-threaded signals now:

1) `_quit_loop_by_signal()` -> `_cleanup()` now simply calls `self._timer.stop()` without setting `self._timer` to `None`. This still results in the same Qt message quoted above (after all, the timer still doesn't belong to the calling thread!), but it looks like the test terminates without any timeout anyways. From what I can gather, while the "low level timer" continues to run (and waste a minimal amount of resources), the QTimer still "detaches" from it and stops running. The commit adds a test to catch this case (currently marked as xfail).

2) The main thread in `wait()` can now still call `self._timer.start()` without an `AttributeError`. However, in theory this could restart the timer after it was already stopped by the signal emission, with a race between `_cleanup()` and `wait()`.

See #586. This fixes the test-case posted by the reporter (added to the testsuite in a simplified version), but not the additional considerations above.

The same fix is also applied to `CallbackBlocker`, though the test there is way more unreliable in triggering the issue, and thus is skipped for taking too long.


Diff:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 5f354b42..88aeb9bd 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -1,9 +1,20 @@
 UNRELEASED
 ----------
 
-* Added official support for Python 3.13.
-* Dropped support for EOL Python 3.8.
-* Dropped support for EOL PySide 2.
+- Added official support for Python 3.13.
+- Dropped support for EOL Python 3.8.
+- Dropped support for EOL PySide 2.
+- Fixed PySide6 exceptions / warnings about being unable to disconnect signals
+  with ``qtbot.waitSignal`` (`#552`_, `#558`_).
+- Reduced the likelyhood of trouble when using ``qtbot.waitSignal(s)`` and
+  ``qtbot.waitCallback`` where the signal/callback is emitted from a non-main
+  thread. In theory, more problems remain and this isn't a proper fix yet. In
+  practice, it seems impossible to provoke any problems in pytest-qt's testsuite.
+  (`#586`_)
+
+..  _#552: https://github.com/pytest-dev/pytest-qt/issues/552
+..  _#558: https://github.com/pytest-dev/pytest-qt/issues/558
+..  _#586: https://github.com/pytest-dev/pytest-qt/issues/586
 
 4.4.0 (2024-02-07)
 ------------------
diff --git a/pytest.ini b/pytest.ini
index 9ade678f..26b0a163 100644
--- a/pytest.ini
+++ b/pytest.ini
@@ -2,5 +2,4 @@
 testpaths = tests
 addopts = --strict-markers --strict-config
 xfail_strict = true
-markers =
-  filterwarnings: pytest's filterwarnings marker
+filterwarnings = error
diff --git a/src/pytestqt/wait_signal.py b/src/pytestqt/wait_signal.py
index 8da3836c..04c66b68 100644
--- a/src/pytestqt/wait_signal.py
+++ b/src/pytestqt/wait_signal.py
@@ -24,12 +24,12 @@ def __init__(self, timeout=5000, raising=True):
         self.raising = raising
         self._signals = None  # will be initialized by inheriting implementations
         self._timeout_message = ""
-        if timeout is None or timeout == 0:
-            self._timer = None
-        else:
-            self._timer = qt_api.QtCore.QTimer(self._loop)
-            self._timer.setSingleShot(True)
+
+        self._timer = qt_api.QtCore.QTimer(self._loop)
+        self._timer.setSingleShot(True)
+        if timeout is not None:
             self._timer.setInterval(timeout)
+        self._timer.timeout.connect(self._quit_loop_by_timeout)
 
     def wait(self):
         """
@@ -43,11 +43,13 @@ def wait(self):
             return
         if self.timeout is None and not self._signals:
             raise ValueError("No signals or timeout specified.")
-        if self._timer is not None:
-            self._timer.timeout.connect(self._quit_loop_by_timeout)
-            self._timer.start()
 
         if self.timeout != 0:
+            if self.timeout is not None:
+                # asserts as a stop-gap for possible multithreading issues
+                assert not self.signal_triggered
+                self._timer.start()
+                assert not self.signal_triggered
             qt_api.exec(self._loop)
 
         if not self.signal_triggered and self.raising:
@@ -62,10 +64,7 @@ def _quit_loop_by_timeout(self):
     def _cleanup(self):
         # store timeout message before the data to construct it is lost
         self._timeout_message = self._get_timeout_error_message()
-        if self._timer is not None:
-            _silent_disconnect(self._timer.timeout, self._quit_loop_by_timeout)
-            self._timer.stop()
-            self._timer = None
+        self._timer.stop()
 
     def _get_timeout_error_message(self):
         """Subclasses have to implement this, returning an appropriate error message for a TimeoutError."""
@@ -649,12 +648,12 @@ def __init__(self, timeout=5000, raising=True):
         self.kwargs = None
         self.called = False
         self._loop = qt_api.QtCore.QEventLoop()
-        if timeout is None:
-            self._timer = None
-        else:
-            self._timer = qt_api.QtCore.QTimer(self._loop)
-            self._timer.setSingleShot(True)
+
+        self._timer = qt_api.QtCore.QTimer(self._loop)
+        self._timer.setSingleShot(True)
+        if timeout is not None:
             self._timer.setInterval(timeout)
+        self._timer.timeout.connect(self._quit_loop_by_timeout)
 
     def wait(self):
         """
@@ -664,8 +663,7 @@ def wait(self):
         __tracebackhide__ = True
         if self.called:
             return
-        if self._timer is not None:
-            self._timer.timeout.connect(self._quit_loop_by_timeout)
+        if self.timeout is not None:
             self._timer.start()
         qt_api.exec(self._loop)
         if not self.called and self.raising:
@@ -687,10 +685,7 @@ def _quit_loop_by_timeout(self):
             self._loop.quit()
 
     def _cleanup(self):
-        if self._timer is not None:
-            _silent_disconnect(self._timer.timeout, self._quit_loop_by_timeout)
-            self._timer.stop()
-            self._timer = None
+        self._timer.stop()
 
     def __call__(self, *args, **kwargs):
         # Not inside the try: block, as if self.called is True, we did quit the
diff --git a/tests/conftest.py b/tests/conftest.py
index 6010c312..e09ea8b3 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -1,4 +1,3 @@
-import functools
 import time
 
 import pytest
@@ -63,10 +62,9 @@ def shutdown(self):
         def single_shot(self, signal, delay):
             t = qt_api.QtCore.QTimer(self)
             t.setSingleShot(True)
-            slot = functools.partial(self._emit, signal)
-            t.timeout.connect(slot)
+            t.timeout.connect(signal)
             t.start(delay)
-            self.timers_and_slots.append((t, slot))
+            self.timers_and_slots.append((t, signal))
 
         def single_shot_callback(self, callback, delay):
             t = qt_api.QtCore.QTimer(self)
@@ -75,9 +73,6 @@ def single_shot_callback(self, callback, delay):
             t.start(delay)
             self.timers_and_slots.append((t, callback))
 
-        def _emit(self, signal):
-            signal.emit()
-
     timer = Timer()
     yield timer
     timer.shutdown()
diff --git a/tests/test_wait_signal.py b/tests/test_wait_signal.py
index 87ca324e..8903dd52 100644
--- a/tests/test_wait_signal.py
+++ b/tests/test_wait_signal.py
@@ -1364,3 +1364,132 @@ def test_timeout_not_raising(self, qtbot):
         assert not callback.called
         assert callback.args is None
         assert callback.kwargs is None
+
+
+@pytest.mark.parametrize(
+    "check_stderr, count",
+    [
+        # Checking stderr messages
+        pytest.param(
+            True,  # check stderr
+            200,  # gets output reliably even with only few runs (often the first)
+            id="stderr",
+        ),
+        # Triggering AttributeError
+        pytest.param(
+            False,  # don't check stderr
+            # Hopefully enough to trigger the AttributeError race condition reliably.
+            # With 500 runs, only 1 of 5 Windows PySide6 CI jobs triggered it (but all
+            # Ubuntu/macOS jobs did).  With 1500 runs, Windows jobs still only triggered
+            # it 0-2 times.
+            #
+            # On my machine (Linux, Intel Core Ultra 9 185H), 500 runs trigger it
+            # reliably and take ~1s in total.
+            2500 if sys.platform == "win32" else 500,
+            id="attributeerror",
+        ),
+    ],
+)
+@pytest.mark.parametrize("multi_blocker", [True, False])
+def test_signal_raised_from_thread(
+    pytester: pytest.Pytester, check_stderr: bool, multi_blocker: bool, count: int
+) -> None:
+    """Wait for a signal with a thread.
+
+    Extracted from https://github.com/pytest-dev/pytest-qt/issues/586
+    """
+    pytester.makepyfile(
+        f"""
+        import pytest
+        from pytestqt.qt_compat import qt_api
+
+
+        class Worker(qt_api.QtCore.QObject):
+            signal = qt_api.Signal()
+
+
+        @pytest.mark.parametrize("_", range({count}))
+        def test_thread(qtbot, capfd, _):
+            worker = Worker()
+            thread = qt_api.QtCore.QThread()
+            worker.moveToThread(thread)
+            thread.start()
+
+            try:
+                if {multi_blocker}:  # multi_blocker
+                    with qtbot.waitSignals([worker.signal], timeout=500) as blocker:
+                        worker.signal.emit()
+                else:
+                    with qtbot.waitSignal(worker.signal, timeout=500) as blocker:
+                        worker.signal.emit()
+            finally:
+                thread.quit()
+                thread.wait()
+
+            if {check_stderr}:  # check_stderr
+                out, err = capfd.readouterr()
+                assert not err
+    """
+    )
+
+    res = pytester.runpytest_subprocess("-x", "-s")
+    outcomes = res.parseoutcomes()
+
+    if outcomes.get("failed", 0) and check_stderr and qt_api.pytest_qt_api == "pyside6":
+        # The test succeeds on PyQt (unsure why!), but we can't check
+        # qt_api.pytest_qt_api at import time, so we can't use
+        # pytest.mark.xfail conditionally.
+        pytest.xfail(
+            "Qt error: QObject::killTimer: "
+            "Timers cannot be stopped from another thread"
+        )
+
+    res.assert_outcomes(passed=outcomes["passed"])  # no failed/error
+
+
+@pytest.mark.skip(reason="Runs ~1min to reproduce bug reliably")
+def test_callback_in_thread(pytester: pytest.Pytester) -> None:
+    """Wait for a callback with a thread.
+
+    Inspired by https://github.com/pytest-dev/pytest-qt/issues/586
+    """
+    # Hopefully enough to trigger the bug reliably.
+    #
+    # On my machine (Linux, Intel Core Ultra 9 185H), sometimes the bug only
+    # triggers after ~30k runs (~44s). Thus, we skip this test by default.
+    count = 50_000
+
+    pytester.makepyfile(
+        f"""
+        import pytest
+        from pytestqt.qt_compat import qt_api
+
+
+        class Worker(qt_api.QtCore.QObject):
+            def __init__(self, callback):
+                super().__init__()
+                self.callback = callback
+
+            def call_callback(self):
+                self.callback()
+
+
+        @pytest.mark.parametrize("_", range({count}))
+        def test_thread(qtbot, _):
+            thread = qt_api.QtCore.QThread()
+
+            try:
+                with qtbot.waitCallback() as callback:
+                    worker = Worker(callback)
+                    worker.moveToThread(thread)
+                    thread.started.connect(worker.call_callback)
+                    thread.start()
+            finally:
+                thread.quit()
+                thread.wait()
+    """
+    )
+
+    res = pytester.runpytest_subprocess("-x")
+    outcomes = res.parseoutcomes()
+    res.assert_outcomes(passed=outcomes["passed"])  # no failed/error


Reviews:
user: login: nicoddemus
body: Awesome work!


state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-24T13:33:34Z
user: login: nicoddemus
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T13:34:16Z
user: login: The-Compiler
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T15:07:05Z
user: login: nicoddemus
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T17:52:37Z

Review comments:
diff_hunk: @@ -43,11 +43,13 @@ def wait(self):
             return
         if self.timeout is None and not self._signals:
             raise ValueError("No signals or timeout specified.")
-        if self._timer is not None:
-            self._timer.timeout.connect(self._quit_loop_by_timeout)
-            self._timer.start()
 
         if self.timeout != 0:
+            if self.timeout is not None:
path: src/pytestqt/wait_signal.py
user: login: nicoddemus
body: Probably makes sense to unify these 2 ifs?
created_at: 2025-03-24T13:28:21Z
author_association: MEMBER
diff_hunk: @@ -63,10 +62,9 @@ def shutdown(self):
         def single_shot(self, signal, delay):
             t = qt_api.QtCore.QTimer(self)
             t.setSingleShot(True)
-            slot = functools.partial(self._emit, signal)
-            t.timeout.connect(slot)
+            t.timeout.connect(signal)
             t.start(delay)
-            self.timers_and_slots.append((t, slot))
+            self.timers_and_slots.append((t, signal))
path: tests/conftest.py
user: login: nicoddemus
body: :+1: 
created_at: 2025-03-24T13:31:14Z
author_association: MEMBER
diff_hunk: @@ -43,11 +43,13 @@ def wait(self):
             return
         if self.timeout is None and not self._signals:
             raise ValueError("No signals or timeout specified.")
-        if self._timer is not None:
-            self._timer.timeout.connect(self._quit_loop_by_timeout)
-            self._timer.start()
 
         if self.timeout != 0:
+            if self.timeout is not None:
path: src/pytestqt/wait_signal.py
user: login: nicoddemus
body: ~~Actually, probably we can drop the `if self.timeout != 0`, I would trust Qt even does that already.~~

Sorry, nevermind:

> A [QTimer](https://doc.qt.io/qt-6/qtimer.html) with a timeout interval of 0 will time out as soon as all the events in the window system's event queue have been processed.
created_at: 2025-03-24T13:34:15Z
author_association: MEMBER
diff_hunk: @@ -43,11 +43,13 @@ def wait(self):
             return
         if self.timeout is None and not self._signals:
             raise ValueError("No signals or timeout specified.")
-        if self._timer is not None:
-            self._timer.timeout.connect(self._quit_loop_by_timeout)
-            self._timer.start()
 
         if self.timeout != 0:
+            if self.timeout is not None:
path: src/pytestqt/wait_signal.py
user: login: The-Compiler
body: There's a `qt_api.exec(self._loop)` below which (at least for keeping the old logic) should be run if the timeout is `None` (= infinite), but should not be run if the timeout is 0 (docs: "If the timeout is set to `0`, it’s expected that the signal arrives directly in the code inside the with `qtbot.waitSignal(...):` block.").

I noticed we *don't* have the same handling in `CallbackBlocker`, which might need a review to make sure it all makes sense and is consistent somehow, but that seems out of scope for this PR.
created_at: 2025-03-24T15:07:05Z
author_association: MEMBER
diff_hunk: @@ -43,11 +43,13 @@ def wait(self):
             return
         if self.timeout is None and not self._signals:
             raise ValueError("No signals or timeout specified.")
-        if self._timer is not None:
-            self._timer.timeout.connect(self._quit_loop_by_timeout)
-            self._timer.start()
 
         if self.timeout != 0:
+            if self.timeout is not None:
path: src/pytestqt/wait_signal.py
user: login: nicoddemus
body: Sure thing, thanks!
created_at: 2025-03-24T17:52:37Z
author_association: MEMBER

PR comments:
user: login: The-Compiler
created_at: 2025-03-23T20:13:53Z
body: Ran into some unexpected issues on CI, but now all should be fine!
author_association: MEMBER
user: login: The-Compiler
created_at: 2025-03-25T07:27:24Z
body: Looks like all is green and ready finally :sweat_smile: 

Should I self-merge?
author_association: MEMBER
user: login: nicoddemus
created_at: 2025-03-25T12:32:19Z
body: > Should I self-merge?

Sure please go ahead!
author_association: MEMBER