Repository: HypothesisWorks/hypothesis
Pull request: first draft of handling marker exceptions wrapped in exceptiongroup
I have barely used Hypothesis, and never worked with the codebase, so there are several gaps in code coverage and question marks on implementation decisions. If somebody more familiar wants to take a quick pass and opine it'd be very helpful :)

I can get code coverage by manually raising the specific `ExceptionGroup`s that'll trigger the different branches, but it'd likely be better to do the actual things that causes those exceptions. So would likewise appreciate quick pointers on how to, if at all possible, cause e.g. multiple `StopTest`, multiple `Frozen`, a single `Frozen`, etc.

fixes #4106

Problem statement:
Our internals should handle `StopTest` and other marker exceptions even when wrapped in an `ExceptionGroup`
Hypothesis currently misbehaves if the marker exceptions we raise are wrapped up in an `ExceptionGroup`, e.g.:

```python
import trio
from hypothesis import given, strategies as st

@given(st.data())
def test(data):
    async def task(pred):
        return data.draw(st.booleans().filter(pred))

    @trio.run
    async def _():
        async with trio.open_nursery() as nursery:
            nursery.start_soon(task, bool)
            nursery.start_soon(task, lambda _: False)
```
```
  + Exception Group Traceback (most recent call last):
  |   ...
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   ...
    | hypothesis.errors.StopTest: 1
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   ...
    | hypothesis.errors.Frozen: Cannot call start_example on frozen ConjectureData
    +------------------------------------
```

In such cases I think it's pretty clear that we should treat an exception group _containing_ a `StopTest` as we would a `StopTest` itself.  I think it's also pretty clear that any `Frozen` exceptions inside an `ExceptionGroup` are causally downstream of some other exception, and can therefore be discarded so long as there's any other exception in the group too.

Hints from issue discussion:
How picky are you about depending on `exceptiongroup` for newer python versions?
3.12.1 added `ExceptionGroup` support for `contextlib.suppress`, so if we wanna use `exceptiongroup.suppress` we need to bump the `install_requirement` to `python < 3.12.1`.
I count 9 instances of `suppress` usage.

Big `try: ... except: ...` blocks such as https://github.com/HypothesisWorks/hypothesis/blob/4d079f7a62a6fdabfb2f2a2b276cb0f721ea62de/hypothesis-python/src/hypothesis/core.py#L1037
Needs to have their exception handlers split out into separate functions. We're then left with the choice of using `exceptiongroup.catch` on all python versions to invoke them, or rolling our own handler in an `except ExceptionGroup`.

I don't think rolling our own is super ugly/complicated in either case, so am open to both options.
I suggest it may be simplest to unwrap/discard the `exceptiongroup` around actual test execution, similar in effect to adding it to the end of `ensure_free_stackframes.__exit__` but in a new context manager. Then, everything outside - including the `suppress` calls - is unaffected.

(i.e., `with unwrap_exceptions(), ensure_free_stackframes(): return self.test(...)`)

[ed: To be clear, the context manager would discard `Frozen` subexceptions, and if there's exactly one remaining subexception then raise that instead of the group]
Thanks @jobh! That made it much easier to get started on a PR :)
I've opened #4110, would love any reviews

Commit message: add future annotations to unbreak py38

Patch to review:
diff --git a/hypothesis-python/src/hypothesis/core.py b/hypothesis-python/src/hypothesis/core.py
index 304ba614d0..bb9bd36346 100644
--- a/hypothesis-python/src/hypothesis/core.py
+++ b/hypothesis-python/src/hypothesis/core.py
@@ -9,6 +9,7 @@
 # obtain one at https://mozilla.org/MPL/2.0/.
 
 """This module provides the core primitives of Hypothesis, such as given."""
+from __future__ import annotations
 
 import base64
 import contextlib
@@ -32,13 +33,9 @@
     BinaryIO,
     Callable,
     Coroutine,
+    Generator,
     Hashable,
-    List,
-    Optional,
-    Tuple,
-    Type,
     TypeVar,
-    Union,
     overload,
 )
 from unittest import TestCase
@@ -61,6 +58,7 @@
     FlakyFailure,
     FlakyReplay,
     Found,
+    Frozen,
     HypothesisException,
     HypothesisWarning,
     InvalidArgument,
@@ -148,7 +146,6 @@
 else:  # pragma: no cover
     EllipsisType = type(Ellipsis)
 
-
 TestFunc = TypeVar("TestFunc", bound=Callable)
 
 
@@ -178,7 +175,7 @@ def __init__(self, *args: Any, **kwargs: Any) -> None:
         if not (args or kwargs):
             raise InvalidArgument("An example must provide at least one argument")
 
-        self.hypothesis_explicit_examples: List[Example] = []
+        self.hypothesis_explicit_examples: list[Example] = []
         self._this_example = Example(tuple(args), kwargs)
 
     def __call__(self, test: TestFunc) -> TestFunc:
@@ -192,10 +189,8 @@ def xfail(
         condition: bool = True,  # noqa: FBT002
         *,
         reason: str = "",
-        raises: Union[
-            Type[BaseException], Tuple[Type[BaseException], ...]
-        ] = BaseException,
-    ) -> "example":
+        raises: type[BaseException] | tuple[type[BaseException], ...] = BaseException,
+    ) -> example:
         """Mark this example as an expected failure, similarly to
         :obj:`pytest.mark.xfail(strict=True) <pytest.mark.xfail>`.
 
@@ -264,7 +259,7 @@ def test(x):
             )
         return self
 
-    def via(self, whence: str, /) -> "example":
+    def via(self, whence: str, /) -> example:
         """Attach a machine-readable label noting whence this example came.
 
         The idea is that tools will be able to add ``@example()`` cases for you, e.g.
@@ -768,6 +763,61 @@ def execute(data, function):
     return default_executor
 
 
+@contextlib.contextmanager
+def unwrap_exception_group() -> Generator[None]:
+    T = TypeVar("T", bound=BaseException)
+
+    def _flatten_group(excgroup: BaseExceptionGroup[T]) -> list[T]:
+        found_exceptions = []
+        for exc in excgroup.exceptions:
+            if isinstance(exc, BaseExceptionGroup):
+                found_exceptions.extend(_flatten_group(exc))
+            else:
+                found_exceptions.append(exc)
+        return found_exceptions
+
+    try:
+        yield
+    except BaseExceptionGroup as excgroup:
+        frozen_exceptions, non_frozen_exceptions = excgroup.split(Frozen)
+
+        # group only contains Frozen, reraise the group
+        # it doesn't matter what we raise, since any exceptions get disregarded
+        # and reraised as StopTest if data got frozen.
+        if non_frozen_exceptions is None:
+            raise
+        # in all other cases they are discarded
+
+        # Can RewindRecursive end up in this group?
+        _, user_exceptions = non_frozen_exceptions.split(
+            lambda e: isinstance(e, (StopTest, HypothesisException))
+        )
+
+        # this might contain marker exceptions, or internal errors, but not frozen.
+        if user_exceptions is not None:
+            raise
+
+        # single marker exception - reraise it
+        flattened_non_frozen_exceptions = _flatten_group(non_frozen_exceptions)
+        if len(flattened_non_frozen_exceptions) == 1:
+            raise flattened_non_frozen_exceptions[0] from None
+
+        # multiple marker exceptions. If we re-raise the whole group we break
+        # a bunch of logic so ....?
+        stoptests, non_stoptests = non_frozen_exceptions.split(StopTest)
+
+        # TODO: stoptest+hypothesisexception ...? Is it possible? If so, what do?
+
+        if non_stoptests:
+            # TODO: multiple marker exceptions is easy to produce, but the logic in the
+            # engine does not handle it... so we just reraise the first one for now.
+            raise _flatten_group(non_stoptests)[0] from None
+        assert stoptests is not None
+
+        # multiple stoptests: raising the one with the lowest testcounter
+        raise min(_flatten_group(stoptests), key=lambda s_e: s_e.testcounter)
+
+
 class StateForActualGivenExecution:
     def __init__(self, stuff, test, settings, random, wrapped_test):
         self.test_runner = get_executor(stuff.selfy)
@@ -841,7 +891,7 @@ def execute_once(
 
             @proxies(self.test)
             def test(*args, **kwargs):
-                with ensure_free_stackframes():
+                with unwrap_exception_group(), ensure_free_stackframes():
                     return self.test(*args, **kwargs)
 
         else:
@@ -853,7 +903,7 @@ def test(*args, **kwargs):
                 arg_gctime = gc_cumulative_time()
                 start = time.perf_counter()
                 try:
-                    with ensure_free_stackframes():
+                    with unwrap_exception_group(), ensure_free_stackframes():
                         result = self.test(*args, **kwargs)
                 finally:
                     finish = time.perf_counter()
@@ -1086,6 +1136,9 @@ def _execute_once_for_engine(self, data: ConjectureData) -> None:
                 # This can happen if an error occurred in a finally
                 # block somewhere, suppressing our original StopTest.
                 # We raise a new one here to resume normal operation.
+                # TODO: this should maybe inspect the stacktrace to see that the above
+                # mentioned story is true? I.e. reraise as StopTest iff there is a
+                # StopTest somewhere in the tree of e.__context__
                 raise StopTest(data.testcounter) from e
             else:
                 # The test failed by raising an exception, so we inform the
@@ -1146,7 +1199,7 @@ def _execute_once_for_engine(self, data: ConjectureData) -> None:
             self._timing_features = {}
 
     def _deliver_information_message(
-        self, *, type: str, title: str, content: Union[str, dict]
+        self, *, type: str, title: str, content: str | dict
     ) -> None:
         deliver_json_blob(
             {
@@ -1249,6 +1302,7 @@ def run_engine(self):
             ran_example.slice_comments = falsifying_example.slice_comments
             tb = None
             origin = None
+            assert info is not None
             assert info._expected_exception is not None
             try:
                 with with_reporter(fragments.append):
@@ -1407,7 +1461,7 @@ class HypothesisHandle:
     @property
     def fuzz_one_input(
         self,
-    ) -> Callable[[Union[bytes, bytearray, memoryview, BinaryIO]], Optional[bytes]]:
+    ) -> Callable[[bytes | bytearray | memoryview | BinaryIO], bytes | None]:
         """Run the test as a fuzz target, driven with the `buffer` of bytes.
 
         Returns None if buffer invalid for the strategy, canonical pruned
@@ -1428,7 +1482,7 @@ def fuzz_one_input(
 def given(
     _: EllipsisType, /
 ) -> Callable[
-    [Callable[..., Optional[Coroutine[Any, Any, None]]]], Callable[[], None]
+    [Callable[..., Coroutine[Any, Any, None] | None]], Callable[[], None]
 ]:  # pragma: no cover
     ...
 
@@ -1437,26 +1491,24 @@ def given(
 def given(
     *_given_arguments: SearchStrategy[Any],
 ) -> Callable[
-    [Callable[..., Optional[Coroutine[Any, Any, None]]]], Callable[..., None]
+    [Callable[..., Coroutine[Any, Any, None] | None]], Callable[..., None]
 ]:  # pragma: no cover
     ...
 
 
 @overload
 def given(
-    **_given_kwargs: Union[SearchStrategy[Any], EllipsisType],
+    **_given_kwargs: SearchStrategy[Any] | EllipsisType,
 ) -> Callable[
-    [Callable[..., Optional[Coroutine[Any, Any, None]]]], Callable[..., None]
+    [Callable[..., Coroutine[Any, Any, None] | None]], Callable[..., None]
 ]:  # pragma: no cover
     ...
 
 
 def given(
-    *_given_arguments: Union[SearchStrategy[Any], EllipsisType],
-    **_given_kwargs: Union[SearchStrategy[Any], EllipsisType],
-) -> Callable[
-    [Callable[..., Optional[Coroutine[Any, Any, None]]]], Callable[..., None]
-]:
+    *_given_arguments: SearchStrategy[Any] | EllipsisType,
+    **_given_kwargs: SearchStrategy[Any] | EllipsisType,
+) -> Callable[[Callable[..., Coroutine[Any, Any, None] | None]], Callable[..., None]]:
     """A decorator for turning a test function that accepts arguments into a
     randomized test.
 
@@ -1725,7 +1777,7 @@ def wrapped_test(*arguments, **kwargs):
                 raise SKIP_BECAUSE_NO_EXAMPLES
 
         def _get_fuzz_target() -> (
-            Callable[[Union[bytes, bytearray, memoryview, BinaryIO]], Optional[bytes]]
+            Callable[[bytes | bytearray | memoryview | BinaryIO], bytes | None]
         ):
             # Because fuzzing interfaces are very performance-sensitive, we use a
             # somewhat more complicated structure here.  `_get_fuzz_target()` is
@@ -1757,8 +1809,8 @@ def _get_fuzz_target() -> (
             minimal_failures: dict = {}
 
             def fuzz_one_input(
-                buffer: Union[bytes, bytearray, memoryview, BinaryIO]
-            ) -> Optional[bytes]:
+                buffer: bytes | bytearray | memoryview | BinaryIO,
+            ) -> bytes | None:
                 # This inner part is all that the fuzzer will actually run,
                 # so we keep it as small and as fast as possible.
                 if isinstance(buffer, io.IOBase):
@@ -1812,9 +1864,9 @@ def find(
     specifier: SearchStrategy[Ex],
     condition: Callable[[Any], bool],
     *,
-    settings: Optional[Settings] = None,
-    random: Optional[Random] = None,
-    database_key: Optional[bytes] = None,
+    settings: Settings | None = None,
+    random: Random | None = None,
+    database_key: bytes | None = None,
 ) -> Ex:
     """Returns the minimal example from the given strategy ``specifier`` that
     matches the predicate function ``condition``."""
@@ -1837,7 +1889,7 @@ def find(
         )
     specifier.validate()
 
-    last: List[Ex] = []
+    last: list[Ex] = []
 
     @settings
     @given(specifier)
diff --git a/hypothesis-python/src/hypothesis/internal/escalation.py b/hypothesis-python/src/hypothesis/internal/escalation.py
index 9c242ba0c2..5fc48360fa 100644
--- a/hypothesis-python/src/hypothesis/internal/escalation.py
+++ b/hypothesis-python/src/hypothesis/internal/escalation.py
@@ -13,9 +13,10 @@
 import sys
 import textwrap
 import traceback
+from functools import partial
 from inspect import getframeinfo
 from pathlib import Path
-from typing import Dict, NamedTuple, Optional, Type
+from typing import Dict, NamedTuple, Optional, Tuple, Type
 
 import hypothesis
 from hypothesis.errors import _Trimmable
@@ -107,20 +108,44 @@ def __str__(self) -> str:
         return f"{self.exc_type.__name__} at {self.filename}:{self.lineno}{ctx}{group}"
 
     @classmethod
-    def from_exception(cls, exception: BaseException, /) -> "InterestingOrigin":
+    def from_exception(
+        cls,
+        exception: BaseException,
+        /,
+        handled_exceptions: Tuple[BaseException, ...] = (),
+    ) -> "InterestingOrigin":
+        if exception in handled_exceptions:
+            exception.__context__ = None
         filename, lineno = None, None
         if tb := get_trimmed_traceback(exception):
             filename, lineno, *_ = traceback.extract_tb(tb)[-1]
+
+        # Quick dirty fix for #4115
+        handled_exceptions = (*handled_exceptions, exception)
+
         return cls(
             type(exception),
             filename,
             lineno,
             # Note that if __cause__ is set it is always equal to __context__, explicitly
             # to support introspection when debugging, so we can use that unconditionally.
-            cls.from_exception(exception.__context__) if exception.__context__ else (),
+            (
+                cls.from_exception(
+                    exception.__context__, handled_exceptions=handled_exceptions
+                )
+                if exception.__context__
+                else ()
+            ),
             # We distinguish exception groups by the inner exceptions, as for __context__
             (
-                tuple(map(cls.from_exception, exception.exceptions))
+                tuple(
+                    map(
+                        partial(
+                            cls.from_exception, handled_exceptions=handled_exceptions
+                        ),
+                        exception.exceptions,
+                    )
+                )
                 if isinstance(exception, BaseExceptionGroup)
                 else ()
             ),
diff --git a/hypothesis-python/tests/cover/test_exceptiongroup.py b/hypothesis-python/tests/cover/test_exceptiongroup.py
new file mode 100644
index 0000000000..517974c82f
--- /dev/null
+++ b/hypothesis-python/tests/cover/test_exceptiongroup.py
@@ -0,0 +1,182 @@
+# This file is part of Hypothesis, which may be found at
+# https://github.com/HypothesisWorks/hypothesis/
+#
+# Copyright the Hypothesis Authors.
+# Individual contributors are listed in AUTHORS.rst and the git log.
+#
+# This Source Code Form is subject to the terms of the Mozilla Public License,
+# v. 2.0. If a copy of the MPL was not distributed with this file, You can
+# obtain one at https://mozilla.org/MPL/2.0/.
+
+from __future__ import annotations
+
+import pytest
+
+from hypothesis import errors, given, strategies as st
+from hypothesis.internal.compat import BaseExceptionGroup, ExceptionGroup
+from hypothesis.strategies import DataObject
+
+
+def test_discard_frozen() -> None:
+    @given(st.data())
+    def discard_frozen(data: DataObject) -> None:
+        # raising Frozen doesn't actually do anything, what matters is
+        # whether the data is frozen.
+        data.conjecture_data.freeze()
+        raise ExceptionGroup("", [errors.Frozen()])
+
+    discard_frozen()
+
+
+def test_discard_multiple_frozen() -> None:
+    @given(st.data())
+    def discard_multiple_frozen(data: DataObject) -> None:
+        data.conjecture_data.freeze()
+        raise ExceptionGroup("", [errors.Frozen(), errors.Frozen()])
+
+    discard_multiple_frozen()
+
+
+def test_user_error_and_frozen() -> None:
+    @given(st.data())
+    def user_error_and_frozen(data: DataObject) -> None:
+        raise ExceptionGroup("", [errors.Frozen(), TypeError()])
+
+    with pytest.raises(ExceptionGroup) as excinfo:
+        user_error_and_frozen()
+    e = excinfo.value
+    assert isinstance(e, ExceptionGroup)
+    assert len(e.exceptions) == 2
+    assert isinstance(e.exceptions[0], errors.Frozen)
+    assert isinstance(e.exceptions[1], TypeError)
+
+
+def test_user_error_and_stoptest() -> None:
+    # if the code base had "proper" handling of exceptiongroups, the StopTest would
+    # probably be handled by an except*.
+    # TODO: which I suppose is an argument in favor of stripping it??
+    @given(st.data())
+    def user_error_and_stoptest(data: DataObject) -> None:
+        raise BaseExceptionGroup(
+            "", [errors.StopTest(data.conjecture_data.testcounter), TypeError()]
+        )
+
+    with pytest.raises(BaseExceptionGroup) as excinfo:
+        user_error_and_stoptest()
+    e = excinfo.value
+    assert isinstance(e, BaseExceptionGroup)
+    assert len(e.exceptions) == 2
+    assert isinstance(e.exceptions[0], errors.StopTest)
+    assert isinstance(e.exceptions[1], TypeError)
+
+
+def test_lone_user_error() -> None:
+    # we don't want to unwrap exceptiongroups, since they might contain
+    # useful debugging info
+    @given(st.data())
+    def lone_user_error(data: DataObject) -> None:
+        raise ExceptionGroup("foo", [TypeError()])
+
+    with pytest.raises(ExceptionGroup) as excinfo:
+        lone_user_error()
+    e = excinfo.value
+    assert isinstance(e, ExceptionGroup)
+    assert len(e.exceptions) == 1
+    assert isinstance(e.exceptions[0], TypeError)
+
+
+def test_nested_stoptest() -> None:
+    @given(st.data())
+    def nested_stoptest(data: DataObject) -> None:
+        raise BaseExceptionGroup(
+            "",
+            [
+                BaseExceptionGroup(
+                    "", [errors.StopTest(data.conjecture_data.testcounter)]
+                )
+            ],
+        )
+
+    nested_stoptest()
+
+
+def test_frozen_and_stoptest() -> None:
+    # frozen+stoptest => strip frozen and let engine handle StopTest
+    # actually.. I don't think I've got a live repo for this either.
+    @given(st.data())
+    def frozen_and_stoptest(data: DataObject) -> None:
+        raise BaseExceptionGroup(
+            "", [errors.StopTest(data.conjecture_data.testcounter), errors.Frozen()]
+        )
+
+    frozen_and_stoptest()
+
+
+def test_multiple_stoptest_1() -> None:
+    # multiple stoptest, reraise the one with lowest testcounter
+    # TODO: I don't know if/how this can happen, nested tests perhaps??
+    @given(st.data())
+    def multiple_stoptest(data: DataObject) -> None:
+        c = data.conjecture_data.testcounter
+        raise BaseExceptionGroup("", [errors.StopTest(c), errors.StopTest(c + 1)])
+
+    multiple_stoptest()
+
+
+def test_multiple_stoptest_2() -> None:
+    # the lower value is raised, which does not match data.conjecture_data.testcounter
+    # so it is not handled by the engine
+    @given(st.data())
+    def multiple_stoptest_2(data: DataObject) -> None:
+        c = data.conjecture_data.testcounter
+        raise BaseExceptionGroup("", [errors.StopTest(c), errors.StopTest(c - 1)])
+
+    with pytest.raises(errors.StopTest):
+        multiple_stoptest_2()
+
+
+def test_stoptest_and_hypothesisexception() -> None:
+    # TODO: can this happen? If so what do?
+    # current code raises the first hypothesisexception and throws away stoptest
+    @given(st.data())
+    def stoptest_and_hypothesisexception(data: DataObject) -> None:
+        c = data.conjecture_data.testcounter
+        raise BaseExceptionGroup("", [errors.StopTest(c), errors.Flaky()])
+
+    with pytest.raises(errors.Flaky):
+        stoptest_and_hypothesisexception()
+
+
+def test_multiple_hypothesisexception() -> None:
+    # this can happen in several ways, see nocover/test_exceptiongroup.py
+    @given(st.data())
+    def stoptest_and_hypothesisexception(data: DataObject) -> None:
+        c = data.conjecture_data.testcounter
+        raise BaseExceptionGroup("", [errors.StopTest(c), errors.Flaky()])
+
+    with pytest.raises(errors.Flaky):
+        stoptest_and_hypothesisexception()
+
+
+# FIXME: temporarily added while debugging #4115
+def test_recursive_exception():
+    @given(st.data())
+    def test_function(data):
+        try:
+            raise ExceptionGroup("", [ValueError()])
+        except ExceptionGroup as eg:
+            raise eg.exceptions[0] from None
+
+    with pytest.raises(ValueError):
+        test_function()
+
+
+def test_recursive_exception2():
+    @given(st.data())
+    def test_function(data):
+        k = ValueError()
+        k.__context__ = k
+        raise k
+
+    with pytest.raises(ValueError):
+        test_function()
diff --git a/hypothesis-python/tests/nocover/test_exceptiongroup.py b/hypothesis-python/tests/nocover/test_exceptiongroup.py
new file mode 100644
index 0000000000..6d82c03a31
--- /dev/null
+++ b/hypothesis-python/tests/nocover/test_exceptiongroup.py
@@ -0,0 +1,197 @@
+# This file is part of Hypothesis, which may be found at
+# https://github.com/HypothesisWorks/hypothesis/
+#
+# Copyright the Hypothesis Authors.
+# Individual contributors are listed in AUTHORS.rst and the git log.
+#
+# This Source Code Form is subject to the terms of the Mozilla Public License,
+# v. 2.0. If a copy of the MPL was not distributed with this file, You can
+# obtain one at https://mozilla.org/MPL/2.0/.
+
+from __future__ import annotations
+
+import asyncio
+import sys
+from typing import Callable
+
+import pytest
+
+from hypothesis import errors, given, reject, strategies as st
+from hypothesis.internal.compat import ExceptionGroup
+from hypothesis.strategies import DataObject
+
+# this file is not typechecked by mypy, which only runs py310
+
+if sys.version_info < (3, 11):
+    pytest.skip("asyncio.TaskGroup not available on <py3.11", allow_module_level=True)
+
+
+def test_exceptiongroup_discard_frozen():
+    """Basic test that raises Frozen+Unsatisfiable.
+    Frozen is thrown out, and Unsatisfiable is raised"""
+
+    @given(st.data())
+    def test_function(data):
+        async def task(pred):
+            return data.draw(st.booleans().filter(pred))
+
+        async def _main():
+            async with asyncio.TaskGroup() as tg:
+                tg.create_task(task(bool))
+                tg.create_task(task(lambda _: False))
+
+        asyncio.run(_main())
+
+    with pytest.raises(errors.Unsatisfiable):
+        test_function()
+
+
+def test_exceptiongroup_nested() -> None:
+    @given(st.data())
+    def test_function(data: DataObject) -> None:
+        async def task(pred: Callable[[bool], bool]) -> None:
+            data.draw(st.booleans().filter(pred))
+
+        async def _main() -> None:
+            async with asyncio.TaskGroup():
+                async with asyncio.TaskGroup() as tg2:
+                    tg2.create_task(task(bool))
+                    tg2.create_task(task(lambda _: False))
+
+        asyncio.run(_main())
+
+    with pytest.raises(errors.Unsatisfiable):
+        test_function()
+
+
+def test_exceptiongroup_user_originated() -> None:
+    @given(st.data())
+    def test_function(data):
+        raise ExceptionGroup("foo", [ValueError(), ValueError()])
+
+    with pytest.raises(ExceptionGroup) as exc_info:
+        test_function()
+    e = exc_info.value
+    assert e.message == "foo"
+    assert isinstance(e, ExceptionGroup)
+    assert len(e.exceptions) == 2
+    assert all(isinstance(child_e, ValueError) for child_e in e.exceptions)
+
+    @given(st.data())
+    def test_single_exc_group(data):
+        raise ExceptionGroup("important message for user", [ValueError()])
+
+    with pytest.raises(ExceptionGroup) as exc_info:
+        test_single_exc_group()
+    e = exc_info.value
+    assert e.message == "important message for user"
+    assert isinstance(e, ExceptionGroup)
+    assert len(e.exceptions) == 1
+    assert isinstance(e.exceptions[0], ValueError)
+
+
+def test_exceptiongroup_multiple_stop() -> None:
+    @given(st.data())
+    def test_function(data):
+        async def task(d: DataObject) -> None:
+            ...
+            # idk how to intentionally raise StopTest here, without mucking
+            # around with internals *a lot* in a way that nobody would ever
+            # be expected to do
+
+        async def _main(d: DataObject) -> None:
+            async with asyncio.TaskGroup() as tg:
+                tg.create_task(task(d))
+                tg.create_task(task(d))
+
+        asyncio.run(_main(data))
+
+    test_function()
+
+
+def test_exceptiongroup_stop_and_hypothesisexception() -> None:
+    # idk how to intentionally raise StopTest
+    ...
+
+
+def test_exceptiongroup_multiple_hypothesisexception() -> None:
+    # multiple UnsatisfiedAssumption => first one is reraised => engine suppresses it
+
+    @given(st.integers(min_value=0, max_value=3))
+    def test_function(val: int) -> None:
+        async def task(value: int) -> None:
+            if value == 0:
+                reject()
+
+        async def _main(value: int) -> None:
+            async with asyncio.TaskGroup() as tg:
+                tg.create_task(task(value))
+                tg.create_task(task(value))
+
+        asyncio.run(_main(val))
+
+    test_function()
+
+
+def test_exceptiongroup_multiple_InvalidArgument() -> None:
+    # multiple InvalidArgument => only first one is reraised... which seems bad.
+    # But raising a group might break ghostwriter(?)
+
+    @given(st.data())
+    def test_function(data: DataObject) -> None:
+        async def task1(d: DataObject) -> None:
+            d.draw(st.integers(max_value=1, min_value=3))
+
+        async def task2(d: DataObject) -> None:
+            d.draw(st.integers(max_value=2, min_value=3))
+
+        async def _main(d: DataObject) -> None:
+            async with asyncio.TaskGroup() as tg:
+                tg.create_task(task1(d))
+                tg.create_task(task2(d))
+
+        asyncio.run(_main(data))
+
+    with pytest.raises(errors.InvalidArgument):
+        test_function()
+
+
+# if intended, will probably remove this test, and either way probably belong somewhere else
+def test_frozen_things():
+    # Hypothesis reraises the TypeError as a StopTest, because the data is Frozen.
+    # Doesn't seem great, but I suppose it is intentional?
+    @given(st.data())
+    def foo(data):
+        data.conjecture_data.freeze()
+        raise TypeError("oops")
+
+    foo()
+
+
+# if above is intended, then this is supposedly also intended?
+def test_frozen_data_and_critical_user_exception():
+    @given(st.data())
+    def test_function(data):
+        data.conjecture_data.freeze()
+
+        async def task(d: DataObject) -> None:
+            d.draw(st.booleans())
+
+        async def task2() -> None:
+            raise TypeError("Critical error")
+
+        async def _main(d: DataObject) -> None:
+            async with asyncio.TaskGroup() as tg:
+                tg.create_task(task(d))
+                tg.create_task(task2())
+
+        asyncio.run(_main(data))
+
+    # does not raise anything, the TypeError is suppressed
+    test_function()
+    # with pytest.raises(ExceptionGroup) as exc_info:
+    # e = exc_info.value
+    # assert isinstance(e, ExceptionGroup)
+    # assert len(e.exceptions) == 2
+    # assert isinstance(e.exceptions[0], errors.Frozen)
+    # assert isinstance(e.exceptions[1], TypeError)


Code review comments:
diff_hunk: @@ -1086,6 +1136,9 @@ def _execute_once_for_engine(self, data: ConjectureData) -> None:
                 # This can happen if an error occurred in a finally
                 # block somewhere, suppressing our original StopTest.
                 # We raise a new one here to resume normal operation.
+                # TODO: this should maybe inspect the stacktrace to see that the above
+                # mentioned story is true? I.e. reraise as StopTest iff there is a
+                # StopTest somewhere in the tree of e.__context__
path: hypothesis-python/src/hypothesis/core.py
text: @user1:
Alas, `raise Whatever() from None` would defeat this inspection.

@author:
that would only override the `__cause__` no? And the comment specifically talks about errors happening in `finally`, where you rarely raise explicit exceptions.
It seems pretty dangerous to ignore all exceptions in case the data is frozen, shouldn't the user be informed if there's errors happening in their teardown code?
Although thinking about it more, the problem is perhaps that the `StopTest.__cause__` is not getting reraised/logged?

@user1:
Yeah, I think you're correct here, we do want to expose errors in teardown code.

> Although thinking about it more, the problem is perhaps that the `StopTest.__cause__` is not getting reraised/logged?

hmm... we never `raise StopTest(...) from ...`, but we certainly do raise it from inside exception handlers that we don't want to expose the user to (via something like `except ValueError: data.mark_invalid()`).  I guess we could inspect `__cause__`, but it'd have so many missed alarms that really doesn't seem worth doing :/

@author:
> hmm... we never raise StopTest(...) from ...

you do it on the very next line in this file :sweat_smile: 
```python
                 raise StopTest(data.testcounter) from e
```
Though I suppose saving it as `__cause__` for reraising in the engine is perhaps not the way to do it, and the caught exception should instead be saved Somewhere Else:tm:

@author:
Oh and to be explicit, I think traversing `__context__` for `StopTest` would be robust. As far as I know there's no standard way of losing entries in it - but not terribly elegant to traverse when you also have to deal with exception groups.

I'll try to write a test and open a separate issue/PR for it

@author:
opened #4132 to continue the discussion

diff_hunk: @@ -9,6 +9,7 @@
 # obtain one at https://mozilla.org/MPL/2.0/.
 
 """This module provides the core primitives of Hypothesis, such as given."""
+from __future__ import annotations
path: hypothesis-python/src/hypothesis/core.py
text: @user1:
Because Hypothesis does a _lot_ of runtime introspection, we prefer to avoid `from __future__ import annotations` - which will eventually go away anyway, c.f. PEPs 649 and 749.

@author:
do you also want to avoid it in test files?

@user1:
not a big deal but would prefer to avoid it, easier for future contributors to imitate a blanket policy rather than conditional policy 🙂 

@author:
hrm, I could swear there was a flake8 plugin with an optional rule that banned `from __future__ import annotations` but can't find it (nor any ruff rule). edit: I guess it's a non-issue with #4131 anyway :tada: