Repository: HypothesisWorks/hypothesis
Pull request #4390: Faster AST collection set union
Author: login: Liam-DeVoe
Created: 2025-05-14T01:36:55Z | Merged: 2025-05-14T20:09:57Z

Also skips files over 200kb.

See https://github.com/HypothesisWorks/hypothesis/pull/4366#issuecomment-2877081864 (cc @zjmlianxi, and thanks for the great benchmarks!)

I have not verified/benchmarked this yet.


Diff:
diff --git a/hypothesis-python/RELEASE.rst b/hypothesis-python/RELEASE.rst
new file mode 100644
index 0000000000..950475699a
--- /dev/null
+++ b/hypothesis-python/RELEASE.rst
@@ -0,0 +1,3 @@
+RELEASE_TYPE: patch
+
+Further improve the performance of the new features introduced in :ref:`version 6.131.1 <v6.131.1>`, by improving unions of large sets.
diff --git a/hypothesis-python/docs/conf.py b/hypothesis-python/docs/conf.py
index a1e8a08961..1c8fbe4ac3 100644
--- a/hypothesis-python/docs/conf.py
+++ b/hypothesis-python/docs/conf.py
@@ -257,6 +257,17 @@ def setup(app):
 .. |@reproduce_failure| replace:: :func:`@reproduce_failure <hypothesis.reproduce_failure>`
 
 .. |ExampleDatabase| replace:: :class:`~hypothesis.database.ExampleDatabase`
+.. |ExampleDatabase.save| replace:: :func:`~hypothesis.database.ExampleDatabase.save`
+.. |ExampleDatabase.delete| replace:: :func:`~hypothesis.database.ExampleDatabase.delete`
+.. |ExampleDatabase.fetch| replace:: :func:`~hypothesis.database.ExampleDatabase.fetch`
+.. |ExampleDatabase.move| replace:: :func:`~hypothesis.database.ExampleDatabase.move`
+.. |ExampleDatabase.add_listener| replace:: :func:`~hypothesis.database.ExampleDatabase.add_listener`
+.. |ExampleDatabase.remove_listener| replace:: :func:`~hypothesis.database.ExampleDatabase.remove_listener`
+.. |ExampleDatabase.clear_listeners| replace:: :func:`~hypothesis.database.ExampleDatabase.clear_listeners`
+.. |ExampleDatabase._start_listening| replace:: :func:`~hypothesis.database.ExampleDatabase._start_listening`
+.. |ExampleDatabase._stop_listening| replace:: :func:`~hypothesis.database.ExampleDatabase._stop_listening`
+.. |ExampleDatabase._broadcast_change| replace:: :func:`~hypothesis.database.ExampleDatabase._broadcast_change`
+
 .. |DirectoryBasedExampleDatabase| replace:: :class:`~hypothesis.database.DirectoryBasedExampleDatabase`
 .. |InMemoryExampleDatabase| replace:: :class:`~hypothesis.database.InMemoryExampleDatabase`
 .. |ReadOnlyDatabase| replace:: :class:`~hypothesis.database.ReadOnlyDatabase`
diff --git a/hypothesis-python/docs/reference/api.rst b/hypothesis-python/docs/reference/api.rst
index f1cfbe90ed..a529d7a8e1 100644
--- a/hypothesis-python/docs/reference/api.rst
+++ b/hypothesis-python/docs/reference/api.rst
@@ -409,41 +409,23 @@ Using a value of ``list(HealthCheck)`` will disable all health checks.
 Database
 --------
 
-When Hypothesis finds a bug it stores enough information in its database to reproduce it. This
-enables you to have a classic testing workflow of find a bug, fix a bug, and be confident that
-this is actually doing the right thing because Hypothesis will start by retrying the examples that
-broke things last time.
+When Hypothesis finds a bug, it stores enough information in its database to reproduce it. The next
+time the test is run, Hypothesis will start by trying the stored example that failed last time.
 
-Limitations
-~~~~~~~~~~~
-
-The database is best thought of as a cache that you never need to invalidate: Information may be
-lost when you upgrade a Hypothesis version or change your test, so you shouldn't rely on it for
-correctness - if there's an example you want to ensure occurs each time then :ref:`there's a feature for
-including them in your source code <providing-explicit-examples>` - but it helps the development
-workflow considerably by making sure that the examples you've just found are reproduced.
-
-The database also records examples that exercise less-used parts of your
-code, so the database may update even when no failing examples were found.
+The database is best thought of as a cache that you never need to invalidate. Entries may be
+transparently dropped when upgrading your Hypothesis version or changing your test. You shouldn't
+rely on the database for correctness; to ensure Hypothesis always tries an example, use |@example|.
 
-Upgrading Hypothesis and changing your tests
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The design of the Hypothesis database is such that you can put arbitrary data in the database
-and not get wrong behaviour. When you upgrade Hypothesis, old data *might* be invalidated, but
-this should happen transparently. It can never be the case that e.g. changing the strategy
-that generates an argument gives you data from the old strategy.
+Hypothesis is designed so that arbitrary data can be placed in the database without causing incorrect
+behavior. It can never be the case that changing the strategy for a test gives you incorrect stored
+data from the previous strategy, for instance.
 
 ExampleDatabase implementations
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Hypothesis' default :obj:`~hypothesis.settings.database` setting creates a
-:class:`~hypothesis.database.DirectoryBasedExampleDatabase` in your current working directory,
-under ``.hypothesis/examples``.  If this location is unusable, e.g. because you do not have
-read or write permissions, Hypothesis will emit a warning and fall back to an
-:class:`~hypothesis.database.InMemoryExampleDatabase`.
-
-Hypothesis provides the following :class:`~hypothesis.database.ExampleDatabase` implementations:
+.. autoclass:: hypothesis.database.ExampleDatabase
+    :members:
+    :private-members: _broadcast_change, _start_listening, _stop_listening
 
 .. autoclass:: hypothesis.database.InMemoryExampleDatabase
 .. autoclass:: hypothesis.database.DirectoryBasedExampleDatabase
@@ -453,14 +435,60 @@ Hypothesis provides the following :class:`~hypothesis.database.ExampleDatabase`
 .. autoclass:: hypothesis.database.BackgroundWriteDatabase
 .. autoclass:: hypothesis.extra.redis.RedisExampleDatabase
 
-Defining your own ExampleDatabase
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+.. _custom-database:
 
-You can define your :class:`~hypothesis.database.ExampleDatabase`, for example
-to use a shared datastore, with just a few methods:
+Implementing your own database
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-.. autoclass:: hypothesis.database.ExampleDatabase
-   :members:
+To define your own |ExampleDatabase| class, implement the |ExampleDatabase.save|, |ExampleDatabase.fetch|, and |ExampleDatabase.delete| methods.
+
+For example, here's a simple database class that uses :mod:`sqlite <sqlite3>` as the backing data store:
+
+.. code-block:: python
+
+    import sqlite3
+    from collections.abc import Iterable
+
+    from hypothesis.database import ExampleDatabase
+
+    class SQLiteExampleDatabase(ExampleDatabase):
+        def __init__(self, db_path: str):
+            self.conn = sqlite3.connect(db_path)
+
+            self.conn.execute(
+                """
+                CREATE TABLE examples (
+                    key BLOB,
+                    value BLOB,
+                    UNIQUE (key, value)
+                )
+            """
+            )
+
+        def save(self, key: bytes, value: bytes) -> None:
+            self.conn.execute(
+                "INSERT OR IGNORE INTO examples VALUES (?, ?)",
+                (key, value),
+            )
+
+        def fetch(self, key: bytes) -> Iterable[bytes]:
+            cursor = self.conn.execute("SELECT value FROM examples WHERE key = ?", (key,))
+            yield from [value[0] for value in cursor.fetchall()]
+
+        def delete(self, key: bytes, value: bytes) -> None:
+            self.conn.execute(
+                "DELETE FROM examples WHERE key = ? AND value = ?",
+                (key, value),
+            )
+
+Database classes are not required to implement |ExampleDatabase.move|. The default implementation of a move is a |ExampleDatabase.delete| of the value in the old key, followed by a |ExampleDatabase.save| of the value in the new key. You can override |ExampleDatabase.move| in a database class to override this behavior, if for instance the backing store offers a more efficient move implementation.
+
+Change listening
+^^^^^^^^^^^^^^^^
+
+To support change listening in a database class, you should call ``self._broadcast_change(event)`` whenever a value is saved, deleted, or moved in the backing database store. How you track this depends on the details of the database class. For instance, in |DirectoryBasedExampleDatabase|, Hypothesis installs a filesystem monitor via :pypi:`watchdog` in order to broadcast change events.
+
+Two related useful methods are ``ExampleDatabase._start_listening`` and ``ExampleDatabase._stop_listening``, which a database class can override to know when to start or stop expensive listening operations. See source code for documentation.
 
 .. _stateful:
 
@@ -916,4 +944,4 @@ To determine whether a test has been defined with Hypothesis or not, use |is_hyp
 
 .. autofunction:: hypothesis.is_hypothesis_test
 
-If you're working with :pypi:`pytest`, our :ref:`pytest plugin <pytest-plugin>` automatically adds the ``@pytest.mark.hypothesis`` mark to all Hypothesis tests. You can use ``node.get_closest_marker("hypothesis")`` or similar methods to detect the existence of this mark.
+If you're working with :pypi:`pytest`, the :ref:`Hypothesis pytest plugin <pytest-plugin>` automatically adds the ``@pytest.mark.hypothesis`` mark to all Hypothesis tests. You can use ``node.get_closest_marker("hypothesis")`` or similar methods to detect the existence of this mark.
diff --git a/hypothesis-python/src/hypothesis/_settings.py b/hypothesis-python/src/hypothesis/_settings.py
index 8eed5c3ccc..a9af415ce1 100644
--- a/hypothesis-python/src/hypothesis/_settings.py
+++ b/hypothesis-python/src/hypothesis/_settings.py
@@ -549,12 +549,18 @@ def derandomize(self):
     @property
     def database(self):
         """
-        An instance of |ExampleDatabase| that will be
-        used to save examples to and load previous examples from. If ``None``,
-        no storage will be used.
+        An instance of |ExampleDatabase| that will be used to save examples to
+        and load previous examples from.
 
-        See the :ref:`database documentation <database>` for a list of built-in
-        example database implementations, and how to define custom implementations.
+        If not set, a |DirectoryBasedExampleDatabase| is created in the current
+        working directory under ``.hypothesis/examples``. If this location is
+        unusable, e.g. due to the lack of read or write permissions, Hypothesis
+        will emit a warning and fall back to an |InMemoryExampleDatabase|.
+
+        If ``None``, no storage will be used.
+
+        See the :ref:`database documentation <database>` for a list of database
+        classes, and how to define custom database classes.
         """
         from hypothesis.database import _db_for_path
 
diff --git a/hypothesis-python/src/hypothesis/control.py b/hypothesis-python/src/hypothesis/control.py
index 9e76b443fe..cc78ff77c1 100644
--- a/hypothesis-python/src/hypothesis/control.py
+++ b/hypothesis-python/src/hypothesis/control.py
@@ -287,7 +287,7 @@ def target(observation: Union[int, float], *, label: str = "") -> Union[int, flo
     ``target()`` with any label more than once per test case.
 
     .. note::
-        **The more examples you run, the better this technique works.**
+        The more examples you run, the better this technique works.
 
         As a rule of thumb, the targeting effect is noticeable above
         :obj:`max_examples=1000 <hypothesis.settings.max_examples>`,
diff --git a/hypothesis-python/src/hypothesis/core.py b/hypothesis-python/src/hypothesis/core.py
index 3eb902fd04..b82a4b15a8 100644
--- a/hypothesis-python/src/hypothesis/core.py
+++ b/hypothesis-python/src/hypothesis/core.py
@@ -277,7 +277,7 @@ def test(x):
 
 
 def seed(seed: Hashable) -> Callable[[TestFunc], TestFunc]:
-    """seed: Start the test execution from a specific seed.
+    """Start the test execution from a specific seed.
 
     May be any hashable object. No exact meaning for seed is provided
     other than that for a fixed seed value Hypothesis will try the same
diff --git a/hypothesis-python/src/hypothesis/database.py b/hypothesis-python/src/hypothesis/database.py
index 75680aff28..0a8bcb117b 100644
--- a/hypothesis-python/src/hypothesis/database.py
+++ b/hypothesis-python/src/hypothesis/database.py
@@ -146,10 +146,52 @@ def __call__(self, *args: Any, **kwargs: Any) -> "ExampleDatabase":
 
 
 class ExampleDatabase(metaclass=_EDMeta):
-    """An abstract base class for storing examples in Hypothesis' internal format.
+    """
+    A Hypothesis database, for use in the |settings.database| setting.
+
+    Using an |ExampleDatabase| allows Hypothesis to
+    remember and replay failing examples, and ensure that your tests are never
+    flaky. We provide several concrete subclasses, and you can write a custom
+    database backed by your preferred way to store a ``dict[bytes, set[bytes]]``.
+
+    Change listening
+    ----------------
+
+    An optional extension to |ExampleDatabase| is change listening. On databases
+    which support it, you can call |ExampleDatabase.add_listener| to add a function
+    as a change listener, which will be called whenever a value is added, deleted,
+    or moved. See |ExampleDatabase.add_listener| for details.
+
+    All databases in Hypothesis support change listening. Custom database classes
+    are not required to support change listening, unless they want to be
+    compatible with features that require change listening. Currently, no Hypothesis
+    features require change listening.
+
+    .. note::
+
+        Change listening is required by `HypoFuzz <https://hypofuzz.com/>`_.
+
+    Methods
+    -------
+
+    Required methods:
+
+    * |ExampleDatabase.save|
+    * |ExampleDatabase.fetch|
+    * |ExampleDatabase.delete|
+
+    Optional methods:
+
+    * |ExampleDatabase.move|
 
-    An ExampleDatabase maps each ``bytes`` key to many distinct ``bytes``
-    values, like a ``Mapping[bytes, set[bytes]]``.
+    Change listening methods:
+
+    * |ExampleDatabase.add_listener|
+    * |ExampleDatabase.remove_listener|
+    * |ExampleDatabase.clear_listeners|
+    * |ExampleDatabase._start_listening|
+    * |ExampleDatabase._stop_listening|
+    * |ExampleDatabase._broadcast_change|
     """
 
     def __init__(self) -> None:
@@ -159,7 +201,7 @@ def __init__(self) -> None:
     def save(self, key: bytes, value: bytes) -> None:
         """Save ``value`` under ``key``.
 
-        If this value is already present for this key, silently do nothing.
+        If ``value`` is already present in ``key``, silently do nothing.
         """
         raise NotImplementedError(f"{type(self).__name__}.save")
 
@@ -170,16 +212,18 @@ def fetch(self, key: bytes) -> Iterable[bytes]:
 
     @abc.abstractmethod
     def delete(self, key: bytes, value: bytes) -> None:
-        """Remove this value from this key.
+        """Remove ``value`` from ``key``.
 
-        If this value is not present, silently do nothing.
+        If ``value`` is not present in ``key``, silently do nothing.
         """
         raise NotImplementedError(f"{type(self).__name__}.delete")
 
     def move(self, src: bytes, dest: bytes, value: bytes) -> None:
-        """Move ``value`` from key ``src`` to key ``dest``. Equivalent to
-        ``delete(src, value)`` followed by ``save(src, value)``, but may
-        have a more efficient implementation.
+        """
+        Move ``value`` from key ``src`` to key ``dest``.
+
+        Equivalent to ``delete(src, value)`` followed by ``save(src, value)``,
+        but may have a more efficient implementation.
 
         Note that ``value`` will be inserted at ``dest`` regardless of whether
         it is currently present at ``src``.
@@ -191,7 +235,24 @@ def move(self, src: bytes, dest: bytes, value: bytes) -> None:
         self.save(dest, value)
 
     def add_listener(self, f: ListenerT, /) -> None:
-        """Add a change listener."""
+        """
+        Add a change listener. ``f`` will be called whenever a value is saved,
+        deleted, or moved in the database.
+
+        ``f`` can be called with two different event values:
+
+        * ``("save", (key, value))``
+        * ``("delete", (key, value))``
+
+        where ``key`` and ``value`` are both ``bytes``.
+
+        There is no ``move`` event. Instead, a move is broadcasted as a
+        ``delete`` event followed by a ``save`` event.
+
+        For the ``delete`` event, ``value`` may be ``None``. This might occur if
+        the database knows that a deletion has occurred in ``key``, but does not
+        know what value was deleted.
+        """
         had_listeners = bool(self._listeners)
         self._listeners.append(f)
         if not had_listeners:
@@ -199,8 +260,9 @@ def add_listener(self, f: ListenerT, /) -> None:
 
     def remove_listener(self, f: ListenerT, /) -> None:
         """
-        Remove a change listener. If the listener is not present, silently do
-        nothing.
+        Removes ``f`` from the list of change listeners.
+
+        If ``f`` is not in the list of change listeners, silently do nothing.
         """
         if f not in self._listeners:
             return
@@ -218,16 +280,20 @@ def clear_listeners(self) -> None:
     def _broadcast_change(self, event: ListenerEventT) -> None:
         """
         Called when a value has been either added to or deleted from a key in
-        the underlying database store. event_type is one of "save" or "delete".
+        the underlying database store. The possible values for ``event`` are:
+
+        * ``("save", (key, value))``
+        * ``("delete", (key, value))``
 
-        ``value`` may be ``None`` for ``event_type == "delete"``, which indicates
-        we don't know what value was deleted from the database.
+        ``value`` may be ``None`` for the ``delete`` event, indicating we know
+        that some value was deleted under this key, but not its exact value.
 
-        Note that you should not assume you are the only reference to the underlying
-        database store. For example, if two DirectoryBasedExampleDatabase reference
-        the same directory, _broadcast_change should be called whenever a file is
-        added or removed from the directory, even if that database was not responsible
-        for changing the file.
+        Note that you should not assume your instance is the only reference to
+        the underlying database store. For example, if two instances of
+        |DirectoryBasedExampleDatabase| reference the same directory,
+        _broadcast_change should be called whenever a file is added or removed
+        from the directory, even if that database was not responsible for
+        changing the file.
         """
         for listener in self._listeners:
             listener(event)
@@ -238,9 +304,10 @@ def _start_listening(self) -> None:
         have any change listeners. Intended to allow databases to wait to start
         expensive listening operations until necessary.
 
-        _start_listening and _stop_listening are guaranteed to alternate, so you
-        do not need to handle the case of multiple consecutive _start_listening
-        calls without an intermediate _stop_listening call.
+        ``_start_listening`` and ``_stop_listening`` are guaranteed to alternate,
+        so you do not need to handle the case of multiple consecutive
+        ``_start_listening`` calls without an intermediate ``_stop_listening``
+        call.
         """
         warnings.warn(
             f"{self.__class__} does not support listening for changes",
@@ -252,9 +319,10 @@ def _stop_listening(self) -> None:
         """
         Called whenever no change listeners remain on the database.
 
-        _stop_listening and _start_listening are guaranteed to alternate, so you
-        do not need to handle the case of multiple consecutive _stop_listening
-        calls without an intermediate _start_listening call.
+        ``_stop_listening`` and ``_start_listening`` are guaranteed to alternate,
+        so you do not need to handle the case of multiple consecutive
+        ``_stop_listening`` calls without an intermediate ``_start_listening``
+        call.
         """
         warnings.warn(
             f"{self.__class__} does not support stopping listening for changes",
@@ -264,7 +332,8 @@ def _stop_listening(self) -> None:
 
 
 class InMemoryExampleDatabase(ExampleDatabase):
-    """A non-persistent example database, implemented in terms of a dict of sets.
+    """A non-persistent example database, implemented in terms of an in-memory
+    dictionary.
 
     This can be useful if you call a test function several times in a single
     session, or for testing other database implementations, but because it
@@ -318,7 +387,7 @@ class DirectoryBasedExampleDatabase(ExampleDatabase):
 
     Each test corresponds to a directory, and each example to a file within that
     directory.  While the contents are fairly opaque, a
-    ``DirectoryBasedExampleDatabase`` can be shared by checking the directory
+    |DirectoryBasedExampleDatabase| can be shared by checking the directory
     into version control, for example with the following ``.gitignore``::
 
         # Ignore files cached by Hypothesis...
@@ -328,8 +397,7 @@ class DirectoryBasedExampleDatabase(ExampleDatabase):
 
     Note however that this only makes sense if you also pin to an exact version of
     Hypothesis, and we would usually recommend implementing a shared database with
-    a network datastore - see :class:`~hypothesis.database.ExampleDatabase`, and
-    the :class:`~hypothesis.database.MultiplexedDatabase` helper.
+    a network datastore - see |ExampleDatabase|, and the |MultiplexedDatabase| helper.
     """
 
     # we keep a database entry of the full values of all the database keys.
diff --git a/hypothesis-python/src/hypothesis/internal/conjecture/providers.py b/hypothesis-python/src/hypothesis/internal/conjecture/providers.py
index 04ecaaec98..6c07b9ebc8 100644
--- a/hypothesis-python/src/hypothesis/internal/conjecture/providers.py
+++ b/hypothesis-python/src/hypothesis/internal/conjecture/providers.py
@@ -202,7 +202,6 @@
     strings=SortedSet(_constant_strings),
 )
 
-
 _local_constants = Constants(
     integers=SortedSet(),
     floats=SortedSet(key=float_to_int),
@@ -250,11 +249,15 @@ def _get_local_constants() -> Constants:
     # with other threads loading a module before we set _sys_modules_len.
     if (sys_modules_len := len(sys.modules)) != _sys_modules_len:
         new_modules = set(sys.modules.values()) - _seen_modules
+        # Repeated SortedSet unions are expensive. Do the initial unions on a
+        # set(), then do a one-time union with _local_constants after.
+        new_constants = Constants()
         for module in new_modules:
-            if getattr(module, "__file__", None) is not None and is_local_module_file(
-                module.__file__
-            ):
-                _local_constants |= constants_from_module(module)
+            if (
+                module_file := getattr(module, "__file__", None)
+            ) is not None and is_local_module_file(module_file):
+                new_constants |= constants_from_module(module)
+        _local_constants |= new_constants
         _seen_modules.update(new_modules)
         _sys_modules_len = sys_modules_len
 
diff --git a/hypothesis-python/src/hypothesis/internal/constants_ast.py b/hypothesis-python/src/hypothesis/internal/constants_ast.py
index ca09c1bba2..381c8fc3e0 100644
--- a/hypothesis-python/src/hypothesis/internal/constants_ast.py
+++ b/hypothesis-python/src/hypothesis/internal/constants_ast.py
@@ -14,14 +14,12 @@
 import math
 import sys
 from ast import Constant, Expr, NodeVisitor, UnaryOp, USub
-from collections.abc import Iterator
+from collections.abc import Iterator, MutableSet
 from functools import lru_cache
 from itertools import chain
 from pathlib import Path
 from types import ModuleType
-from typing import TYPE_CHECKING, Union
-
-from sortedcontainers import SortedSet
+from typing import TYPE_CHECKING, Optional, Union
 
 import hypothesis
 from hypothesis.configuration import storage_directory
@@ -33,25 +31,27 @@
 
 ConstantT: "TypeAlias" = Union[int, float, bytes, str]
 
+# unfortunate collision with builtin. I don't want to name the init arg bytes_.
+bytesT = bytes
+
 
 class Constants:
     def __init__(
         self,
         *,
-        # spiritually "an AbstractSet, with the addition of a .add method"
-        integers: Union[set[int], SortedSet[int]] = None,
-        floats: Union[set[float], SortedSet[float]] = None,
-        bytes: Union[set[bytes], SortedSet[bytes]] = None,
-        strings: Union[set[str], SortedSet[str]] = None,
+        integers: Optional[MutableSet[int]] = None,
+        floats: Optional[MutableSet[float]] = None,
+        bytes: Optional[MutableSet[bytes]] = None,
+        strings: Optional[MutableSet[str]] = None,
     ):
-        self.integers = set() if integers is None else integers
-        self.floats = set() if floats is None else floats
-        self.bytes = set() if bytes is None else bytes
-        self.strings = set() if strings is None else strings
+        self.integers: MutableSet[int] = set() if integers is None else integers
+        self.floats: MutableSet[float] = set() if floats is None else floats
+        self.bytes: MutableSet[bytesT] = set() if bytes is None else bytes
+        self.strings: MutableSet[str] = set() if strings is None else strings
 
     def set_for_type(
         self, constant_type: Union[type[ConstantT], ChoiceTypeT]
-    ) -> Union[set[ConstantT], SortedSet[ConstantT]]:
+    ) -> Union[MutableSet[int], MutableSet[float], MutableSet[bytes], MutableSet[str]]:
         if constant_type is int or constant_type == "integer":
             return self.integers
         elif constant_type is float or constant_type == "float":
@@ -63,17 +63,17 @@ def set_for_type(
         raise ValueError(f"unknown constant_type {constant_type}")
 
     def add(self, constant: ConstantT) -> None:
-        self.set_for_type(type(constant)).add(constant)
+        self.set_for_type(type(constant)).add(constant)  # type: ignore
 
     def __contains__(self, constant: ConstantT) -> bool:
         return constant in self.set_for_type(type(constant))
 
     def __or__(self, other: "Constants") -> "Constants":
         return Constants(
-            integers=self.integers | other.integers,
-            floats=self.floats | other.floats,
-            bytes=self.bytes | other.bytes,
-            strings=self.strings | other.strings,
+            integers=self.integers | other.integers,  # type: ignore
+            floats=self.floats | other.floats,  # type: ignore
+            bytes=self.bytes | other.bytes,  # type: ignore
+            strings=self.strings | other.strings,  # type: ignore
         )
 
     def __iter__(self) -> Iterator[ConstantT]:


Reviews:
user: login: Zac-HD
body: see comments, then :shipit: 
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-14T08:54:32Z
user: login: Liam-DeVoe
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-14T19:41:23Z
user: login: Liam-DeVoe
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-14T19:41:24Z
user: login: zjmlianxi
state: COMMENTED
author_association: NONE
submitted_at: 2025-05-15T10:03:09Z
user: login: Zac-HD
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-15T23:28:23Z
user: login: Liam-DeVoe
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-15T23:45:24Z
user: login: Zac-HD
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-16T00:02:33Z
user: login: zjmlianxi
state: COMMENTED
author_association: NONE
submitted_at: 2025-05-16T02:45:06Z
user: login: Zac-HD
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-16T05:58:11Z

Review comments:
diff_hunk: @@ -549,12 +549,18 @@ def derandomize(self):
     @property
     def database(self):
         """
-        An instance of |ExampleDatabase| that will be
-        used to save examples to and load previous examples from. If ``None``,
-        no storage will be used.
+        An instance of |ExampleDatabase| that will be used to save examples to
+        and load previous examples from.
 
-        See the :ref:`database documentation <database>` for a list of built-in
-        example database implementations, and how to define custom implementations.
+        If not set, a |DirectoryBasedExampleDatabase| is created in the current
+        working directory under ``.hypothesis/examples``. If this location is
+        unusable, e.g. due to the lack of read or write permissions, Hypothesis
+        will emit a warning a fall back to an |InMemoryExampleDatabase|.
path: hypothesis-python/src/hypothesis/_settings.py
user: login: Zac-HD
body: ```suggestion
        will emit a warning and fall back to an |InMemoryExampleDatabase|.
```
created_at: 2025-05-14T08:27:29Z
author_association: MEMBER
diff_hunk: @@ -217,6 +217,16 @@
 _sys_modules_len: Optional[int] = None
 
 
+# Skip files over 200kb. For reference, the largest source file
+# in Hypothesis is strategies/_internal/core.py at 107kb at time
+# of writing.
+def _module_too_large(file: str) -> bool:
path: hypothesis-python/src/hypothesis/internal/conjecture/providers.py
user: login: Zac-HD
body: I'd rather not skip for filesize - our filesystem cache should make this cheap on the second call anyway.
created_at: 2025-05-14T08:32:20Z
author_association: MEMBER
diff_hunk: @@ -33,25 +31,27 @@
 
 ConstantT: "TypeAlias" = Union[int, float, bytes, str]
 
+# unfortunate collision with builtin. I don't want to name the init arg bytes_.
+bytesT = bytes
path: hypothesis-python/src/hypothesis/internal/constants_ast.py
user: login: Zac-HD
body: ~~what about `byteses`, precious?~~
created_at: 2025-05-14T08:33:01Z
author_association: MEMBER
diff_hunk: @@ -146,10 +146,47 @@ def __call__(self, *args: Any, **kwargs: Any) -> "ExampleDatabase":
 
 
 class ExampleDatabase(metaclass=_EDMeta):
-    """An abstract base class for storing examples in Hypothesis' internal format.
+    """
+    A Hypothesis database, for use in the |settings.database| setting.
+
+    A Hypothesis database acts like a ``Mapping[bytes, set[bytes]]``, mapping
+    each ``bytes`` key to a set of ``bytes`` values.
+
+    Change listening
+    ----------------
+
+    An optional extension to |ExampleDatabase| is change listening. On databases
+    which support it, you can call |ExampleDatabase.add_listener| to add a function
+    as a change listener, which will be called whenever a value is added, deleted,
+    or moved. See |ExampleDatabase.add_listener| for details.
+
+    All databases in Hypothesis support change listening. Custom database classes
+    are not required to sipport change listening, unless they want to be
path: hypothesis-python/src/hypothesis/database.py
user: login: Zac-HD
body: ```suggestion
    are not required to support change listening, unless they want to be
```
created_at: 2025-05-14T08:34:59Z
author_association: MEMBER
diff_hunk: @@ -146,10 +146,47 @@ def __call__(self, *args: Any, **kwargs: Any) -> "ExampleDatabase":
 
 
 class ExampleDatabase(metaclass=_EDMeta):
-    """An abstract base class for storing examples in Hypothesis' internal format.
+    """
+    A Hypothesis database, for use in the |settings.database| setting.
+
+    A Hypothesis database acts like a ``Mapping[bytes, set[bytes]]``, mapping
+    each ``bytes`` key to a set of ``bytes`` values.
+
+    Change listening
+    ----------------
+
+    An optional extension to |ExampleDatabase| is change listening. On databases
+    which support it, you can call |ExampleDatabase.add_listener| to add a function
+    as a change listener, which will be called whenever a value is added, deleted,
+    or moved. See |ExampleDatabase.add_listener| for details.
+
+    All databases in Hypothesis support change listening. Custom database classes
+    are not required to sipport change listening, unless they want to be
+    compatible with features that require change listening. Currently, no Hypothesis
+    features require change listening.
+
+    .. note::
+
+        Change listening is required by `HypoFuzz <https://hypofuzz.com/>`_.
+
+    Methods
+    -------
+
+    Required methods:
+
+    * |ExampleDatabase.save|
+    * |ExampleDatabase.fetch|
+    * |ExampleDatabase.delete|
+
+    Optional methods:
+
+    * |ExampleDatabase.move|
 
-    An ExampleDatabase maps each ``bytes`` key to many distinct ``bytes``
-    values, like a ``Mapping[bytes, set[bytes]]``.
+    Change listening methods:
+
+    * |ExampleDatabase.add_listener|
+    * |ExampleDatabase.remove_listener|
+    * |ExampleDatabase.clear_listeners|
path: hypothesis-python/src/hypothesis/database.py
user: login: Zac-HD
body: - I'd just list these under "Optional methods"
- shouldn't we be listing `._start_listening()` and `._stop_listening()` as the things-to-implement here, and autodoc-ing them?  (or direct calls to `._broadcast_change()`, if it's cheap)  The add/remove/clear methods can just be inherited, I think.
created_at: 2025-05-14T08:42:45Z
author_association: MEMBER
diff_hunk: @@ -146,10 +146,47 @@ def __call__(self, *args: Any, **kwargs: Any) -> "ExampleDatabase":
 
 
 class ExampleDatabase(metaclass=_EDMeta):
-    """An abstract base class for storing examples in Hypothesis' internal format.
+    """
+    A Hypothesis database, for use in the |settings.database| setting.
+
+    A Hypothesis database acts like a ``Mapping[bytes, set[bytes]]``, mapping
+    each ``bytes`` key to a set of ``bytes`` values.
path: hypothesis-python/src/hypothesis/database.py
user: login: Zac-HD
body: I think we want to open with a short motivation, and the type is really just as a "you can implement this" hint.  Something like

> Using an ExampleDatabase (via the database= setting) allows Hypothesis to remember and replay failing examples, and ensure that your tests are never flaky.  We provide several concrete subclasses, and you can write a custom dabase backed by your preferred way to store a `dict[bytes, set[bytes]]`.
created_at: 2025-05-14T08:51:30Z
author_association: MEMBER
diff_hunk: @@ -146,10 +146,47 @@ def __call__(self, *args: Any, **kwargs: Any) -> "ExampleDatabase":
 
 
 class ExampleDatabase(metaclass=_EDMeta):
-    """An abstract base class for storing examples in Hypothesis' internal format.
+    """
+    A Hypothesis database, for use in the |settings.database| setting.
+
+    A Hypothesis database acts like a ``Mapping[bytes, set[bytes]]``, mapping
+    each ``bytes`` key to a set of ``bytes`` values.
+
+    Change listening
+    ----------------
+
+    An optional extension to |ExampleDatabase| is change listening. On databases
+    which support it, you can call |ExampleDatabase.add_listener| to add a function
+    as a change listener, which will be called whenever a value is added, deleted,
+    or moved. See |ExampleDatabase.add_listener| for details.
+
+    All databases in Hypothesis support change listening. Custom database classes
+    are not required to sipport change listening, unless they want to be
+    compatible with features that require change listening. Currently, no Hypothesis
+    features require change listening.
+
+    .. note::
+
+        Change listening is required by `HypoFuzz <https://hypofuzz.com/>`_.
+
+    Methods
+    -------
+
+    Required methods:
+
+    * |ExampleDatabase.save|
+    * |ExampleDatabase.fetch|
+    * |ExampleDatabase.delete|
+
+    Optional methods:
+
+    * |ExampleDatabase.move|
 
-    An ExampleDatabase maps each ``bytes`` key to many distinct ``bytes``
-    values, like a ``Mapping[bytes, set[bytes]]``.
+    Change listening methods:
+
+    * |ExampleDatabase.add_listener|
+    * |ExampleDatabase.remove_listener|
+    * |ExampleDatabase.clear_listeners|
path: hypothesis-python/src/hypothesis/database.py
user: login: Liam-DeVoe
body: "optional" implies you should override/implement them, which isn't necessarily the case for all of the change listener methods. You're probably overriding `_{start,stop}_listening`, but you're probably not overriding add/remove/clear or broadcast_change, and either putting every change listener method into "optional" or putting just start/stop into "optional" and separating `_broadcast_change` feels weird. 
created_at: 2025-05-14T19:41:23Z
author_association: MEMBER
diff_hunk: @@ -217,6 +217,16 @@
 _sys_modules_len: Optional[int] = None
 
 
+# Skip files over 200kb. For reference, the largest source file
+# in Hypothesis is strategies/_internal/core.py at 107kb at time
+# of writing.
+def _module_too_large(file: str) -> bool:
path: hypothesis-python/src/hypothesis/internal/conjecture/providers.py
user: login: Liam-DeVoe
body: Unless running in CI, or some other ephemeral filesystem..I think the odds of a >1mb autogenerated python file appearing in some project is nonzero (e.g. matplotlib autogenerates, though I don't know what size the files get to).

I'll drop this for this PR though
created_at: 2025-05-14T19:41:24Z
author_association: MEMBER
diff_hunk: @@ -217,6 +217,16 @@
 _sys_modules_len: Optional[int] = None
 
 
+# Skip files over 200kb. For reference, the largest source file
+# in Hypothesis is strategies/_internal/core.py at 107kb at time
+# of writing.
+def _module_too_large(file: str) -> bool:
path: hypothesis-python/src/hypothesis/internal/conjecture/providers.py
user: login: zjmlianxi
body: Please forgive my continued discussion of this issue.   

After removing the "skip files over 200KB" feature in version 6.131.17, the execution time for `_get_local_constants` in my repository increased from 3 seconds to 15 seconds. This file-filtering function is critical for repositories like mine, which contain numerous large data files. I identified the five files with the highest number of constants in their AST results in my repository, with counts of 53,157, 20,825, 12,769, 8,430, and 7,407, and file sizes of 11MB, 2.5MB, 6.9MB, 13MB, and 1.9MB, respectively.  

In version 6.131.15, merging sets took approximately 45 times longer than AST processing.  
![image](https://github.com/user-attachments/assets/0afaef40-5573-4b4d-81d7-a31aaefc81af)  

By optimizing the data structure, version 6.131.17 reduced the total execution time from 180 seconds to 15 seconds, with set merging now taking about twice as long as AST processing.  
![image](https://github.com/user-attachments/assets/963e7bd4-ed23-4b7e-8557-cdcd5808c5a2)  

The proportion of time spent on AST processing has increased. Should we reconsider filtering files before AST processing? Is it necessary to collect all constants from a repository, especially when the number is extremely large? How would incomplete constants in repository code affect test validity?  

Although the difference between 15 seconds and 3 seconds for a single operation seems small, it could significantly impact performance in a busy continuous integration environment.  
created_at: 2025-05-15T10:03:08Z
author_association: NONE
diff_hunk: @@ -217,6 +217,16 @@
 _sys_modules_len: Optional[int] = None
 
 
+# Skip files over 200kb. For reference, the largest source file
+# in Hypothesis is strategies/_internal/core.py at 107kb at time
+# of writing.
+def _module_too_large(file: str) -> bool:
path: hypothesis-python/src/hypothesis/internal/conjecture/providers.py
user: login: Zac-HD
body: Nothing to forgive - we really value this feedback.  Collecting all constants is not necessary, but does make us more likely to find some classes of bugs; we also strongly prefer to avoid making features configurable if we can possibly avoid it because the sheer number would get overwhelming.

It seems clear that we need some way to either speed up this case or quickly decide to skip very large files.  @tybug what would you think about aborting AST walking after we find 1024 constants or something?  (by default, we want hypofuzz to be able to set a higher limit).  ISTM that it's unlikely to be especially helpful when the pool is enormous, just because the chance of picking the _right_ constant is lower.
created_at: 2025-05-15T23:28:23Z
author_association: MEMBER
diff_hunk: @@ -217,6 +217,16 @@
 _sys_modules_len: Optional[int] = None
 
 
+# Skip files over 200kb. For reference, the largest source file
+# in Hypothesis is strategies/_internal/core.py at 107kb at time
+# of writing.
+def _module_too_large(file: str) -> bool:
path: hypothesis-python/src/hypothesis/internal/conjecture/providers.py
user: login: Liam-DeVoe
body: I was thinking the same re: aborting after n constants. This won't help ast *parse*, though - I'd still advocate for a reasonable upper kb limit at which we don't attempt to parse at all. (Hypofuzz will want to customize all of this etc).
created_at: 2025-05-15T23:45:24Z
author_association: MEMBER
diff_hunk: @@ -217,6 +217,16 @@
 _sys_modules_len: Optional[int] = None
 
 
+# Skip files over 200kb. For reference, the largest source file
+# in Hypothesis is strategies/_internal/core.py at 107kb at time
+# of writing.
+def _module_too_large(file: str) -> bool:
path: hypothesis-python/src/hypothesis/internal/conjecture/providers.py
user: login: Zac-HD
body: Yeah, maybe 2MB for the limit?  
created_at: 2025-05-16T00:02:33Z
author_association: MEMBER
diff_hunk: @@ -217,6 +217,16 @@
 _sys_modules_len: Optional[int] = None
 
 
+# Skip files over 200kb. For reference, the largest source file
+# in Hypothesis is strategies/_internal/core.py at 107kb at time
+# of writing.
+def _module_too_large(file: str) -> bool:
path: hypothesis-python/src/hypothesis/internal/conjecture/providers.py
user: login: zjmlianxi
body: I’ve considered stopping iteration when, for example, `len(new_constants)` exceeds `MAX_CONSTANT_SIZE` to optimize the process. However, I’m unsure whether we should also account for specific constant types, such as `len(new_constants.integers) > MAX_CONSTANT_TYPE_SIZE,` `len(new_constants.floats) > MAX_CONSTANT_TYPE_SIZE`, `len(new_constants.bytes) > MAX_CONSTANT_TYPE_SIZE`, and `len(new_constants.strings) > MAX_CONSTANT_TYPE_SIZE.` My repository contains 202,830 constants (7 bytes, 20,335 floats, 141,855 integers, and 40,633 strings). Since byte collection is consistently insufficient, early termination may not be effective. If we only check `len(new_constants)`, we might end up with a collection like 0 bytes, 0 floats, 1,024 integers, and 0 strings, and I’m unsure if such a result is valuable.  

I measured the processing time under various file size limits for reference (results varied widely; the shortest time from 5 tests was selected).  
| NO LIMIT | 2M | 1M | 512K | 256K |
| 11.35s | 5.73s | 4.22s | 3.78s | 3.03s |
Setting the file size limit to 2MB significantly reduces processing time.

created_at: 2025-05-16T02:45:06Z
author_association: NONE
diff_hunk: @@ -217,6 +217,16 @@
 _sys_modules_len: Optional[int] = None
 
 
+# Skip files over 200kb. For reference, the largest source file
+# in Hypothesis is strategies/_internal/core.py at 107kb at time
+# of writing.
+def _module_too_large(file: str) -> bool:
path: hypothesis-python/src/hypothesis/internal/conjecture/providers.py
user: login: Zac-HD
body: > we might end up with a collection like 0 bytes, 0 floats, 1,024 integers, and 0 strings, and I’m unsure if such a result is valuable.

I was thinking that we'd abandon the partial collection, adding no constants for any file with too many.
created_at: 2025-05-16T05:58:11Z
author_association: MEMBER

PR comments:
user: login: zjmlianxi
created_at: 2025-05-14T03:50:54Z
body: This version reduces test time from 179 seconds to 3 seconds in https://github.com/HypothesisWorks/hypothesis/pull/4366#issuecomment-2876240115. It's now acceptable for me. Great work!

By the way, catching `os.getsize` errors is necessary. I found that files in a zip folder raise a `NotADirectoryError`。
author_association: NONE
user: login: Liam-DeVoe
created_at: 2025-05-14T04:48:46Z
body: I appreciate your detailed information! I'd certainly be happy to try to get that number lower, if you have any further performance pointers.
author_association: MEMBER