Repository: HypothesisWorks/hypothesis

Issue:
max_examples in @hypothesis.settings not working
I am not entirely sure how `@hypothesis.settings(max_examples=...)` should work.
My knowledge is that it specifies the number of examples to create.

I have:

```
@given(data=st.integers(), ndim=st.integers(min_value=5, max_value=10))
@settings(max_examples=4)
def test_settings(data, ndim):
    print("DIM ", ndim)
    ...
```
I see 12 results of "DIM", sometimes it's 3 (in a deterministic way). I am not sure why since I have specified 3 to be the max number of examples.


Discussion:

Resolving patch:
diff --git a/hypothesis-python/setup.py b/hypothesis-python/setup.py
--- a/hypothesis-python/setup.py
+++ b/hypothesis-python/setup.py
@@ -102,7 +102,7 @@ def local_file(name):
     extras_require=extras,
     install_requires=[
         "attrs>=19.2.0",
-        "exceptiongroup>=1.0.0rc8 ; python_version<'3.11'",
+        "exceptiongroup>=1.0.0 ; python_version<'3.11'",
         "sortedcontainers>=2.1.0,<3.0.0",
     ],
     python_requires=">=3.7",
diff --git a/hypothesis-python/src/hypothesis/_settings.py b/hypothesis-python/src/hypothesis/_settings.py
--- a/hypothesis-python/src/hypothesis/_settings.py
+++ b/hypothesis-python/src/hypothesis/_settings.py
@@ -8,7 +8,7 @@
 # v. 2.0. If a copy of the MPL was not distributed with this file, You can
 # obtain one at https://mozilla.org/MPL/2.0/.
 
-"""A module controlling settings for Hypothesis to use in falsification.
+"""The settings module configures runtime options for Hypothesis.
 
 Either an explicit settings object can be used or the default object on
 this module can be modified.
@@ -120,9 +120,8 @@ def __setattr__(cls, name, value):
 
 
 class settings(metaclass=settingsMeta):
-    """A settings object controls a variety of parameters that are used in
-    falsification. These may control both the falsification strategy and the
-    details of the data that is generated.
+    """A settings object configures options including verbosity, runtime controls,
+    persistence, determinism, and more.
 
     Default values are picked up from the settings.default object and
     changes made there will be picked up in newly created settings.
@@ -364,7 +363,12 @@ def _max_examples_validator(x):
     validator=_max_examples_validator,
     description="""
 Once this many satisfying examples have been considered without finding any
-counter-example, falsification will terminate.
+counter-example, Hypothesis will stop looking.
+
+Note that we might call your test function fewer times if we find a bug early
+or can tell that we've exhausted the search space; or more if we discard some
+examples due to use of .filter(), assume(), or a few other things that can
+prevent the test case from completing successfully.
 
 The default value is chosen to suit a workflow where the test will be part of
 a suite that is regularly executed locally or on a CI server, balancing total
diff --git a/hypothesis-python/src/hypothesis/strategies/_internal/core.py b/hypothesis-python/src/hypothesis/strategies/_internal/core.py
--- a/hypothesis-python/src/hypothesis/strategies/_internal/core.py
+++ b/hypothesis-python/src/hypothesis/strategies/_internal/core.py
@@ -1497,6 +1497,7 @@ class DrawFn(Protocol):
         def list_and_index(draw: DrawFn) -> Tuple[int, str]:
             i = draw(integers())  # type inferred as 'int'
             s = draw(text())  # type inferred as 'str'
+            return i, s
 
     """
 


Test patch: