Repository: HypothesisWorks/hypothesis

Issue:
Deprecate find()
`find(strategy, condition)` can be replaced with `strategy.filter(condition).example()`.  It does some other stuff too, but (a) `find()` is actually not very useful, and (b) it's a messy and these days second-class execution engine compared to `@given` - see #1609 and in particular #1213.

This is basically a straight clean-up exercise to reduce the API surface that we have to maintain, and ensure that each piece is orthogonal.


Discussion:
I had a look at this and ran into a couple of issues:

* `example()` uses `find()` under the hood, but unlike `find()` it only draws the first example. It doesn't do any shrinking behaviour – but we have test helpers like `minimal()` that make use of that shrinking behaviour. We could inline it twice, but that feels icky.

* When I converted `find()` to use `strategy.filter(condition).example()` under the hood, I got an error in `test_core.py`:

      hypothesis.errors.InvalidArgument: Cannot call filter on a DataStrategy. You should probably be using @composite for whatever it is you're trying to do.

I don’t have time for more work tonight, but there are two things to think about for the next person to work on this.
Even if we keep using `find()` internally for a while, deprecating it as a public interface would be a valuable contribution.  In practice that would mean moving `find()` to a non-public namespace, replacing it with a shim that calls `note_deprecation` and then calls the non-public version, and changing our internal uses to the private version.

Long term we would certainly like to remove it entirely, but the internal changes can happen later.
...turns out that "using `find` internally" causes a bug in repeated use of `.example()` for certain strategies:

```python
import hypothesis.strategies as st
strat = st.deferred(
    lambda : st.one_of(
        st.integers(),
        st.dictionaries(st.integers(), strat, min_size=3)
    )
)
for i in range(1000):
    x = strat.example()
```

leads to `Flaky: Inconsistent test results! Test case was Conclusion(status=Status.VALID, interesting_origin=None) on first run but Conclusion(status=Status.INTERESTING, interesting_origin=None) on second`
Implementing `find()` in terms of `@given` is pretty easy: https://github.com/HypothesisWorks/hypothesis/compare/master...Zac-HD:deprecate-find

However this is followed by a huge pile of individually-minor but collectively annoying problems, so I'm inclined to switch over the internals *and* deprecated it so we get the smaller API surface too.  Timeline: fairly soon, so the six-month deprecation period allows us to delete it in an early-2020 major version update.

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/_strategies.py b/hypothesis-python/src/hypothesis/_strategies.py
--- a/hypothesis-python/src/hypothesis/_strategies.py
+++ b/hypothesis-python/src/hypothesis/_strategies.py
@@ -2154,10 +2154,6 @@ def data():
     """This isn't really a normal strategy, but instead gives you an object
     which can be used to draw data interactively from other strategies.
 
-    It can only be used within :func:`@given <hypothesis.given>`, not
-    :func:`find() <hypothesis.find>`. This is because the lifetime
-    of the object cannot outlast the test body.
-
     See :ref:`the rest of the documentation <interactive-draw>` for more
     complete information.
 
diff --git a/hypothesis-python/src/hypothesis/core.py b/hypothesis-python/src/hypothesis/core.py
--- a/hypothesis-python/src/hypothesis/core.py
+++ b/hypothesis-python/src/hypothesis/core.py
@@ -42,6 +42,7 @@
     PrintSettings,
     Verbosity,
     local_settings,
+    note_deprecation,
     settings as Settings,
 )
 from hypothesis.control import BuildContext
@@ -1054,6 +1055,11 @@ def find(
     # type: (...) -> Any
     """Returns the minimal example from the given strategy ``specifier`` that
     matches the predicate function ``condition``."""
+    note_deprecation(
+        "`find(s, f)` is deprecated, because it is rarely used but takes "
+        "ongoing work to maintain as we upgrade other parts of Hypothesis.",
+        since="RELEASEDAY",
+    )
     if settings is None:
         settings = Settings(max_examples=2000)
     settings = Settings(settings, suppress_health_check=HealthCheck.all())


Test patch:
diff --git a/hypothesis-python/tests/cover/test_arbitrary_data.py b/hypothesis-python/tests/cover/test_arbitrary_data.py
--- a/hypothesis-python/tests/cover/test_arbitrary_data.py
+++ b/hypothesis-python/tests/cover/test_arbitrary_data.py
@@ -21,7 +21,7 @@
 
 from hypothesis import find, given, reporting, strategies as st
 from hypothesis.errors import InvalidArgument
-from tests.common.utils import capture_out, raises
+from tests.common.utils import capture_out, checks_deprecated_behaviour, raises
 
 
 @given(st.integers(), st.data())
@@ -82,6 +82,7 @@ def test(data1, data2):
     assert "Draw 2: 0" in result
 
 
+@checks_deprecated_behaviour
 def test_errors_when_used_in_find():
     with raises(InvalidArgument):
         find(st.data(), lambda x: x.draw(st.booleans()))
diff --git a/hypothesis-python/tests/cover/test_core.py b/hypothesis-python/tests/cover/test_core.py
--- a/hypothesis-python/tests/cover/test_core.py
+++ b/hypothesis-python/tests/cover/test_core.py
@@ -23,8 +23,10 @@
 import hypothesis.strategies as s
 from hypothesis import find, given, reject, settings
 from hypothesis.errors import NoSuchExample, Unsatisfiable
+from tests.common.utils import checks_deprecated_behaviour
 
 
+@checks_deprecated_behaviour
 def test_stops_after_max_examples_if_satisfying():
     tracker = []
 
@@ -40,6 +42,7 @@ def track(x):
     assert len(tracker) == max_examples
 
 
+@checks_deprecated_behaviour
 def test_stops_after_ten_times_max_examples_if_not_satisfying():
     count = [0]
 
@@ -98,5 +101,12 @@ def inner(x):
     assert len([x for x in values if x > 100]) == 1
 
 
+@checks_deprecated_behaviour
 def test_can_find_with_db_eq_none():
     find(s.integers(), bool, settings(database=None))
+
+
+@checks_deprecated_behaviour
+def test_no_such_example():
+    with pytest.raises(NoSuchExample):
+        find(s.none(), bool, database_key=b"no such example")
diff --git a/hypothesis-python/tests/cover/test_random_module.py b/hypothesis-python/tests/cover/test_random_module.py
--- a/hypothesis-python/tests/cover/test_random_module.py
+++ b/hypothesis-python/tests/cover/test_random_module.py
@@ -26,7 +26,7 @@
 from hypothesis.errors import InvalidArgument
 from hypothesis.internal import entropy
 from hypothesis.internal.entropy import deterministic_PRNG
-from tests.common.utils import capture_out
+from tests.common.utils import capture_out, checks_deprecated_behaviour
 
 
 def test_can_seed_random():
@@ -131,6 +131,7 @@ def test(r):
         assert state_a != state_b
 
 
+@checks_deprecated_behaviour
 def test_find_does_not_pollute_state():
     with deterministic_PRNG():
 
diff --git a/hypothesis-python/tests/cover/test_runner_strategy.py b/hypothesis-python/tests/cover/test_runner_strategy.py
--- a/hypothesis-python/tests/cover/test_runner_strategy.py
+++ b/hypothesis-python/tests/cover/test_runner_strategy.py
@@ -24,6 +24,7 @@
 from hypothesis import find, given, strategies as st
 from hypothesis.errors import InvalidArgument
 from hypothesis.stateful import RuleBasedStateMachine, rule
+from tests.common.utils import checks_deprecated_behaviour
 
 
 def test_cannot_use_without_a_runner():
@@ -35,11 +36,13 @@ def f(x):
         f()
 
 
+@checks_deprecated_behaviour
 def test_cannot_use_in_find_without_default():
     with pytest.raises(InvalidArgument):
         find(st.runner(), lambda x: True)
 
 
+@checks_deprecated_behaviour
 def test_is_default_in_find():
     t = object()
     assert find(st.runner(t), lambda x: True) == t
diff --git a/hypothesis-python/tests/cover/test_validation.py b/hypothesis-python/tests/cover/test_validation.py
--- a/hypothesis-python/tests/cover/test_validation.py
+++ b/hypothesis-python/tests/cover/test_validation.py
@@ -224,6 +224,7 @@ def test(x, y):
         test()
 
 
+@checks_deprecated_behaviour
 def test_cannot_find_non_strategies():
     with pytest.raises(InvalidArgument):
         find(bool, bool)
diff --git a/hypothesis-python/tests/cover/test_verbosity.py b/hypothesis-python/tests/cover/test_verbosity.py
--- a/hypothesis-python/tests/cover/test_verbosity.py
+++ b/hypothesis-python/tests/cover/test_verbosity.py
@@ -24,7 +24,7 @@
 from hypothesis.reporting import default as default_reporter, with_reporter
 from hypothesis.strategies import booleans, integers, lists
 from tests.common.debug import minimal
-from tests.common.utils import capture_out, fails
+from tests.common.utils import capture_out, checks_deprecated_behaviour, fails
 
 
 @contextmanager
@@ -72,6 +72,7 @@ def test_includes_progress_in_verbose_mode():
     assert u"Falsifying example: " in out
 
 
+@checks_deprecated_behaviour
 def test_prints_initial_attempts_on_find():
 
     with capture_verbosity() as o:
diff --git a/hypothesis-python/tests/nocover/test_boundary_exploration.py b/hypothesis-python/tests/nocover/test_boundary_exploration.py
--- a/hypothesis-python/tests/nocover/test_boundary_exploration.py
+++ b/hypothesis-python/tests/nocover/test_boundary_exploration.py
@@ -20,8 +20,9 @@
 import pytest
 
 import hypothesis.strategies as st
-from hypothesis import HealthCheck, Verbosity, find, given, reject, settings
-from hypothesis.errors import NoSuchExample
+from hypothesis import HealthCheck, Verbosity, given, reject, settings
+from hypothesis.errors import Unsatisfiable
+from tests.common.debug import minimal
 from tests.common.utils import no_shrink
 
 
@@ -38,12 +39,12 @@ def predicate(x):
             return cache.setdefault(x, data.draw(st.booleans(), label=repr(x)))
 
     try:
-        find(
+        minimal(
             strat,
             predicate,
             settings=settings(
                 max_examples=10, database=None, verbosity=Verbosity.quiet
             ),
         )
-    except NoSuchExample:
+    except Unsatisfiable:
         reject()
diff --git a/hypothesis-python/tests/nocover/test_database_usage.py b/hypothesis-python/tests/nocover/test_database_usage.py
--- a/hypothesis-python/tests/nocover/test_database_usage.py
+++ b/hypothesis-python/tests/nocover/test_database_usage.py
@@ -22,13 +22,18 @@
 from hypothesis.database import InMemoryExampleDatabase
 from hypothesis.errors import NoSuchExample, Unsatisfiable
 from hypothesis.internal.compat import hbytes
-from tests.common.utils import all_values, non_covering_examples
+from tests.common.utils import (
+    all_values,
+    checks_deprecated_behaviour,
+    non_covering_examples,
+)
 
 
 def has_a_non_zero_byte(x):
     return any(hbytes(x))
 
 
+@checks_deprecated_behaviour
 def test_saves_incremental_steps_in_database():
     key = b"a database key"
     database = InMemoryExampleDatabase()
@@ -41,6 +46,7 @@ def test_saves_incremental_steps_in_database():
     assert len(all_values(database)) > 1
 
 
+@checks_deprecated_behaviour
 def test_clears_out_database_as_things_get_boring():
     key = b"a database key"
     database = InMemoryExampleDatabase()
@@ -73,6 +79,7 @@ def stuff():
         assert False
 
 
+@checks_deprecated_behaviour
 def test_trashes_invalid_examples():
     key = b"a database key"
     database = InMemoryExampleDatabase()
@@ -97,6 +104,7 @@ def stuff():
     assert len(all_values(database)) < original
 
 
+@checks_deprecated_behaviour
 def test_respects_max_examples_in_database_usage():
     key = b"a database key"
     database = InMemoryExampleDatabase()
diff --git a/hypothesis-python/tests/nocover/test_deferred_errors.py b/hypothesis-python/tests/nocover/test_deferred_errors.py
--- a/hypothesis-python/tests/nocover/test_deferred_errors.py
+++ b/hypothesis-python/tests/nocover/test_deferred_errors.py
@@ -23,6 +23,7 @@
 from hypothesis import find, given
 from hypothesis._strategies import defines_strategy
 from hypothesis.errors import InvalidArgument
+from tests.common.utils import checks_deprecated_behaviour
 
 
 def test_does_not_error_on_initial_calculation():
@@ -49,6 +50,7 @@ def test(x):
         test()
 
 
+@checks_deprecated_behaviour
 def test_errors_on_find():
     s = st.lists(st.integers(), min_size=5, max_size=2)
     with pytest.raises(InvalidArgument):
diff --git a/hypothesis-python/tests/nocover/test_find.py b/hypothesis-python/tests/nocover/test_find.py
--- a/hypothesis-python/tests/nocover/test_find.py
+++ b/hypothesis-python/tests/nocover/test_find.py
@@ -25,6 +25,7 @@
 from hypothesis.errors import NoSuchExample
 from hypothesis.strategies import booleans, dictionaries, floats, integers, lists
 from tests.common.debug import minimal
+from tests.common.utils import checks_deprecated_behaviour
 
 
 def test_can_find_an_int():
@@ -49,6 +50,7 @@ def test_can_find_nans():
         assert 2 <= len(x) <= 3
 
 
+@checks_deprecated_behaviour
 def test_condition_is_name():
     settings = Settings(max_examples=20)
     with pytest.raises(NoSuchExample) as e:
diff --git a/hypothesis-python/tests/nocover/test_imports.py b/hypothesis-python/tests/nocover/test_imports.py
--- a/hypothesis-python/tests/nocover/test_imports.py
+++ b/hypothesis-python/tests/nocover/test_imports.py
@@ -22,8 +22,9 @@
 
 
 def test_can_star_import_from_hypothesis():
-    find(
-        lists(integers()),
-        lambda x: sum(x) > 1,
-        settings=settings(max_examples=10000, verbosity=Verbosity.quiet),
-    )
+    @given(lists(integers()))
+    @settings(max_examples=10000, verbosity=Verbosity.quiet)
+    def f(x):
+        pass
+
+    f()
diff --git a/hypothesis-python/tests/nocover/test_randomization.py b/hypothesis-python/tests/nocover/test_randomization.py
--- a/hypothesis-python/tests/nocover/test_randomization.py
+++ b/hypothesis-python/tests/nocover/test_randomization.py
@@ -23,9 +23,10 @@
 
 import hypothesis.strategies as st
 from hypothesis import Verbosity, find, given, settings
-from tests.common.utils import no_shrink
+from tests.common.utils import checks_deprecated_behaviour, no_shrink
 
 
+@checks_deprecated_behaviour
 def test_seeds_off_random():
     s = settings(phases=no_shrink, database=None)
     r = random.getstate()
diff --git a/hypothesis-python/tests/py2/test_generate_ints.py b/hypothesis-python/tests/py2/test_generate_ints.py
--- a/hypothesis-python/tests/py2/test_generate_ints.py
+++ b/hypothesis-python/tests/py2/test_generate_ints.py
@@ -20,10 +20,10 @@
 import pytest
 
 import hypothesis.strategies as st
-from hypothesis import find
+from tests.common.debug import minimal
 
 
 @pytest.mark.parametrize("kwargs", [{}, {"min_value": 0}, {"max_value": 0}])
 def test_generates_an_int_as_the_min_value(kwargs):
-    n = find(st.integers(**kwargs), lambda x: True)
+    n = minimal(st.integers(**kwargs))
     assert isinstance(n, int)