Repository: HypothesisWorks/hypothesis

Issue:
RuleBasedStateMachine is prone to Unsatisfiable errors
A state machine frequently ends up as `hypothesis.errors.Unsatisfiable` when the input strategies to its rules themselves are frequently marked as invalid.

For example,

```python
class MyStateMachine(RuleBasedStateMachine):

    @rule(data=st.lists(st.text(), min_size=5, unique=True))
    def rule1(self, data):
        assert data is not None


TestMyStateMachine = MyStateMachine.TestCase
```

Yields:

```
E   hypothesis.errors.Unsatisfiable: Unable to satisfy assumptions of run_state_machine

venv/lib/python3.10/site-packages/hypothesis/stateful.py:112: Unsatisfiable
--------------------------------------------------------------------------------------------------------- Hypothesis ---------------------------------------------------------------------------------------------------------
You can add @seed(187837441642656874040035655188699191288) to this test or run pytest with --hypothesis-seed=187837441642656874040035655188699191288 to reproduce this failure.
=================================================================================================== Hypothesis Statistics ====================================================================================================
myproj/test_hypothesis.py::TestMyStateMachine::runTest:

  - during generate phase (31.27 seconds):
    - Typical runtimes: ~ 0-55 ms, of which < 1ms in data generation
    - 0 passing examples, 0 failing examples, 1000 invalid examples
    - Events:
      * 56.80%, Retried draw from text().filter(not_yet_in_unique_list) to satisfy filter
      * 43.20%, Aborted test because unable to satisfy just(Rule(targets=(), function=rule1, arguments={'data': lists(text(), min_size=5, unique=True)}, preconditions=(), bundles=())).filter(RuleStrategy(machine=MyStateMachine({...})).is_valid).filter(lambda r: <unknown>)

  - Stopped because settings.max_examples=100, but < 10% of examples satisfied assumptions
```

This is the minimal example I could find, my actual state machine is much larger but exhibits the same error.

The state machine works fine when used with simpler or more reliable strategy. '

The `st.lists(st.text(), min_size=5, unique=True)` strategy also fine when used with `@given` (i.e. not in a state machine), although the stats show that it does frequently return invalid examples.


Discussion:
@Zac-HD I see why you tagged this `performance` but I do want to note that this is a pretty big obstacle to us being able to use state machines correctly.

In order to work around this issue, we need to make sure that all our composite strategies rarely or never mark examples as invalid; i.e, we basically cannot use `assume` or `filter`, or any of the built-in strategies that leverage filtering. 

Specifically, we have to do the opposite of what the Hypothesis docs recommend for composite strategies: https://hypothesis.readthedocs.io/en/latest/data.html#composite-strategies. In the `reimplementing_sets_strategy` example, we specifically need to do things the "bad" way, since the "good" way means we run into this issue whenever we try to use the strategy in a state machine. 

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/internal/conjecture/data.py b/hypothesis-python/src/hypothesis/internal/conjecture/data.py
--- a/hypothesis-python/src/hypothesis/internal/conjecture/data.py
+++ b/hypothesis-python/src/hypothesis/internal/conjecture/data.py
@@ -1693,7 +1693,7 @@ def __init__(
         if observer is None:
             observer = DataObserver()
         assert isinstance(observer, DataObserver)
-        self.__bytes_drawn = 0
+        self._bytes_drawn = 0
         self.observer = observer
         self.max_length = max_length
         self.is_find = False
@@ -2264,8 +2264,8 @@ def draw_bits(
 
         if forced is not None:
             buf = int_to_bytes(forced, n_bytes)
-        elif self.__bytes_drawn < len(self.__prefix):
-            index = self.__bytes_drawn
+        elif self._bytes_drawn < len(self.__prefix):
+            index = self._bytes_drawn
             buf = self.__prefix[index : index + n_bytes]
             if len(buf) < n_bytes:
                 assert self.__random is not None
@@ -2274,7 +2274,7 @@ def draw_bits(
             assert self.__random is not None
             buf = uniform(self.__random, n_bytes)
         buf = bytearray(buf)
-        self.__bytes_drawn += n_bytes
+        self._bytes_drawn += n_bytes
 
         assert len(buf) == n_bytes
 
diff --git a/hypothesis-python/src/hypothesis/stateful.py b/hypothesis-python/src/hypothesis/stateful.py
--- a/hypothesis-python/src/hypothesis/stateful.py
+++ b/hypothesis-python/src/hypothesis/stateful.py
@@ -49,6 +49,7 @@
 from hypothesis.errors import InvalidArgument, InvalidDefinition
 from hypothesis.internal.compat import add_note
 from hypothesis.internal.conjecture import utils as cu
+from hypothesis.internal.conjecture.engine import BUFFER_SIZE
 from hypothesis.internal.healthcheck import fail_health_check
 from hypothesis.internal.observability import TESTCASE_CALLBACKS
 from hypothesis.internal.reflection import (
@@ -151,6 +152,10 @@ def output(s):
                     must_stop = True
                 elif steps_run <= _min_steps:
                     must_stop = False
+                elif cd._bytes_drawn > (0.8 * BUFFER_SIZE):
+                    # Better to stop after fewer steps, than always overrun and retry.
+                    # See https://github.com/HypothesisWorks/hypothesis/issues/3618
+                    must_stop = True
 
                 start_draw = perf_counter()
                 if cd.draw_boolean(p=2**-16, forced=must_stop):


Test patch:
diff --git a/hypothesis-python/tests/cover/test_stateful.py b/hypothesis-python/tests/cover/test_stateful.py
--- a/hypothesis-python/tests/cover/test_stateful.py
+++ b/hypothesis-python/tests/cover/test_stateful.py
@@ -1288,3 +1288,13 @@ def fail_fast(self, a1, a2, a3, a4, a5, a6, b1, b2, b3):
 state.teardown()
 """.strip()
     )
+
+
+class LotsOfEntropyPerStepMachine(RuleBasedStateMachine):
+    # Regression tests for https://github.com/HypothesisWorks/hypothesis/issues/3618
+    @rule(data=binary(min_size=512, max_size=512))
+    def rule1(self, data):
+        assert data
+
+
+TestLotsOfEntropyPerStepMachine = LotsOfEntropyPerStepMachine.TestCase