Repository: HypothesisWorks/hypothesis (Python)

Issue:
Multiple stateful @preconditions cause all but one to be silently ignored
While investigating an initialize/invariant fix for #2860, I noticed that unlike most of the other decorators, `@precondition` has no check preventing multiple preconditions for the same method.

Unfortunately, this doesn't result in the intuitive outcome of multiple preconditions being and-ed together. Instead, they each silently overwrite each other, with only the outermost precondition prevailing.

```python

def test_multiple_precondition_bug():
    class MultiplePreconditionMachine(RuleBasedStateMachine):
        @rule(x=integers())
        def good_method(self, x):
            pass

        @precondition(lambda self: True)
        @precondition(lambda self: False)
        @rule(x=integers())
        def bad_method_a(self, x):
            """This rule unexpectedly runs, even though it shouldn't."""
            assert False

        @precondition(lambda self: False)
        @precondition(lambda self: True)
        @rule(x=integers())
        def bad_method_b(self, x):
            """This rule never runs, but for the wrong reason."""
            assert False

    run_state_machine_as_test(MultiplePreconditionMachine)

```

```
Falsifying example:
state = MultiplePreconditionMachine()
state.bad_method_a(x=0)
state.teardown()
```


Discussion:
A few more stateful issues that I want to write down in case I forget them later:

- Methods can have both `@invariant` and `@rule`, which works but is very unlikely to be useful.
- If a method has decorators `@invariant @precondition @rule` in that exact order, the precondition is applied to the rule but not the invariant. (In other cases it applies to both.)
- Some of the decorators return a proxy function wrapping the original method, but this proxy appears to serve no actual useful purpose. Perhaps a relic of older implementation decisions?
I propose that we

- Make applying `@invariant` and `@rule` to the same method an explicit error (which also fixes the next point).  This has never been supported and it's also strange enough that I don't think Hyrum's Law applies.
- Separately from the other changes, try removing those wrappers as a performance improvement.
Ah, the wrappers do have a purpose - they mean we can disallow the first example and support the second:

```python
class NestedRuleDecoratorsMachine(RuleBasedStateMachine): 
    @rule(data=just("rule1data"))
    @rule(data=just("rule2data"))
    def myfunc(self, data):
        print(data)
```
https://github.com/HypothesisWorks/hypothesis/blob/d76062140369488ccff387d6bc78cf2c5995d80b/hypothesis-python/tests/cover/test_stateful.py#L98-L103

In general we do have a fair bit of legacy code in `stateful.py` though, which has been unnecessary since we dropped `GenericStateMachine` but not yet cleaned up.



Patch:
diff --git a/hypothesis-python/RELEASE.rst b/hypothesis-python/RELEASE.rst
new file mode 100644
index 0000000000..6c658832b5
--- /dev/null
+++ b/hypothesis-python/RELEASE.rst
@@ -0,0 +1,5 @@
+RELEASE_TYPE: minor
+
+This release fixes :doc:`stateful testing methods <stateful>` with multiple
+:func:`~hypothesis.stateful.precondition` decorators.  Previously, only the
+outer-most precondition was checked (:issue:`2681`).
diff --git a/hypothesis-python/src/hypothesis/stateful.py b/hypothesis-python/src/hypothesis/stateful.py
index 9cc966f463..16e7863703 100644
--- a/hypothesis-python/src/hypothesis/stateful.py
+++ b/hypothesis-python/src/hypothesis/stateful.py
@@ -333,7 +333,7 @@ def _add_result_to_targets(self, targets, result):
 
     def check_invariants(self):
         for invar in self.invariants():
-            if invar.precondition and not invar.precondition(self):
+            if not all(precond(self) for precond in invar.preconditions):
                 continue
             invar.function(self)
 
@@ -367,7 +367,7 @@ class Rule:
     targets = attr.ib()
     function = attr.ib(repr=qualname)
     arguments = attr.ib()
-    precondition = attr.ib()
+    preconditions = attr.ib()
     bundles = attr.ib(init=False)
 
     def __attrs_post_init__(self):
@@ -508,7 +508,7 @@ def _convert_targets(targets, target):
 
 RULE_MARKER = "hypothesis_stateful_rule"
 INITIALIZE_RULE_MARKER = "hypothesis_stateful_initialize_rule"
-PRECONDITION_MARKER = "hypothesis_stateful_precondition"
+PRECONDITIONS_MARKER = "hypothesis_stateful_preconditions"
 INVARIANT_MARKER = "hypothesis_stateful_invariant"
 
 
@@ -543,12 +543,12 @@ def accept(f):
             raise InvalidDefinition(
                 "A function cannot be used for two distinct rules. ", Settings.default
             )
-        precondition = getattr(f, PRECONDITION_MARKER, None)
+        preconditions = getattr(f, PRECONDITIONS_MARKER, ())
         rule = Rule(
             targets=converted_targets,
             arguments=kwargs,
             function=f,
-            precondition=precondition,
+            preconditions=preconditions,
         )
 
         @proxies(f)
@@ -568,6 +568,7 @@ def initialize(*, targets=(), target=None, **kwargs):
     methods will be called before any ``@rule()`` decorated methods, in an arbitrary
     order.  Each ``@initialize()`` method will be called exactly once per run, unless
     one raises an exception - after which only the ``.teardown()`` method will be run.
+    ``@initialize()`` methods may not have preconditions.
     """
     converted_targets = _convert_targets(targets, target)
     for k, v in kwargs.items():
@@ -580,8 +581,8 @@ def accept(f):
             raise InvalidDefinition(
                 "A function cannot be used for two distinct rules. ", Settings.default
             )
-        precondition = getattr(f, PRECONDITION_MARKER, None)
-        if precondition:
+        preconditions = getattr(f, PRECONDITIONS_MARKER, ())
+        if preconditions:
             raise InvalidDefinition(
                 "An initialization rule cannot have a precondition. ", Settings.default
             )
@@ -589,7 +590,7 @@ def accept(f):
             targets=converted_targets,
             arguments=kwargs,
             function=f,
-            precondition=precondition,
+            preconditions=preconditions,
         )
 
         @proxies(f)
@@ -610,7 +611,8 @@ class VarReference:
 def precondition(precond):
     """Decorator to apply a precondition for rules in a RuleBasedStateMachine.
     Specifies a precondition for a rule to be considered as a valid step in the
-    state machine. The given function will be called with the instance of
+    state machine, which is more efficient than using :func:`~hypothesis.assume`
+    within the rule.  The ``precond`` function will be called with the instance of
     RuleBasedStateMachine and should return True or False. Usually it will need
     to look at attributes on that instance.
 
@@ -624,8 +626,9 @@ class MyTestMachine(RuleBasedStateMachine):
             def divide_with(self, numerator):
                 self.state = numerator / self.state
 
-    This is better than using assume in your rule since more valid rules
-    should be able to be run.
+    If multiple preconditions are applied to a single rule, it is only considered
+    a valid step when all of them return True.  Preconditions may be applied to
+    invariants as well as rules.
     """
 
     def decorator(f):
@@ -640,21 +643,23 @@ def precondition_wrapper(*args, **kwargs):
             )
 
         rule = getattr(f, RULE_MARKER, None)
-        if rule is None:
-            setattr(precondition_wrapper, PRECONDITION_MARKER, precond)
-        else:
-            new_rule = Rule(
-                targets=rule.targets,
-                arguments=rule.arguments,
-                function=rule.function,
-                precondition=precond,
-            )
-            setattr(precondition_wrapper, RULE_MARKER, new_rule)
-
         invariant = getattr(f, INVARIANT_MARKER, None)
-        if invariant is not None:
-            new_invariant = Invariant(function=invariant.function, precondition=precond)
+        if rule is not None:
+            assert invariant is None
+            new_rule = attr.evolve(rule, preconditions=rule.preconditions + (precond,))
+            setattr(precondition_wrapper, RULE_MARKER, new_rule)
+        elif invariant is not None:
+            assert rule is None
+            new_invariant = attr.evolve(
+                invariant, preconditions=invariant.preconditions + (precond,)
+            )
             setattr(precondition_wrapper, INVARIANT_MARKER, new_invariant)
+        else:
+            setattr(
+                precondition_wrapper,
+                PRECONDITIONS_MARKER,
+                getattr(f, PRECONDITIONS_MARKER, ()) + (precond,),
+            )
 
         return precondition_wrapper
 
@@ -664,7 +669,7 @@ def precondition_wrapper(*args, **kwargs):
 @attr.s()
 class Invariant:
     function = attr.ib()
-    precondition = attr.ib()
+    preconditions = attr.ib()
 
 
 def invariant():
@@ -689,8 +694,8 @@ def accept(f):
                 "A function cannot be used for two distinct invariants.",
                 Settings.default,
             )
-        precondition = getattr(f, PRECONDITION_MARKER, None)
-        rule = Invariant(function=f, precondition=precondition)
+        preconditions = getattr(f, PRECONDITIONS_MARKER, ())
+        rule = Invariant(function=f, preconditions=preconditions)
 
         @proxies(f)
         def invariant_wrapper(*args, **kwargs):
@@ -758,7 +763,7 @@ def do_draw(self, data):
         return (rule, data.draw(rule.arguments_strategy))
 
     def is_valid(self, rule):
-        if rule.precondition and not rule.precondition(self.machine):
+        if not all(precond(self.machine) for precond in rule.preconditions):
             return False
 
         for b in rule.bundles:


Test patch:
diff --git a/hypothesis-python/tests/cover/test_stateful.py b/hypothesis-python/tests/cover/test_stateful.py
index c9605158b0..2e2aad21b5 100644
--- a/hypothesis-python/tests/cover/test_stateful.py
+++ b/hypothesis-python/tests/cover/test_stateful.py
@@ -1132,3 +1132,61 @@ def mostly_fails(self, d):
     output = o.getvalue()
     assert "v1 = state.init_data(value=0)" in output
     assert "v1 = state.init_data(value=v1)" not in output
+
+
+def test_multiple_precondition_bug():
+    # See https://github.com/HypothesisWorks/hypothesis/issues/2861
+    class MultiplePreconditionMachine(RuleBasedStateMachine):
+        @rule(x=integers())
+        def good_method(self, x):
+            pass
+
+        @precondition(lambda self: True)
+        @precondition(lambda self: False)
+        @rule(x=integers())
+        def bad_method_a(self, x):
+            raise AssertionError("This rule runs, even though it shouldn't.")
+
+        @precondition(lambda self: False)
+        @precondition(lambda self: True)
+        @rule(x=integers())
+        def bad_method_b(self, x):
+            raise AssertionError("This rule might be skipped for the wrong reason.")
+
+        @precondition(lambda self: True)
+        @rule(x=integers())
+        @precondition(lambda self: False)
+        def bad_method_c(self, x):
+            raise AssertionError("This rule runs, even though it shouldn't.")
+
+        @rule(x=integers())
+        @precondition(lambda self: True)
+        @precondition(lambda self: False)
+        def bad_method_d(self, x):
+            raise AssertionError("This rule runs, even though it shouldn't.")
+
+        @precondition(lambda self: True)
+        @precondition(lambda self: False)
+        @invariant()
+        def bad_invariant_a(self):
+            raise AssertionError("This invariant runs, even though it shouldn't.")
+
+        @precondition(lambda self: False)
+        @precondition(lambda self: True)
+        @invariant()
+        def bad_invariant_b(self):
+            raise AssertionError("This invariant runs, even though it shouldn't.")
+
+        @precondition(lambda self: True)
+        @invariant()
+        @precondition(lambda self: False)
+        def bad_invariant_c(self):
+            raise AssertionError("This invariant runs, even though it shouldn't.")
+
+        @invariant()
+        @precondition(lambda self: True)
+        @precondition(lambda self: False)
+        def bad_invariant_d(self):
+            raise AssertionError("This invariant runs, even though it shouldn't.")
+
+    run_state_machine_as_test(MultiplePreconditionMachine)