Repository: HypothesisWorks/hypothesis

Issue:
Add initialization rules to RuleBasedStateMachine
A common usage pattern in RuleBasedStateMachine is that you have a a single mutable data structure instance that is shared across the whole test.

This is currently only well supported if you can just straightforwardly initialize that data structure without relying on Hypothesis provided data and do that in `__init__`.

If you need Hypothesis to provide the data, currently the best emulation is something like the following:

```python

from hypothesis.stateful import RuleBasedStateMachine, rule, precondition
import hypothesis.strategies as st


class NeedsInit(RuleBasedStateMachine):
    def __init__(self):
       super(NeedsInit, self).__init__()
       self.some_data = None

   @precondition(lambda: self.some_data is None)
   @rule(data=st.lists(st.integers())
   def init_data(self, data):
       self.some_data =data

   @precondition(lambda: self.some_data is not None)
   @rule()
   def do_stuff(self):
       ...
```

(disclaimer: I wrote that from memory without running it).

This is obviously quite awkward! You have to add preconditions to each rule.

It would be nice if we could instead write:

```python
from hypothesis.stateful import RuleBasedStateMachine, rule, initialize
import hypothesis.strategies as st

class NeedsInit(RuleBasedStateMachine):
   @initialize(data=st.lists(st.integers())
   def init_data(self, data):
       self.some_data =data

   @rule()
   def do_stuff(self):
       ...
```

Suggested semantics/implementation:

* We add a `state_machine_init` method or something like that to `GenericStateMachine` as the relevant hook.
* `state_machine_init` is called with a `st.data()` strategy before the first `check_invariants` call
* `RuleBasedStateMachine` implements `state_machine_init` to run each `@initialize` decorated method exactly once in an arbitrary order (possibly an explicitly randomised order using `permutations`? That way people won't depend on the order).


Discussion:
You can close #1214 as duplicate if you like. This would fill my feature request.
The API I had imagined is below. Do you prefer yours?

````
from hypothesis.stateful import RuleBasedStateMachine, rule
import hypothesis.strategies as st

class NeedsInit(RuleBasedStateMachine):
   def __init__(self, some_data=st.lists(st.integers()):
       self.some_data = some_data

   @rule()
   def do_stuff(self):
       ...
```
> Do you prefer yours?

Yes, pretty strongly. I'd possibly be OK with an `__init__` based approach (though I think separating out the initialize rules is probably better), but I'm very against using default arguments to indicate strategies - it's inconsistent with pretty much everywhere else we use strategies.
+1 to `@initialize` rules - this will also allow a single class to test multiple constructors for an object.
> this will also allow a single class to test multiple constructors for an object.

Hmm. You may have accidentally just argued against this feature, because there are two different interpretations here and we seem to have hit on different ones.

Possible semantics of multiple initialization rules:

* Exactly one of them is run
* All of them are run, in some arbitrary order

I had intended the latter and you seem to have inferred the former. This might be fine if we explicitly document it, but it might lead to some significant user confusion.
So I might almost be tempted to lean a third way, and suggest `@initialize`, but at most one per class, then different constructors get tested by different classes.

So if you have different constructors, you push your rules into a mixin, and create variants of that mixin with different `@initialize` rules. Each constructor gets a dedicated test case.

This is just spitballing, I’m not convinced this is actually a good idea. 🙃 
FWIW I think the use-case of different constructors can be solved by just branching within the initialize method. So you can have something like:

```python
@initialize(data=st.data())
def setup(self, data):
    if data.draw(st.booleans()):
        self.thing = ...
    else:
        self.thing = ...
```

So I'm not convinced that "What should we do about multiple constructors?" is an interesting question for shaping the design of this.

I don't have a problem with allowing at most one `@initialize` per class though. It might reduce the confusion, simplify the implementation, and is an easy restriction for us to lift later if someone comes up with a compelling use case.
> I'm not convinced that "What should we do about multiple constructors?" is an interesting question for shaping the design of this.

I think I agree. Most classes I write don’t have multiple constructors, and of the ones that do, most of them have a “primary” constructor, and others that just tidy up the inputs (e.g. taking a `str` instead of a `datetime`, and parsing it before sending to the primary constructor).
>  Most classes I write don’t have multiple constructors, 

I mean in Python *no* classes have multiple constructors. The only real way to get something like that is to have multiple subclasses.
Or classmethods!

I meant "you can define multiple methods with `@initialize`, of which exactly one will be run per iteration".

Take [Pandas dataframes](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html) for example; they have an `__init__` method but also classmethods `from_records`, `from_dict`, and `from_items`.  Having an `@initialize` for each option would let you test that <whatever property> holds no matter how you create the object.

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/stateful.py b/hypothesis-python/src/hypothesis/stateful.py
--- a/hypothesis-python/src/hypothesis/stateful.py
+++ b/hypothesis-python/src/hypothesis/stateful.py
@@ -28,6 +28,7 @@
 
 import inspect
 import traceback
+from copy import copy
 from unittest import TestCase
 
 import attr
@@ -174,7 +175,7 @@ def teardown(self):
         """Called after a run has finished executing to clean up any necessary
         state.
 
-        Does nothing by default
+        Does nothing by default.
         """
         pass
 
@@ -303,6 +304,7 @@ def do_draw(self, data):
 
 
 RULE_MARKER = u'hypothesis_stateful_rule'
+INITIALIZE_RULE_MARKER = u'hypothesis_stateful_initialize_rule'
 PRECONDITION_MARKER = u'hypothesis_stateful_precondition'
 INVARIANT_MARKER = u'hypothesis_stateful_invariant'
 
@@ -330,7 +332,8 @@ def rule(targets=(), target=None, **kwargs):
 
     def accept(f):
         existing_rule = getattr(f, RULE_MARKER, None)
-        if existing_rule is not None:
+        existing_initialize_rule = getattr(f, INITIALIZE_RULE_MARKER, None)
+        if existing_rule is not None or existing_initialize_rule is not None:
             raise InvalidDefinition(
                 'A function cannot be used for two distinct rules. ',
                 Settings.default,
@@ -348,6 +351,49 @@ def rule_wrapper(*args, **kwargs):
     return accept
 
 
+def initialize(targets=(), target=None, **kwargs):
+    """Decorator for RuleBasedStateMachine.
+
+    An initialize decorator behaves like a rule, but the decorated
+    method is called at most once in a run. All initialize decorated
+    methods will be called before any rule decorated methods, in an
+    arbitrary order.
+    """
+    if target is not None:
+        targets += (target,)
+
+    converted_targets = []
+    for t in targets:
+        while isinstance(t, Bundle):
+            t = t.name
+        converted_targets.append(t)
+
+    def accept(f):
+        existing_rule = getattr(f, RULE_MARKER, None)
+        existing_initialize_rule = getattr(f, INITIALIZE_RULE_MARKER, None)
+        if existing_rule is not None or existing_initialize_rule is not None:
+            raise InvalidDefinition(
+                'A function cannot be used for two distinct rules. ',
+                Settings.default,
+            )
+        precondition = getattr(f, PRECONDITION_MARKER, None)
+        if precondition:
+            raise InvalidDefinition(
+                'An initialization rule cannot have a precondition. ',
+                Settings.default,
+            )
+        rule = Rule(targets=tuple(converted_targets), arguments=kwargs,
+                    function=f, precondition=precondition)
+
+        @proxies(f)
+        def rule_wrapper(*args, **kwargs):
+            return f(*args, **kwargs)
+
+        setattr(rule_wrapper, INITIALIZE_RULE_MARKER, rule)
+        return rule_wrapper
+    return accept
+
+
 @attr.s()
 class VarReference(object):
     name = attr.ib()
@@ -378,6 +424,13 @@ def decorator(f):
         def precondition_wrapper(*args, **kwargs):
             return f(*args, **kwargs)
 
+        existing_initialize_rule = getattr(f, INITIALIZE_RULE_MARKER, None)
+        if existing_initialize_rule is not None:
+            raise InvalidDefinition(
+                'An initialization rule cannot have a precondition. ',
+                Settings.default,
+            )
+
         rule = getattr(f, RULE_MARKER, None)
         if rule is None:
             setattr(precondition_wrapper, PRECONDITION_MARKER, precond)
@@ -449,6 +502,8 @@ class RuleBasedStateMachine(GenericStateMachine):
     _rules_per_class = {}  # type: Dict[type, List[classmethod]]
     _invariants_per_class = {}  # type: Dict[type, List[classmethod]]
     _base_rules_per_class = {}  # type: Dict[type, List[classmethod]]
+    _initializers_per_class = {}  # type: Dict[type, List[classmethod]]
+    _base_initializers_per_class = {}  # type: Dict[type, List[classmethod]]
 
     def __init__(self):
         if not self.rules():
@@ -460,6 +515,7 @@ def __init__(self):
         self.names_to_values = {}  # type: Dict[Text, Any]
         self.__stream = CUnicodeIO()
         self.__printer = RepresentationPrinter(self.__stream)
+        self._initialize_rules_to_run = copy(self.initialize_rules())
 
     def __pretty(self, value):
         if isinstance(value, VarReference):
@@ -490,6 +546,23 @@ def new_name(self):
     def bundle(self, name):
         return self.bundles.setdefault(name, [])
 
+    @classmethod
+    def initialize_rules(cls):
+        try:
+            return cls._initializers_per_class[cls]
+        except KeyError:
+            pass
+
+        for k, v in inspect.getmembers(cls):
+            r = getattr(v, INITIALIZE_RULE_MARKER, None)
+            if r is not None:
+                cls.define_initialize_rule(
+                    r.targets, r.function, r.arguments, r.precondition,
+                )
+        cls._initializers_per_class[cls] = \
+            cls._base_initializers_per_class.pop(cls, [])
+        return cls._initializers_per_class[cls]
+
     @classmethod
     def rules(cls):
         try:
@@ -521,6 +594,23 @@ def invariants(cls):
         cls._invariants_per_class[cls] = target
         return cls._invariants_per_class[cls]
 
+    @classmethod
+    def define_initialize_rule(
+            cls, targets, function, arguments, precondition=None):
+        converted_arguments = {}
+        for k, v in arguments.items():
+            converted_arguments[k] = v
+        if cls in cls._initializers_per_class:
+            target = cls._initializers_per_class[cls]
+        else:
+            target = cls._base_initializers_per_class.setdefault(cls, [])
+
+        return target.append(
+            Rule(
+                targets, function, converted_arguments, precondition,
+            )
+        )
+
     @classmethod
     def define_rule(cls, targets, function, arguments, precondition=None):
         converted_arguments = {}
@@ -538,6 +628,14 @@ def define_rule(cls, targets, function, arguments, precondition=None):
         )
 
     def steps(self):
+        # Pick initialize rules first
+        if self._initialize_rules_to_run:
+            return one_of([
+                tuples(just(rule), fixed_dictionaries(rule.arguments))
+                for rule in self._initialize_rules_to_run
+            ])
+
+        # All initialize rules has been run once, go with the regular rules
         strategies = []
         for rule in self.rules():
             converted_arguments = {}
@@ -596,6 +694,8 @@ def execute_step(self, step):
             )
             for target in rule.targets:
                 self.bundle(target).append(VarReference(name))
+        if self._initialize_rules_to_run:
+            self._initialize_rules_to_run.remove(rule)
 
     def check_invariants(self):
         for invar in self.invariants():


Test patch:
diff --git a/hypothesis-python/tests/cover/test_stateful.py b/hypothesis-python/tests/cover/test_stateful.py
--- a/hypothesis-python/tests/cover/test_stateful.py
+++ b/hypothesis-python/tests/cover/test_stateful.py
@@ -30,7 +30,7 @@
     checks_deprecated_behaviour
 from hypothesis.database import ExampleDatabase
 from hypothesis.stateful import Bundle, GenericStateMachine, \
-    RuleBasedStateMachine, rule, invariant, precondition, \
+    RuleBasedStateMachine, rule, invariant, initialize, precondition, \
     run_state_machine_as_test
 from hypothesis.strategies import just, none, lists, binary, tuples, \
     choices, booleans, integers, sampled_from
@@ -771,3 +771,185 @@ def fail(self, source):
     assert 'v1 = state.create()' in result
     assert 'v2 = state.transfer(source=v1)' in result
     assert 'state.fail(source=v2)' in result
+
+
+def test_initialize_rule():
+    class WithInitializeRules(RuleBasedStateMachine):
+        initialized = []
+
+        @initialize()
+        def initialize_a(self):
+            self.initialized.append('a')
+
+        @initialize()
+        def initialize_b(self):
+            self.initialized.append('b')
+
+        @initialize()
+        def initialize_c(self):
+            self.initialized.append('c')
+
+        @rule()
+        def fail_fast(self):
+            assert False
+
+    with capture_out() as o:
+        with pytest.raises(AssertionError):
+            run_state_machine_as_test(WithInitializeRules)
+
+    assert set(WithInitializeRules.initialized[-3:]) == {'a', 'b', 'c'}
+    result = o.getvalue().splitlines()
+    assert result[0] == 'state = WithInitializeRules()'
+    # Initialize rules call order is shuffled
+    assert {result[1], result[2], result[3]} == {
+        'state.initialize_a()', 'state.initialize_b()', 'state.initialize_c()'
+    }
+    assert result[4] == 'state.fail_fast()'
+    assert result[5] == 'state.teardown()'
+
+
+def test_initialize_rule_populate_bundle():
+    class WithInitializeBundleRules(RuleBasedStateMachine):
+        a = Bundle('a')
+
+        @initialize(target=a, dep=just('dep'))
+        def initialize_a(self, dep):
+            return 'a v1 with (%s)' % dep
+
+        @rule(param=a)
+        def fail_fast(self, param):
+            assert False
+
+    with capture_out() as o:
+        with pytest.raises(AssertionError):
+            run_state_machine_as_test(WithInitializeBundleRules)
+
+    result = o.getvalue()
+    assert result == """state = WithInitializeBundleRules()
+v1 = state.initialize_a(dep='dep')
+state.fail_fast(param=v1)
+state.teardown()
+"""
+
+
+def test_initialize_rule_dont_mix_with_precondition():
+    with pytest.raises(InvalidDefinition) as exc:
+        class BadStateMachine(RuleBasedStateMachine):
+
+            @precondition(lambda self: True)
+            @initialize()
+            def initialize(self):
+                pass
+
+    assert 'An initialization rule cannot have a precondition.' in str(
+        exc.value)
+
+    # Also test decorator application in reverse order
+
+    with pytest.raises(InvalidDefinition) as exc:
+        class BadStateMachineReverseOrder(RuleBasedStateMachine):
+
+            @initialize()
+            @precondition(lambda self: True)
+            def initialize(self):
+                pass
+
+    assert 'An initialization rule cannot have a precondition.' in str(
+        exc.value)
+
+
+def test_initialize_rule_dont_mix_with_regular_rule():
+    with pytest.raises(InvalidDefinition) as exc:
+        class BadStateMachine(RuleBasedStateMachine):
+
+            @rule()
+            @initialize()
+            def initialize(self):
+                pass
+
+    assert 'A function cannot be used for two distinct rules.' in str(
+        exc.value)
+
+
+def test_initialize_rule_cannot_be_double_applied():
+    with pytest.raises(InvalidDefinition) as exc:
+        class BadStateMachine(RuleBasedStateMachine):
+
+            @initialize()
+            @initialize()
+            def initialize(self):
+                pass
+
+    assert 'A function cannot be used for two distinct rules.' in str(
+        exc.value)
+
+
+def test_initialize_rule_in_state_machine_with_inheritance():
+    class ParentStateMachine(RuleBasedStateMachine):
+        initialized = []
+
+        @initialize()
+        def initialize_a(self):
+            self.initialized.append('a')
+
+    class ChildStateMachine(ParentStateMachine):
+
+        @initialize()
+        def initialize_b(self):
+            self.initialized.append('b')
+
+        @rule()
+        def fail_fast(self):
+            assert False
+
+    with capture_out() as o:
+        with pytest.raises(AssertionError):
+            run_state_machine_as_test(ChildStateMachine)
+
+    assert set(ChildStateMachine.initialized[-2:]) == {'a', 'b'}
+    result = o.getvalue().splitlines()
+    assert result[0] == 'state = ChildStateMachine()'
+    # Initialize rules call order is shuffled
+    assert {result[1], result[2]} == {
+        'state.initialize_a()', 'state.initialize_b()'}
+    assert result[3] == 'state.fail_fast()'
+    assert result[4] == 'state.teardown()'
+
+
+def test_can_manually_call_initialize_rule():
+    class StateMachine(RuleBasedStateMachine):
+        initialize_called_counter = 0
+
+        @initialize()
+        def initialize(self):
+            self.initialize_called_counter += 1
+            return self.initialize_called_counter
+
+        @rule()
+        def fail_eventually(self):
+            assert self.initialize() == 2
+
+    with capture_out() as o:
+        with pytest.raises(AssertionError):
+            run_state_machine_as_test(StateMachine)
+
+    result = o.getvalue()
+    assert result == """state = StateMachine()
+state.initialize()
+state.fail_eventually()
+state.fail_eventually()
+state.teardown()
+"""
+
+
+def test_new_initialize_rules_are_picked_up_before_and_after_rules_call():
+    class Foo(RuleBasedStateMachine):
+        pass
+    Foo.define_initialize_rule(
+        targets=(), function=lambda self: 1, arguments={}
+    )
+    assert len(Foo.initialize_rules()) == 1
+    Foo.define_initialize_rule(
+        targets=(), function=lambda self: 2, arguments={}
+    )
+    assert len(Foo.initialize_rules()) == 2