Repository: HypothesisWorks/hypothesis

Issue:
The "we're dropping Python 2" checklist
_This post is a living document, and regularly edited to reflect our latest plans._

Per [our compatibility policy](https://hypothesis.readthedocs.io/en/latest/supported.html#python-versions), Hypothesis will drop Python 2 support soon after [upstream support ends on 2020-01-01](https://devguide.python.org/#status-of-python-branches) ([countdown here](https://pythonclock.org/)).  This issue exists to describe, discuss, and track progress on all the improvements that are possible once we go Python-3-only.

First, 

- [x] #2282: release Hypothesis 5.0 without Python 2 support on Jan 1st.
      That PR has its own checklist, with minimal changes to get to the new public API.

Once Hypothesis 5.0 is released, I'll open a second PR with the large tool-assisted changes.  This will be a very large diff, but quite mechanical and not too hard to review.  IMO it's best to get this in as a single hard-changeover, to minimise the merge conflicts and churn it could cause for other PRs.

- [x] #2299: fully-automated removal of py2 compat code, with some minor manual fixes.
      That PR *also* has its own checklist, and has **no** changes to the public API.

That's a `+1,020 −2,461` line PR, which is quite enough to review even as obvious mechanical fixes.  Once *that's* in, we have the manual fixes - the bulk of which are "shrink `compat.py`":

- [x] #2301: replacing compatibility shims and imports with native Python 3 code.

Finally, I'll:

- [x] #2309: convert type comments to annotations, fix conditional imports of the `typing` module, and fix bad annotations

Subsequent changes - of which there are many! - can be done in smaller and mostly independent chunks.  This reduces the size of merge conflicts, facilitates code review, and ensures that we can mix feature development, unrelated bugfixes, and refactoring work as they appeal to each contributor 😄

- [x] Finish refactoring stateful testing (initial PR just makes GenericStateMachine a private API), done in #2332
- [x] Consider replacing uses of `inspect.getfullargspec` with `inspect.signature`
- [ ] Consider specifying how to write short bytestrings in our project style guide
      e.g. preferring `b"\x00"` over any of `b"\0"`, `bytes(1)`, or `bytes([0])`.
- [ ] Move remaining functions from `compat.py` to live with related code instead
- [x] Misc. cleanups, i.e. #2315 

I'm sure there are many things I'm missing - if you think of one that doesn't already have its own issue or pull request, please comment below and I'll update the list!  My goal here is to assemble a complete checklist of refactorings and upgrades that we *probably* want to do; so interpret these as options rather than requirements.

---------------

Special bonus checklist: things to revisit when we drop Python 3.5 later this year!

- Use `com2ann` to convert type-hints on variables to annotations
- Check  `git grep -nE "# pragma: no (branch|cover)|[pP]y(thon)? ?3|sys\.version" -- . ':!*changes.rst'` to see all of the locations we've tagged for version compatibility reasons (and a few false-positives)
- Generally clean up a lot of the `from_type()` backend
- Bump target versions for `black` and `pyupgrade`
- Consider converting from percent-formatting to f-strings


Discussion:
Is python 3.5 still desired? If not updating to f-strings can tidy the code substantially.


Our compatibility policy is "until end of upstream support", so we'll be keeping 3.5 [until next September](https://devguide.python.org/#status-of-python-branches). I'm definitely looking forward to a 3.6+ codebase though! 

(strictly speaking our policy only says "Hypothesis *is* ...", not "will remain" - so we're in the clear to drop it early if we choose, as proposed below)
> Our compatibility policy is "until end of upstream support"

though personally I'm keen to be more aggressive about this, and dropping 3.5 early is a pretty safe choice IMO.
I've been thinking about this too - the *only* reason I'm keen to keep compatibility for 3.5 given the relatively low usage is to avoid blocking users from upgrading to the latest Hypothesis.  (honestly this shouldn't be an issue anyway, but it is anyway for some)  Simplicity of communication is nice, but "3.6+" is even easier to say!  I think a reasonable pathway would be to:

1. drop Python 2 at the new year and rip out the compatibility stuff shortly afterwards,
2. release Hypothesis 5.0 to clear out the deprecated stuff, and
3. release a minor version after that which drops Python 3.5 support - aiming for early Feb.

End result, 3.5 users have ~8 months on "modern" Hypothesis but without any extra features / bugfixes / whatever else we do over that timeframe; and we have a nicer development experience.  If any users care a *lot*, I'm happy to do paid backports in an isolated branch!

Sound good?
>  Modify our build system to auto-publish the release at exactly midnight (@DRMacIver)

Ha! You're on the ball. I was just coming over to this issue to add that.
> This will be Hypothesis 4.XX.0, i.e. a minor version.

Wait, just rereading this... I missed this the first time. Why will this be a minor version? I don't think we should drop Python 2 in a minor version.

If the reason is that we don't want to do both turn deprecations into errors and drop Python 2 support in the same release (which I'd be fine with us doing but I can see the argument against) I'd rather us do two major versions in short succession (it wouldn't be the first time!) than drop Python 2 support in a minor version.
I'm fine with doing 5.0 at the stroke of midnight - it shouldn't actually be hard to manage, so long as we push the refactoring off to follow-up patches (which was already the plan).  Top post updated.

I've got that week off and I'm happy to ~~write~~ delete a lot of code, but you-as-codeowner will be doing a lot of reviews...
If you don't mind, I want to help on this beside django and drf.
I don't mind at all - I'll write the initial set of three or four big changes (I've already scripted most of it), but after that I'd be happy for you to write some of the more contained ones in the first week of January. Review comments are also helpful! 
I will happily help shepherd the `extra.numpy` material along for this triumphant journey 
I've re-organised the post-5.0 checklist into two parts - one large but fairly mechanical PR that I'll aim to get out in the  first day or two of January, and a second part with smaller changes which more of us can work on concurrently :smile: 
can you try pyupgrade?
`pyupgrade` is a standard part of our formatting task, so we're already using it!

If you mean "to remove the `if PY2:` blocks, I've experimented on top of my `drop-py2` branch and it works pretty well - as with most of the ambitious formatting tools it does 80-90% of the work and then I'll do some manual cleanup to make it look like the code has always been that way :smile: 
As of the current PR, plus drafts linked in the top comment: 

> Showing **361 changed files** with **1,867 additions** and **4,705 deletions**.

My ~~condolences~~ thanks to everyone involved in review! :joy:

I'm already enjoying life without awkward compat shims - it makes a surprisingly big difference.  Or perhaps not so surprising; [pygount](https://pypi.org/project/pygount/) suggests that I've deleted a *net* 6% of the library even before considering tests... and the changed portion is generally much nicer to work with.
> Our compatibility policy is "until end of upstream support", so we'll be keeping 3.5 [until next September](https://devguide.python.org/#status-of-python-branches). I'm definitely looking forward to a 3.6+ codebase though!

Could I ask for a major version bump then? It makes version pinning clearer...

<sub>/me is still running hypothesis on python 2.6 and 3.3</sub>
Surely you already have to pin to a much older version though?
> Surely you already have to pin to a much older version though?

yes, that's why I'm asking for making it easier/more obvious to know which version to pin to (I don't pin all dependencies, only the ones I have to)

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/stateful.py b/hypothesis-python/src/hypothesis/stateful.py
--- a/hypothesis-python/src/hypothesis/stateful.py
+++ b/hypothesis-python/src/hypothesis/stateful.py
@@ -24,6 +24,7 @@
 import inspect
 from collections.abc import Iterable
 from copy import copy
+from functools import lru_cache
 from io import StringIO
 from typing import Any, Dict, List
 from unittest import TestCase
@@ -86,26 +87,21 @@ def run_state_machine_as_test(state_machine_factory, *, settings=None):
     @settings
     @given(st.data())
     def run_state_machine(factory, data):
+        cd = data.conjecture_data
         machine = factory()
-        if not isinstance(machine, _GenericStateMachine):
-            raise InvalidArgument(
-                "Expected RuleBasedStateMachine but state_machine_factory() "
-                "returned %r (type=%s)" % (machine, type(machine).__name__)
-            )
-        data.conjecture_data.hypothesis_runner = machine
+        check_type(RuleBasedStateMachine, machine, "state_machine_factory()")
+        cd.hypothesis_runner = machine
 
         print_steps = (
             current_build_context().is_final or current_verbosity() >= Verbosity.debug
         )
         try:
             if print_steps:
-                machine.print_start()
+                report("state = %s()" % (machine.__class__.__name__,))
             machine.check_invariants()
             max_steps = settings.stateful_step_count
             steps_run = 0
 
-            cd = data.conjecture_data
-
             while True:
                 # We basically always want to run the maximum number of steps,
                 # but need to leave a small probability of terminating early
@@ -134,22 +130,46 @@ def run_state_machine(factory, data):
                             break
                 steps_run += 1
 
-                value = data.conjecture_data.draw(machine.steps())
-                # Assign 'result' here in case 'execute_step' fails below
+                # Choose a rule to run, preferring an initialize rule if there are
+                # any which have not been run yet.
+                if machine._initialize_rules_to_run:
+                    init_rules = [
+                        st.tuples(st.just(rule), st.fixed_dictionaries(rule.arguments))
+                        for rule in machine._initialize_rules_to_run
+                    ]
+                    rule, data = cd.draw(st.one_of(init_rules))
+                    machine._initialize_rules_to_run.remove(rule)
+                else:
+                    rule, data = cd.draw(machine._rules_strategy)
+                data_to_print = dict(data)
+
+                # Assign 'result' here in case executing the rule fails below
                 result = multiple()
                 try:
-                    result = machine.execute_step(value)
+                    data = dict(data)
+                    for k, v in list(data.items()):
+                        if isinstance(v, VarReference):
+                            data[k] = machine.names_to_values[v.name]
+                    result = rule.function(machine, **data)
+                    if rule.targets:
+                        if isinstance(result, MultipleResults):
+                            for single_result in result.values:
+                                machine._add_result_to_targets(
+                                    rule.targets, single_result
+                                )
+                        else:
+                            machine._add_result_to_targets(rule.targets, result)
                 finally:
                     if print_steps:
                         # 'result' is only used if the step has target bundles.
                         # If it does, and the result is a 'MultipleResult',
                         # then 'print_step' prints a multi-variable assignment.
-                        machine.print_step(value, result)
+                        machine._print_step(rule, data_to_print, result)
                 machine.check_invariants()
-                data.conjecture_data.stop_example()
+                cd.stop_example()
         finally:
             if print_steps:
-                machine.print_end()
+                report("state.teardown()")
             machine.teardown()
 
     # Use a machine digest to identify stateful tests in the example database
@@ -168,10 +188,7 @@ def run_state_machine(factory, data):
     run_state_machine(state_machine_factory)
 
 
-class GenericStateMachineMeta(type):
-    def __init__(self, *args, **kwargs):
-        super().__init__(*args, **kwargs)
-
+class StateMachineMeta(type):
     def __setattr__(self, name, value):
         if name == "settings" and isinstance(value, Settings):
             raise AttributeError(
@@ -184,42 +201,169 @@ def __setattr__(self, name, value):
         return type.__setattr__(self, name, value)
 
 
-class _GenericStateMachine(
-    GenericStateMachineMeta("_GenericStateMachine", (object,), {})  # type: ignore
-):
-    # TODO: this is now internal-only and we can refactor RuleBasedStateMachine
+class RuleBasedStateMachine(metaclass=StateMachineMeta):
+    """A RuleBasedStateMachine gives you a structured way to define state machines.
 
-    def steps(self):
-        """Return a SearchStrategy instance the defines the available next
-        steps."""
-        raise NotImplementedError("%r.steps()" % (self,))
+    The idea is that a state machine carries a bunch of types of data
+    divided into Bundles, and has a set of rules which may read data
+    from bundles (or just from normal strategies) and push data onto
+    bundles. At any given point a random applicable rule will be
+    executed.
+    """
 
-    def execute_step(self, step):
-        """Execute a step that has been previously drawn from self.steps()
+    _rules_per_class = {}  # type: Dict[type, List[classmethod]]
+    _invariants_per_class = {}  # type: Dict[type, List[classmethod]]
+    _base_rules_per_class = {}  # type: Dict[type, List[classmethod]]
+    _initializers_per_class = {}  # type: Dict[type, List[classmethod]]
+    _base_initializers_per_class = {}  # type: Dict[type, List[classmethod]]
 
-        Returns the result of the step execution.
-        """
-        raise NotImplementedError("%r.execute_step()" % (self,))
+    def __init__(self):
+        if not self.rules():
+            raise InvalidDefinition("Type %s defines no rules" % (type(self).__name__,))
+        self.bundles = {}  # type: Dict[str, list]
+        self.name_counter = 1
+        self.names_to_values = {}  # type: Dict[str, Any]
+        self.__stream = StringIO()
+        self.__printer = RepresentationPrinter(self.__stream)
+        self._initialize_rules_to_run = copy(self.initialize_rules())
+        self._rules_strategy = RuleStrategy(self)
 
-    def print_start(self):
-        """Called right at the start of printing.
+    def __pretty(self, value):
+        if isinstance(value, VarReference):
+            return value.name
+        self.__stream.seek(0)
+        self.__stream.truncate(0)
+        self.__printer.output_width = 0
+        self.__printer.buffer_width = 0
+        self.__printer.buffer.clear()
+        self.__printer.pretty(value)
+        self.__printer.flush()
+        return self.__stream.getvalue()
 
-        By default does nothing.
-        """
+    def __repr__(self):
+        return "%s(%s)" % (type(self).__name__, nicerepr(self.bundles))
 
-    def print_end(self):
-        """Called right at the end of printing.
+    def _new_name(self):
+        result = "v%d" % (self.name_counter,)
+        self.name_counter += 1
+        return result
 
-        By default does nothing.
-        """
+    def _last_names(self, n):
+        assert self.name_counter > n
+        count = self.name_counter
+        return ["v%d" % (i,) for i in range(count - n, count)]
 
-    def print_step(self, step, result):
-        """Print a step to the current reporter.
+    def bundle(self, name):
+        return self.bundles.setdefault(name, [])
 
-        This is called right after a step is executed.
-        """
+    @classmethod
+    def initialize_rules(cls):
+        try:
+            return cls._initializers_per_class[cls]
+        except KeyError:
+            pass
+
+        for _, v in inspect.getmembers(cls):
+            r = getattr(v, INITIALIZE_RULE_MARKER, None)
+            if r is not None:
+                cls.define_initialize_rule(
+                    r.targets, r.function, r.arguments, r.precondition
+                )
+        cls._initializers_per_class[cls] = cls._base_initializers_per_class.pop(cls, [])
+        return cls._initializers_per_class[cls]
+
+    @classmethod
+    def rules(cls):
+        try:
+            return cls._rules_per_class[cls]
+        except KeyError:
+            pass
+
+        for _, v in inspect.getmembers(cls):
+            r = getattr(v, RULE_MARKER, None)
+            if r is not None:
+                cls.define_rule(r.targets, r.function, r.arguments, r.precondition)
+        cls._rules_per_class[cls] = cls._base_rules_per_class.pop(cls, [])
+        return cls._rules_per_class[cls]
+
+    @classmethod
+    def invariants(cls):
+        try:
+            return cls._invariants_per_class[cls]
+        except KeyError:
+            pass
+
+        target = []
+        for _, v in inspect.getmembers(cls):
+            i = getattr(v, INVARIANT_MARKER, None)
+            if i is not None:
+                target.append(i)
+        cls._invariants_per_class[cls] = target
+        return cls._invariants_per_class[cls]
+
+    @classmethod
+    def define_initialize_rule(cls, targets, function, arguments, precondition=None):
+        converted_arguments = {}
+        for k, v in arguments.items():
+            converted_arguments[k] = v
+        if cls in cls._initializers_per_class:
+            target = cls._initializers_per_class[cls]
+        else:
+            target = cls._base_initializers_per_class.setdefault(cls, [])
+
+        return target.append(Rule(targets, function, converted_arguments, precondition))
+
+    @classmethod
+    def define_rule(cls, targets, function, arguments, precondition=None):
+        converted_arguments = {}
+        for k, v in arguments.items():
+            converted_arguments[k] = v
+        if cls in cls._rules_per_class:
+            target = cls._rules_per_class[cls]
+        else:
+            target = cls._base_rules_per_class.setdefault(cls, [])
+
+        return target.append(Rule(targets, function, converted_arguments, precondition))
+
+    def _print_step(self, rule, data, result):
+        data_repr = {}
+        for k, v in data.items():
+            data_repr[k] = self.__pretty(v)
         self.step_count = getattr(self, "step_count", 0) + 1
-        report("Step #%d: %s" % (self.step_count, nicerepr(step)))
+        # If the step has target bundles, and the result is a MultipleResults
+        # then we want to assign to multiple variables.
+        if isinstance(result, MultipleResults):
+            n_output_vars = len(result.values)
+        else:
+            n_output_vars = 1
+        output_assignment = (
+            "%s = " % (", ".join(self._last_names(n_output_vars)),)
+            if rule.targets and n_output_vars >= 1
+            else ""
+        )
+        report(
+            "%sstate.%s(%s)"
+            % (
+                output_assignment,
+                rule.function.__name__,
+                ", ".join("%s=%s" % kv for kv in data_repr.items()),
+            )
+        )
+
+    def _add_result_to_targets(self, targets, result):
+        name = self._new_name()
+        self.__printer.singleton_pprinters.setdefault(
+            id(result), lambda obj, p, cycle: p.text(name)
+        )
+        self.names_to_values[name] = result
+        for target in targets:
+            self.bundles.setdefault(target, []).append(VarReference(name))
+
+    def check_invariants(self):
+        for invar in self.invariants():
+            if invar.precondition and not invar.precondition(self):
+                continue
+            invar.function(self)
 
     def teardown(self):
         """Called after a run has finished executing to clean up any necessary
@@ -228,36 +372,21 @@ def teardown(self):
         Does nothing by default.
         """
 
-    def check_invariants(self):
-        """Called after initializing and after executing each step."""
-
-    _test_case_cache = {}  # type: dict
-
     TestCase = TestCaseProperty()
 
     @classmethod
+    @lru_cache()
     def _to_test_case(state_machine_class):
-        try:
-            return state_machine_class._test_case_cache[state_machine_class]
-        except KeyError:
-            pass
-
         class StateMachineTestCase(TestCase):
             settings = Settings(deadline=None, suppress_health_check=HealthCheck.all())
 
-        # We define this outside of the class and assign it because you can't
-        # assign attributes to instance method values in Python 2
-        def runTest(self):
-            run_state_machine_as_test(state_machine_class)
-
-        runTest.is_hypothesis_test = True
-        StateMachineTestCase.runTest = runTest
-        base_name = state_machine_class.__name__
-        StateMachineTestCase.__name__ = base_name + ".TestCase"
-        StateMachineTestCase.__qualname__ = (
-            getattr(state_machine_class, "__qualname__", base_name) + ".TestCase"
-        )
-        state_machine_class._test_case_cache[state_machine_class] = StateMachineTestCase
+            def runTest(self):
+                run_state_machine_as_test(state_machine_class)
+
+            runTest.is_hypothesis_test = True
+
+        StateMachineTestCase.__name__ = state_machine_class.__name__ + ".TestCase"
+        StateMachineTestCase.__qualname__ = qualname(state_machine_class) + ".TestCase"
         return StateMachineTestCase
 
 
@@ -667,207 +796,3 @@ def is_valid(self, rule):
             if not bundle:
                 return False
         return True
-
-
-class RuleBasedStateMachine(_GenericStateMachine):
-    """A RuleBasedStateMachine gives you a structured way to define state machines.
-
-    The idea is that a state machine carries a bunch of types of data
-    divided into Bundles, and has a set of rules which may read data
-    from bundles (or just from normal strategies) and push data onto
-    bundles. At any given point a random applicable rule will be
-    executed.
-    """
-
-    _rules_per_class = {}  # type: Dict[type, List[classmethod]]
-    _invariants_per_class = {}  # type: Dict[type, List[classmethod]]
-    _base_rules_per_class = {}  # type: Dict[type, List[classmethod]]
-    _initializers_per_class = {}  # type: Dict[type, List[classmethod]]
-    _base_initializers_per_class = {}  # type: Dict[type, List[classmethod]]
-
-    def __init__(self):
-        if not self.rules():
-            raise InvalidDefinition("Type %s defines no rules" % (type(self).__name__,))
-        self.bundles = {}  # type: Dict[str, list]
-        self.name_counter = 1
-        self.names_to_values = {}  # type: Dict[str, Any]
-        self.__stream = StringIO()
-        self.__printer = RepresentationPrinter(self.__stream)
-        self._initialize_rules_to_run = copy(self.initialize_rules())
-        self.__rules_strategy = RuleStrategy(self)
-
-    def __pretty(self, value):
-        if isinstance(value, VarReference):
-            return value.name
-        self.__stream.seek(0)
-        self.__stream.truncate(0)
-        self.__printer.output_width = 0
-        self.__printer.buffer_width = 0
-        self.__printer.buffer.clear()
-        self.__printer.pretty(value)
-        self.__printer.flush()
-        return self.__stream.getvalue()
-
-    def __repr__(self):
-        return "%s(%s)" % (type(self).__name__, nicerepr(self.bundles))
-
-    def upcoming_name(self):
-        return "v%d" % (self.name_counter,)
-
-    def last_names(self, n):
-        assert self.name_counter > n
-        count = self.name_counter
-        return ["v%d" % (i,) for i in range(count - n, count)]
-
-    def new_name(self):
-        result = self.upcoming_name()
-        self.name_counter += 1
-        return result
-
-    def bundle(self, name):
-        return self.bundles.setdefault(name, [])
-
-    @classmethod
-    def initialize_rules(cls):
-        try:
-            return cls._initializers_per_class[cls]
-        except KeyError:
-            pass
-
-        for _, v in inspect.getmembers(cls):
-            r = getattr(v, INITIALIZE_RULE_MARKER, None)
-            if r is not None:
-                cls.define_initialize_rule(
-                    r.targets, r.function, r.arguments, r.precondition
-                )
-        cls._initializers_per_class[cls] = cls._base_initializers_per_class.pop(cls, [])
-        return cls._initializers_per_class[cls]
-
-    @classmethod
-    def rules(cls):
-        try:
-            return cls._rules_per_class[cls]
-        except KeyError:
-            pass
-
-        for _, v in inspect.getmembers(cls):
-            r = getattr(v, RULE_MARKER, None)
-            if r is not None:
-                cls.define_rule(r.targets, r.function, r.arguments, r.precondition)
-        cls._rules_per_class[cls] = cls._base_rules_per_class.pop(cls, [])
-        return cls._rules_per_class[cls]
-
-    @classmethod
-    def invariants(cls):
-        try:
-            return cls._invariants_per_class[cls]
-        except KeyError:
-            pass
-
-        target = []
-        for _, v in inspect.getmembers(cls):
-            i = getattr(v, INVARIANT_MARKER, None)
-            if i is not None:
-                target.append(i)
-        cls._invariants_per_class[cls] = target
-        return cls._invariants_per_class[cls]
-
-    @classmethod
-    def define_initialize_rule(cls, targets, function, arguments, precondition=None):
-        converted_arguments = {}
-        for k, v in arguments.items():
-            converted_arguments[k] = v
-        if cls in cls._initializers_per_class:
-            target = cls._initializers_per_class[cls]
-        else:
-            target = cls._base_initializers_per_class.setdefault(cls, [])
-
-        return target.append(Rule(targets, function, converted_arguments, precondition))
-
-    @classmethod
-    def define_rule(cls, targets, function, arguments, precondition=None):
-        converted_arguments = {}
-        for k, v in arguments.items():
-            converted_arguments[k] = v
-        if cls in cls._rules_per_class:
-            target = cls._rules_per_class[cls]
-        else:
-            target = cls._base_rules_per_class.setdefault(cls, [])
-
-        return target.append(Rule(targets, function, converted_arguments, precondition))
-
-    def steps(self):
-        # Pick initialize rules first
-        if self._initialize_rules_to_run:
-            return st.one_of(
-                [
-                    st.tuples(st.just(rule), st.fixed_dictionaries(rule.arguments))
-                    for rule in self._initialize_rules_to_run
-                ]
-            )
-
-        return self.__rules_strategy
-
-    def print_start(self):
-        report("state = %s()" % (self.__class__.__name__,))
-
-    def print_end(self):
-        report("state.teardown()")
-
-    def print_step(self, step, result):
-        rule, data = step
-        data_repr = {}
-        for k, v in data.items():
-            data_repr[k] = self.__pretty(v)
-        self.step_count = getattr(self, "step_count", 0) + 1
-        # If the step has target bundles, and the result is a MultipleResults
-        # then we want to assign to multiple variables.
-        if isinstance(result, MultipleResults):
-            n_output_vars = len(result.values)
-        else:
-            n_output_vars = 1
-        output_assignment = (
-            "%s = " % (", ".join(self.last_names(n_output_vars)),)
-            if rule.targets and n_output_vars >= 1
-            else ""
-        )
-        report(
-            "%sstate.%s(%s)"
-            % (
-                output_assignment,
-                rule.function.__name__,
-                ", ".join("%s=%s" % kv for kv in data_repr.items()),
-            )
-        )
-
-    def _add_result_to_targets(self, targets, result):
-        name = self.new_name()
-        self.__printer.singleton_pprinters.setdefault(
-            id(result), lambda obj, p, cycle: p.text(name)
-        )
-        self.names_to_values[name] = result
-        for target in targets:
-            self.bundle(target).append(VarReference(name))
-
-    def execute_step(self, step):
-        rule, data = step
-        data = dict(data)
-        for k, v in list(data.items()):
-            if isinstance(v, VarReference):
-                data[k] = self.names_to_values[v.name]
-        result = rule.function(self, **data)
-        if rule.targets:
-            if isinstance(result, MultipleResults):
-                for single_result in result.values:
-                    self._add_result_to_targets(rule.targets, single_result)
-            else:
-                self._add_result_to_targets(rule.targets, result)
-        if self._initialize_rules_to_run:
-            self._initialize_rules_to_run.remove(rule)
-        return result
-
-    def check_invariants(self):
-        for invar in self.invariants():
-            if invar.precondition and not invar.precondition(self):
-                continue
-            invar.function(self)


Test patch:
diff --git a/hypothesis-python/tests/cover/test_stateful.py b/hypothesis-python/tests/cover/test_stateful.py
--- a/hypothesis-python/tests/cover/test_stateful.py
+++ b/hypothesis-python/tests/cover/test_stateful.py
@@ -26,7 +26,6 @@
 from hypothesis.stateful import (
     Bundle,
     RuleBasedStateMachine,
-    _GenericStateMachine,
     consumes,
     initialize,
     invariant,
@@ -35,52 +34,11 @@
     rule,
     run_state_machine_as_test,
 )
-from hypothesis.strategies import (
-    binary,
-    booleans,
-    integers,
-    just,
-    lists,
-    none,
-    sampled_from,
-    tuples,
-)
+from hypothesis.strategies import binary, booleans, data, integers, just, lists
 from tests.common.utils import capture_out, raises
 
 NO_BLOB_SETTINGS = Settings(print_blob=False)
 
-
-class SetStateMachine(_GenericStateMachine):
-    def __init__(self):
-        self.elements = []
-
-    def steps(self):
-        strat = tuples(just(False), integers(0, 5))
-        if self.elements:
-            strat |= tuples(just(True), sampled_from(self.elements))
-        return strat
-
-    def execute_step(self, step):
-        delete, value = step
-        if delete:
-            self.elements.remove(value)
-            assert value not in self.elements
-        else:
-            self.elements.append(value)
-
-
-class OrderedStateMachine(_GenericStateMachine):
-    def __init__(self):
-        self.counter = 0
-
-    def steps(self):
-        return integers(self.counter - 1, self.counter + 50)
-
-    def execute_step(self, step):
-        assert step >= self.counter
-        self.counter = step
-
-
 Leaf = namedtuple("Leaf", ("label",))
 Split = namedtuple("Split", ("left", "right"))
 
@@ -297,22 +255,18 @@ def test_multiple_rules_same_func():
 
 
 def test_can_get_test_case_off_machine_instance():
-    assert SetStateMachine().TestCase is SetStateMachine().TestCase
-    assert SetStateMachine().TestCase is not None
+    assert DepthMachine().TestCase is DepthMachine().TestCase
+    assert DepthMachine().TestCase is not None
 
 
-class FlakyDrawLessMachine(_GenericStateMachine):
-    def steps(self):
-        cb = current_build_context()
-        if cb.is_final:
-            return binary(min_size=1, max_size=1)
+class FlakyDrawLessMachine(RuleBasedStateMachine):
+    @rule(d=data())
+    def action(self, d):
+        if current_build_context().is_final:
+            d.draw(binary(min_size=1, max_size=1))
         else:
-            return binary(min_size=1024, max_size=1024)
-
-    def execute_step(self, step):
-        cb = current_build_context()
-        if not cb.is_final:
-            assert 0 not in bytearray(step)
+            buffer = binary(min_size=1024, max_size=1024)
+            assert 0 not in buffer
 
 
 def test_flaky_draw_less_raises_flaky():
@@ -320,11 +274,9 @@ def test_flaky_draw_less_raises_flaky():
         FlakyDrawLessMachine.TestCase().runTest()
 
 
-class FlakyStateMachine(_GenericStateMachine):
-    def steps(self):
-        return just(())
-
-    def execute_step(self, step):
+class FlakyStateMachine(RuleBasedStateMachine):
+    @rule()
+    def action(self):
         assert current_build_context().is_final
 
 
@@ -333,15 +285,14 @@ def test_flaky_raises_flaky():
         FlakyStateMachine.TestCase().runTest()
 
 
-class FlakyRatchettingMachine(_GenericStateMachine):
+class FlakyRatchettingMachine(RuleBasedStateMachine):
     ratchet = 0
 
-    def steps(self):
+    @rule(d=data())
+    def action(self, d):
         FlakyRatchettingMachine.ratchet += 1
         n = FlakyRatchettingMachine.ratchet
-        return lists(integers(), min_size=n, max_size=n)
-
-    def execute_step(self, step):
+        d.draw(lists(integers(), min_size=n, max_size=n))
         assert False
 
 
@@ -550,14 +501,13 @@ class Foo(RuleBasedStateMachine):
 def test_minimizes_errors_in_teardown():
     counter = [0]
 
-    class Foo(_GenericStateMachine):
-        def __init__(self):
+    class Foo(RuleBasedStateMachine):
+        @initialize()
+        def init(self):
             counter[0] = 0
 
-        def steps(self):
-            return tuples()
-
-        def execute_step(self, value):
+        @rule()
+        def increment(self):
             counter[0] += 1
 
         def teardown(self):
@@ -568,15 +518,13 @@ def teardown(self):
     assert counter[0] == 1
 
 
-class RequiresInit(_GenericStateMachine):
+class RequiresInit(RuleBasedStateMachine):
     def __init__(self, threshold):
         super().__init__()
         self.threshold = threshold
 
-    def steps(self):
-        return integers()
-
-    def execute_step(self, value):
+    @rule(value=integers())
+    def action(self, value):
         if value > self.threshold:
             raise ValueError("%d is too high" % (value,))
 
@@ -588,15 +536,13 @@ def test_can_use_factory_for_tests():
         )
 
 
-class FailsEventually(_GenericStateMachine):
+class FailsEventually(RuleBasedStateMachine):
     def __init__(self):
         super().__init__()
         self.counter = 0
 
-    def steps(self):
-        return none()
-
-    def execute_step(self, _):
+    @rule()
+    def increment(self):
         self.counter += 1
         assert self.counter < 10
 
@@ -643,14 +589,14 @@ def test_saves_failing_example_in_database():
     db = ExampleDatabase(":memory:")
     with raises(AssertionError):
         run_state_machine_as_test(
-            SetStateMachine, settings=Settings(database=db, max_examples=100)
+            DepthMachine, settings=Settings(database=db, max_examples=100)
         )
     assert any(list(db.data.values()))
 
 
 def test_can_run_with_no_db():
     with raises(AssertionError):
-        run_state_machine_as_test(SetStateMachine, settings=Settings(database=None))
+        run_state_machine_as_test(DepthMachine, settings=Settings(database=None))
 
 
 def test_stateful_double_rule_is_forbidden(recwarn):