Repository: HypothesisWorks/hypothesis

Issue:
Improve documentation of stateful testing
As per discussion in #1189, the documentation of stateful testing is currently a bit of a shambles.

Specific issues I'm aware of:

* [x] autoclass documentation for `GenericStateMachine`
* [x] autoclass documentation for `RuleBasedStateMachine` (but do we actually need this? It doesn't introduce any new members!)
* [x] It's totally unclear why you would ever want to use `GenericStateMachine`
* [ ] `Bundle` is very underdocumented (the docs don't even say it's a strategy!)
* [ ] Better examples
* [ ] More examples. e.g.
    * [ ] Example of `RuleBasedStateMachine` that doesn't use bundles.

There's also the general problem that the documentation is weird because the API is weird, but that's out of scope for this issue.


Discussion:
Note that some of this will involve deciding what exactly is and isn't public API in the stateful testing. e.g. see discussion of the new printing methods in #1189.
#1205 helps a bit but doesn't close this issue :smile: 
Notes on this topic from a conversation with David, so we don't forget them later:

We can introduce the important ideas of stateful testing in several stages, which is both easier to learn and probably the right way to pick which one to use.

1. Interactive drawing with `@given` and the `data()` strategy.  Use this iff you need to run a test, where some part of the input depends on an intermediate result from the code under test.  (If your data has complicated dependencies but can be generated 'up front', you need the related-but-simpler `composite` decorator to define a custom strategy)
2. The rule-based stateful class, using instance variable(s) to store any state.  A single variable for the thing-being-worked-on is usually sufficient, but this can include sampling from collections within an `@rule` method.
3. Bundles.  What are they good for?  Distinguishing inputs by identity instead of value, which can be important if you're e.g. appending to lists and at each step you need to know which list you touched instead of what it looked like afterwards.  Stick to (1) or (2) if you can...

Overall, we should probably break the page down into sections: the taxonomy above, then a technical API reference (autogenerated from docstrings), and finally a bunch of motivating examples.  See #774 for more on that.
The upcoming deprecation of `GenericStateMachine` in #1693 means we don't have to include more than a side note about it in the new docs :grin: 
Per #1785, we should note that using `run_state_machine_as_test` will prevent statistics reporting for that run.  This is because Pytest never sees an item with the data to report.

Perhaps we should deprecate `run_state_machine_as_test` in favor of `@initialize` methods?

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/_settings.py b/hypothesis-python/src/hypothesis/_settings.py
--- a/hypothesis-python/src/hypothesis/_settings.py
+++ b/hypothesis-python/src/hypothesis/_settings.py
@@ -417,6 +417,15 @@ class Setting(object):
     description="""
 Once this many satisfying examples have been considered without finding any
 counter-example, falsification will terminate.
+
+The default value is chosen to suit a workflow where the test will be part of
+a suite that is regularly executed locally or on a CI server, balancing total
+running time against the chance of missing a bug.
+
+If you are writing one-off tests, running tens of thousands of examples is
+quite reasonable as Hypothesis may miss uncommon bugs with default settings.
+For very complex code, we have observed Hypothesis finding novel bugs after
+*several million* examples while testing :pypi:`SymPy`.
 """,
 )
 
diff --git a/hypothesis-python/src/hypothesis/_strategies.py b/hypothesis-python/src/hypothesis/_strategies.py
--- a/hypothesis-python/src/hypothesis/_strategies.py
+++ b/hypothesis-python/src/hypothesis/_strategies.py
@@ -1146,6 +1146,7 @@ def builds(
     If the callable is a class defined with :pypi:`attrs`, missing required
     arguments will be inferred from the attribute on a best-effort basis,
     e.g. by checking :ref:`attrs standard validators <attrs:api_validators>`.
+    Dataclasses are handled natively by the inference from type hints.
 
     Examples from this strategy shrink by shrinking the argument values to
     the callable.


Test patch: