Repository: HypothesisWorks/hypothesis

Issue:
Check for settings assigned as attribute on StateMachine class
#1638 allows `@settings(...)` to be used as a class decorator on state machine classes, which substantially improves the ergonomics.  However, the old mechanism of assigning an attribute is still supported.  #1642 points out a fairly subtle trap in this behavior though, by fixing it in the docs:

```python
DatabaseComparison.settings = settings(...)  # Wrong!  This doesn't have any effect.
DatabaseComparison.TestCase.settings = settings(...)  # The right way to do it.
```

Fortunately we can use essentially the same solution as in #1572, defining a custom `__setattr__` on a (new) metaclass of `GenericStateMachine` that will turn the incorrect usage from silent failure into an explicit error with a helpful message (which can also suggest the decorator form).

This should error when the `.settings` attribute on the class is assigned an instance of `settings`, but allow other types to be assigned to minimize false-positives.  We should not restrict attributes on instances if we can avoid it.


Discussion:
This issue would be a good introduction to Hypothesis for an intermediate programmer who understands metaclasses, or at least can follow the example of `hypothesis.settings` to customise the behaviour of the class object but not instances thereof.
Hey Zac, I would like to give this a shot! Before I start, I have a question: There is a 2.7/3.x compatibility issue with the way metaclasses are assigned to classes. There are solutions to this in `six` and `future`, but it looks like these are not existing dependencies within hypothesis. What would be the best way to proceed?
I would simply copy the implementation of `settingsMeta`, which provides the `__setattr__` for the `settings` class object.

https://github.com/HypothesisWorks/hypothesis/blob/dd46aca7e521fccc81ef619704dfd363ebbe8c3a/hypothesis-python/src/hypothesis/_settings.py#L99-L135

As far as I'm aware this is compatible with both Python2 and Python3, though I could be wrong!
Thanks for clarifying, Zac. I misunderstood how `settingsMeta` worked to create a metaclass. This does avoid the compatability error that I was thinking of, which was 3.x's metaclass keword argument vs 2.7's \_\_metaclass\_\_ property.

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/stateful.py b/hypothesis-python/src/hypothesis/stateful.py
--- a/hypothesis-python/src/hypothesis/stateful.py
+++ b/hypothesis-python/src/hypothesis/stateful.py
@@ -121,7 +121,25 @@ def run_state_machine_as_test(state_machine_factory, settings=None):
     )
 
 
-class GenericStateMachine(object):
+class GenericStateMachineMeta(type):
+
+    def __init__(self, *args, **kwargs):
+        super(GenericStateMachineMeta, self).__init__(*args, **kwargs)
+
+    def __setattr__(self, name, value):
+        if name == 'settings' and isinstance(value, Settings):
+            raise AttributeError(
+                ('Assigning {cls}.settings = {value} does nothing. Assign '
+                 'to {cls}.TestCase.settings, or use @{value} as a decorator '
+                 'on the {cls} class.').format(cls=self.__name__, value=value)
+            )
+        return type.__setattr__(self, name, value)
+
+
+class GenericStateMachine(
+    GenericStateMachineMeta('GenericStateMachine',  # type: ignore
+                            (object,), {})
+):
     """A GenericStateMachine is the basic entry point into Hypothesis's
     approach to stateful testing.
 


Test patch:
diff --git a/hypothesis-python/tests/cover/test_settings.py b/hypothesis-python/tests/cover/test_settings.py
--- a/hypothesis-python/tests/cover/test_settings.py
+++ b/hypothesis-python/tests/cover/test_settings.py
@@ -33,7 +33,8 @@
     checks_deprecated_behaviour
 from hypothesis.database import ExampleDatabase, \
     DirectoryBasedExampleDatabase
-from hypothesis.stateful import RuleBasedStateMachine, rule
+from hypothesis.stateful import GenericStateMachine, \
+    RuleBasedStateMachine, rule
 from hypothesis._settings import Verbosity, PrintSettings, settings, \
     default_variable, note_deprecation
 from hypothesis.utils.conventions import not_set
@@ -459,3 +460,13 @@ def test_settings_decorator_applied_to_non_state_machine_class_raises_error():
         @settings()
         class NonStateMachine:
             pass
+
+
+def test_assigning_to_settings_attribute_on_state_machine_raises_error():
+    with pytest.raises(AttributeError):
+        class StateMachine(GenericStateMachine):
+            pass
+        StateMachine.settings = settings()
+
+    state_machine_instance = StateMachine()
+    state_machine_instance.settings = 'any value'