Repository: HypothesisWorks/hypothesis

Issue:
When `st.from_type()` falls back to `st.builds()`, try passing `infer` for arguments with default values
For functions, `builds` uses a `just(default_value)` strategy for kwargs with a `default_value`, but it also allows you to use `infer` to instead use something to the effect of `just(default_value) | from_type(type_annotation)` [1].

For Pydantic models, Hypothesis inspects the `__signature__` [2] and will also use the equivalent of a `just(default_value)` strategy for fields with a `default_value` (cf. example below). It would be nice if we could also choose to use an `infer`-like strategy in this scenario. What would be the recommended workaround for that?

Side question: why was the default strategy for kwargs/fields with default values chosen to be `just(default_value)` instead of `just(default_value) | from_type(type_annotation)`?

```python
from typing import Optional
import hypothesis.strategies as st
from pydantic import BaseModel

class Foo(BaseModel):
    bar: int
    baz: Optional[str] = None

st.from_type(Foo).example()  # `baz` will always be `None`, even though you might expect to see some `str`s
```

[1] https://hypothesis.readthedocs.io/en/latest/details.html#inferred-strategies
[2] https://github.com/HypothesisWorks/hypothesis/pull/2388


Discussion:
> Why was the default strategy for kwargs/fields with default values chosen to be `just(default_value)` instead of `just(default_value) | from_type(type_annotation)`?

I think this, and the difference between `builds(Foo)` and `from_type(Foo)`, is actually the heart of the matter.

`builds(Foo, ...)` *calls `Foo`* with arguments drawn from the provided strategies, and we later added support for inferring strategies for missing required arguments - which was previously an error.  There's not actually a `just(default_value)` involved here; we're simply not providing any argument for parameters which have a default value *unless* explicitly told to do so via `baz=infer`.

On the other hand, `from_type(Foo)` does not have any particular meaning beyond "somehow generates instances of `Foo` - as implemented, we try looking up registered strategies for the exact type, then subtypes, and finally use `builds()` if we couldn't find anything.  The 'working as intended' workaround is to `st.register_type_strategy(Foo, st.builds(Foo, baz=infer)` before `st.from_type(Foo)`.

-----------

However, I think it would make sense to have `from_type(Foo)` call `builds(Foo, baz=infer)` internally - i.e. pass `infer` for args with default values - because the current conservative behaviour of `builds()` is an implementation detail of `from_type()` rather than part of the contract.

We would need to be careful not to break backwards compatibility by raising errors on `arg: UnresolvableType = ...`; this is certainly possible with some more introspection and a call to `try: inferred_strategy.validate(); except ...`.
@Zac-HD This also has implications when registering types with contracts in Hypothesis. When you register a strategy for `Foo`, `from_type` works correctly, where `builds(Foo)` does not look up the registered strategy for the type.

Is that behavior intented?
Yes, the current behaviour is intended.  

For any registered type, `from_type()` will return whatever you registered, while `builds()` will call the constructor - which is the intended behaviour.  The idea for this issue is that *if* nothing is registered and `from_type()` delegates to `builds()`, it would be appropriate to infer strategies for *all* arguments - after all, if that's not what the user wants they can always use `builds()` directly and/or register a strategy!
@Zac-HD thanks for the clarification. I'll document that the types with contracts in constructor should always use `from_type` and never `builds`. As these classes with contracts are automatically registered, this should work as expected.

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/strategies/_internal/core.py b/hypothesis-python/src/hypothesis/strategies/_internal/core.py
--- a/hypothesis-python/src/hypothesis/strategies/_internal/core.py
+++ b/hypothesis-python/src/hypothesis/strategies/_internal/core.py
@@ -1593,9 +1593,29 @@ def as_strategy(strat_or_callable, thing, final=True):
             "Could not resolve %r to a strategy; consider "
             "using register_type_strategy" % (thing,)
         )
-    # Finally, try to build an instance by calling the type object
+    # Finally, try to build an instance by calling the type object.  Unlike builds(),
+    # this block *does* try to infer strategies for arguments with default values.
+    # That's because of the semantic different; builds() -> "call this with ..."
+    # so we only infer when *not* doing so would be an error; from_type() -> "give
+    # me arbitrary instances" so the greater variety is acceptable.
+    # And if it's *too* varied, express your opinions with register_type_strategy()
     if not isabstract(thing):
-        return builds(thing)
+        try:
+            hints = get_type_hints(thing)
+            params = signature(thing).parameters
+        except Exception:
+            return builds(thing)
+        kwargs = {}
+        for k, p in params.items():
+            if (
+                k in hints
+                and k != "return"
+                and p.default is not Parameter.empty
+                and p.kind in (Parameter.POSITIONAL_OR_KEYWORD, Parameter.KEYWORD_ONLY)
+            ):
+                kwargs[k] = just(p.default) | _from_type(hints[k])
+        return builds(thing, **kwargs)
+    # And if it's an abstract type, we'll resolve to a union of subclasses instead.
     subclasses = thing.__subclasses__()
     if not subclasses:
         raise ResolutionFailed(


Test patch:
diff --git a/hypothesis-python/tests/cover/test_lookup.py b/hypothesis-python/tests/cover/test_lookup.py
--- a/hypothesis-python/tests/cover/test_lookup.py
+++ b/hypothesis-python/tests/cover/test_lookup.py
@@ -793,3 +793,13 @@ def __init__(self, value: int) -> None:
 def test_signature_is_the_most_important_source(data):
     """Signature types should take precedence over all other annotations."""
     data.draw(st.builds(AnnotatedConstructorWithSignature))
+
+
+class AnnotatedAndDefault:
+    def __init__(self, foo: bool = None):
+        self.foo = foo
+
+
+def test_from_type_can_be_default_or_annotation():
+    find_any(st.from_type(AnnotatedAndDefault), lambda x: x.foo is None)
+    find_any(st.from_type(AnnotatedAndDefault), lambda x: isinstance(x.foo, bool))
diff --git a/hypothesis-python/tests/nocover/test_build_signature.py b/hypothesis-python/tests/nocover/test_build_signature.py
--- a/hypothesis-python/tests/nocover/test_build_signature.py
+++ b/hypothesis-python/tests/nocover/test_build_signature.py
@@ -17,6 +17,7 @@
 from typing import List, get_type_hints
 
 from hypothesis import given, strategies as st
+from tests.common.debug import find_any
 
 
 def use_this_signature(self, a: int, b: bool = None, *, x: float, y: str):
@@ -42,9 +43,21 @@ def test_builds_uses_signature_attribute(val):
     assert isinstance(val, Model)
 
 
-@given(st.from_type(Model))
+class ModelForFromType(Model):
+    def __init__(self, **kwargs):
+        assert set(kwargs) == {"a", "b", "x", "y"}
+        self.b = kwargs["b"]
+        assert self.b is None or isinstance(self.b, bool)
+
+
+@given(st.from_type(ModelForFromType))
 def test_from_type_uses_signature_attribute(val):
-    assert isinstance(val, Model)
+    assert isinstance(val, ModelForFromType)
+
+
+def test_from_type_can_be_default_or_annotation():
+    find_any(st.from_type(ModelForFromType), lambda m: m.b is None)
+    find_any(st.from_type(ModelForFromType), lambda m: isinstance(m.b, bool))
 
 
 def use_annotations(
@@ -93,3 +106,14 @@ def __init__(self, **kwargs):
 @given(st.builds(ModelWithBadAliasSignature))
 def test_build_with_non_types_in_signature(val):
     assert isinstance(val, ModelWithBadAliasSignature)
+
+
+class UnconventionalSignature:
+    def __init__(x: int = 0, self: bool = True):
+        assert not isinstance(x, int)
+        x.self = self
+
+
+def test_build_in_from_type_with_self_named_something_else():
+    find_any(st.from_type(UnconventionalSignature), lambda x: x.self is True)
+    find_any(st.from_type(UnconventionalSignature), lambda x: x.self is False)