Repository: HypothesisWorks/hypothesis

Issue:
Issues with recursive data classes
Given the following:
```python
from dataclasses import dataclass
from hypothesis import given, strategies as st

@dataclass
class User():
  id: int
  following: list['User']  # works with typing.List

@given(st.builds(User))
def test_user(u):
    pass
```

I always receive the error `hypothesis.errors.InvalidArgument: thing=User must be a type`. I have tried various permutations of using `strategies.recursive` and `strategies.deferred` to define custom strategies and pass them to `@given`, but have been unable to get this to work. I'm not sure if this is a bug or an issue with the documentation (perhaps it requires more recursive examples).


Discussion:
Ah, this looks like a bug - `typing.List['User']` does work as intended, so we'll just need to track down the difference and ensure that the same logic applies when you use builtins as generics (new in py39).

Thanks for the report - I'm on a camping trip this month, but if nobody else manages to fix it before July I should get to it then 🙂 
Awesome, thanks. I'll try using the old school List until then.
Just a note: using the old typing.List does work, but is painfully slow. Running the tests above takes about ~7 seconds. I have a hunch `recursive` is the key to this but the documentation doesn't make it clear how to use it for more complex types.
I will have a look! 👍 
I found the reason for this. And as always it is `typing` API 😒 

So, let's see the difference:

```python
from typing import List
from dataclasses import dataclass

@dataclass 
class UserOld:
    following: List['User']

@dataclass 
class UserNew:
    following: list['User']
```

And now the interesting part:

```python
print(UserOld.__annotations__)
# {'following': typing.List[ForwardRef('User')]}

print(UserNew.__annotations__)
# {'following': list['User']}
```

Since we treat `ForwardRef` specially, it is later resolved to a type: `typing.List[__main__.User]`
But, `list['User']` does not get resolved to a similar type, because `'User'` is not a forward ref and is just a string.

@Zac-HD what should we do? I know that typing API in python3.9 is different. Should we treat strings in 3.9 the same way as we treat `ForwardRef` in previous versions?
Unclear what we *could* do, since we can't resolve the names in scope if typing failed to do so. Feel free to try, but I think the best we can manage in general might be a more specific error message. 
It looks like a bug in Python to me, because it does not typecheck type arguments to `list`:

```python
>>> from typing import List
>>> List[int]  # no error, ok
typing.List[int]
>>> List[1]  # error, ok
Traceback (most recent call last):
  ...
TypeError: Parameters to generic types must be types. Got 1.
>>> list[int]  # no error, ok
list[int]
>>> list[1]   # no error, not ok
list[1]
```

You can try the same with `python@3.9.1`

And `typing._type_check` is where `'User'` is converted to `ForwardRef('User')`.
Or probably this is just a new typing API, I am not sure at this point.

PEP does not mention this decision: https://www.python.org/dev/peps/pep-0585/#backwards-compatibility

P.S. Not just `list`, but other generics as well: `set[1]`, etc
Does this work with `from __future__ import annotations`?
> Does this work with from __future__ import annotations?

No, still fails for the same reason. `__annotations__` prop is `{'following': "list['User']"}`
We correctly handle `list` as a string, but `typing.get_type_hints` function fails to resolve `'User'` string into `User` type.
But in the `__future__` you can just write `list[User]`, which should hopefully resolve correctly.
Yes, this works:

```python
from __future__ import annotations

from dataclasses import dataclass
from hypothesis import given, strategies as st
from typing import List

@dataclass
class User():
  following: list[User]  # works with typing.List


print(st.from_type(User).example())
```

@JelleZijlstra thanks a lot for your help!

I guess we need to improve the docs for this error. That's the best we can do.

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/strategies/_internal/core.py b/hypothesis-python/src/hypothesis/strategies/_internal/core.py
--- a/hypothesis-python/src/hypothesis/strategies/_internal/core.py
+++ b/hypothesis-python/src/hypothesis/strategies/_internal/core.py
@@ -1007,7 +1007,15 @@ def as_strategy(strat_or_callable, thing, final=True):
                 else:
                     literals.append(arg)
             return sampled_from(literals)
-        raise InvalidArgument(f"thing={thing} must be a type")
+        if isinstance(thing, str):
+            # See https://github.com/HypothesisWorks/hypothesis/issues/3016
+            raise InvalidArgument(
+                f"Got {thing!r} as a type annotation, but the forward-reference "
+                "could not be resolved from a string to a type.  Consider using "
+                "`from __future__ import annotations` instead of forward-reference "
+                "strings."
+            )
+        raise InvalidArgument(f"thing={thing!r} must be a type")  # pragma: no cover
     # Now that we know `thing` is a type, the first step is to check for an
     # explicitly registered strategy.  This is the best (and hopefully most
     # common) way to resolve a type to a strategy.  Note that the value in the


Test patch:
diff --git a/hypothesis-python/tests/cover/test_lookup_py39.py b/hypothesis-python/tests/cover/test_lookup_py39.py
--- a/hypothesis-python/tests/cover/test_lookup_py39.py
+++ b/hypothesis-python/tests/cover/test_lookup_py39.py
@@ -13,11 +13,13 @@
 #
 # END HEADER
 
+import dataclasses
 import typing
 
 import pytest
 
 from hypothesis import strategies as st
+from hypothesis.errors import InvalidArgument
 
 
 @pytest.mark.parametrize(
@@ -36,3 +38,16 @@
 )
 def test_typing_Annotated(annotated_type, expected_strategy_repr):
     assert repr(st.from_type(annotated_type)) == expected_strategy_repr
+
+
+@dataclasses.dataclass
+class User:
+    id: int
+    following: list["User"]  # works with typing.List
+
+
+def test_string_forward_ref_message():
+    # See https://github.com/HypothesisWorks/hypothesis/issues/3016
+    s = st.builds(User)
+    with pytest.raises(InvalidArgument, match="`from __future__ import annotations`"):
+        s.example()