Repository: HypothesisWorks/hypothesis

Issue:
Hypothesis is not compatible with Python 3.7.0b3's typing module.
In order to account for changes to the Python typing module made by https://github.com/python/cpython/commit/d911e40e788fb679723d78b6ea11cabf46caed5a I am applying the following patch which may be ok, but more likely is not (the typing module is not something with which I am at all familiar):

```diff
From 27a17ef17be0e518ac06152442966e4dbb193fc5 Mon Sep 17 00:00:00 2001
From: Ray Donnelly <mingw.android@gmail.com>
Date: Tue, 1 May 2018 16:45:42 +0100
Subject: [PATCH] Fix typing _ForwardRef and TypingMeta for Python 3.7

These fixes are likely incorrect though.
---
 hypothesis-python/src/hypothesis/strategies.py | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/hypothesis-python/src/hypothesis/strategies.py b/hypothesis-python/src/hypothesis/strategies.py
index e324784b..82ebe842 100644
--- a/hypothesis-python/src/hypothesis/strategies.py
+++ b/hypothesis-python/src/hypothesis/strategies.py
@@ -1067,6 +1067,11 @@ def from_type(thing):
     from hypothesis.searchstrategy import types
     try:
         import typing
+        # Since https://github.com/python/cpython/commit/d911e40e788fb679723d78b6ea11cabf46caed5a
+        if hasattr(typing, '_ForwardRef'):
+            typing_ForwardRef = typing._ForwardRef
+        else:
+            typing_ForwardRef = typing.ForwardRef
         if not isinstance(thing, type):
             # At runtime, `typing.NewType` returns an identity function rather
             # than an actual type, but we can check that for a possible match
@@ -1083,7 +1088,7 @@ def from_type(thing):
                 return one_of([from_type(t) for t in args])
         # We can't resolve forward references, and under Python 3.5 (only)
         # a forward reference is an instance of type.  Hence, explicit check:
-        elif type(thing) == typing._ForwardRef:  # pragma: no cover
+        elif type(thing) == typing_ForwardRef:  # pragma: no cover
             raise ResolutionFailed(
                 'thing=%s cannot be resolved.  Upgrading to python>=3.6 may '
                 'fix this problem via improvements to the typing module.'
@@ -1112,7 +1117,11 @@ def from_type(thing):
     # subclass and instance checks.
     try:
         import typing
-        if isinstance(thing, typing.TypingMeta):
+        if hasattr(typing, 'TypingMeta'):
+            typing_TypeVar = typing.TypingMeta
+        else:
+            typing_TypeVar = typing.TypeVar
+        if isinstance(thing, typing_TypeVar):
             return types.from_typing_type(thing)
     except ImportError:  # pragma: no cover
         pass
-- 
2.17.0
```

Best regards.
We should test against Python 3.7
See https://www.python.org/downloads/release/python-370b2/

As well as helping to test Python itself, we get to check that nothing will suddenly break come June.

3.7-related considerations for Hypothesis:

- The delayed evaluation of type hints should be fine (`typing.get_type_hints` still does the heavy lifting there), but there may be some other changes to the typing module that require tweaks to code or tests around strategy inference.  #1004 still looks hopeless.
- `builds()` should introspect the new dataclasses.  See #954 for `attrs` equivalent.


Discussion:
If you need a build of Python 3.7.0b3 (and 207 python packages) against which to test `hypothesis`, and you are a user of the `Anaconda Distribution` and `macOS`, you can use `conda create -n py27 -c c3i_test python=3.7` (other OSes to follow).
Thanks for reporting this!  Compatibility bugs are always a high priority for us :smile: 

@DRMacIver / @alexwlchan, can one of you help out with #1148?  `typing` module changes are explicitly called out there (in fact we would have caught this earlier), but my attempt to add it to CI failed :sob: 

I'm not keen to merge a fix before it can be tested, but with that out of the way this will be at the top of my list.
> I'm not keen to merge a fix before it can be tested

Can/do you use `miniconda` in your CI? If so you could use `c3i_test` to get Python 3.7.0b3.
Is there any progress on this? Python 3.7 has been released now.
> Is there any progress on this? Python 3.7 has been released now.

It has, but the release situation is frankly terrible and we're not sure what to do about it. 😢 

The hard dependency on a new version of OpenSSL means that the release basically doesn't work on any of our CI infrastructure, and we're not able to fix that without making everything substantially worse or doing a bunch of build engineering to build OpenSSL ourself, which we haven't found time for yet.
Maybe try `conda`?, we handle all of that stuff for you.
https://github.com/conda-forge/hypothesis-feedstock
Welp.  Too late for pre-releases, but we can use the issue to track general progress on 3.7 support 🚀 🙄 
See https://github.com/HypothesisWorks/hypothesis/pull/1369

Basically: This isn't going to happen any time soon.

The following scenarios would allow it to happen:

1. Travis makes it possible to test on a modern version of Linux
2. CPython makes it easy to statically link OpenSSL
3. Someone else (pyenv?) does the wrangling that would allow us install this with a custom OpenSSL easily.
4. We switch our build environment over to using anaconda somehow.

Basically I'm grumpy and pissed off about this whole thing and super not invested in supporting Python 3.7, so I'm declaring this to be somebody else's problem.
We now have 3.7 running on Travis, thanks to #1376 (:heart_eyes: @asottile)

It currently has an allowed failure because of [a console unicode issue here](https://github.com/HypothesisWorks/hypothesis/blob/b1518f16f9b60969dc0419ea30ff25f1df6ec674/hypothesis-python/tests/pytest/test_capture.py#L75-L84), thanks to [PEP 538](https://www.python.org/dev/peps/pep-0538/).  All we need to do for *this* is to accept the character (`ϩ`) even if it's not displayed as `\u03e9`!

I do expect some other problems to come up though :smile: 

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/internal/compat.py b/hypothesis-python/src/hypothesis/internal/compat.py
--- a/hypothesis-python/src/hypothesis/internal/compat.py
+++ b/hypothesis-python/src/hypothesis/internal/compat.py
@@ -37,8 +37,13 @@
     from ordereddict import OrderedDict  # type: ignore
     from counter import Counter  # type: ignore
 
+try:
+    from collections import abc
+except ImportError:
+    import collections as abc  # type: ignore
+
 if False:
-    from typing import Type  # noqa
+    from typing import Type, Tuple  # noqa
 
 
 PY2 = sys.version_info[0] == 2
@@ -273,6 +278,21 @@ def qualname(f):
         return f.__name__
 
 
+try:
+    import typing
+except ImportError:
+    typing_root_type = ()  # type: Tuple[type, ...]
+    ForwardRef = None
+else:
+    if hasattr(typing, '_Final'):  # new in Python 3.7
+        typing_root_type = (
+            typing._Final, typing._GenericAlias)  # type: ignore
+        ForwardRef = typing.ForwardRef  # type: ignore
+    else:
+        typing_root_type = (typing.TypingMeta, typing.TypeVar)  # type: ignore
+        ForwardRef = typing._ForwardRef  # type: ignore
+
+
 if PY2:
     FullArgSpec = namedtuple('FullArgSpec', 'args, varargs, varkw, defaults, '
                              'kwonlyargs, kwonlydefaults, annotations')
diff --git a/hypothesis-python/src/hypothesis/searchstrategy/types.py b/hypothesis-python/src/hypothesis/searchstrategy/types.py
--- a/hypothesis-python/src/hypothesis/searchstrategy/types.py
+++ b/hypothesis-python/src/hypothesis/searchstrategy/types.py
@@ -27,27 +27,38 @@
 
 import hypothesis.strategies as st
 from hypothesis.errors import InvalidArgument, ResolutionFailed
-from hypothesis.internal.compat import PY2, text_type
+from hypothesis.internal.compat import PY2, ForwardRef, abc, text_type, \
+    typing_root_type
 
 
 def type_sorting_key(t):
     """Minimise to None, then non-container types, then container types."""
-    if not isinstance(t, type):
+    if not is_a_type(t):
         raise InvalidArgument('thing=%s must be a type' % (t,))
     if t is None or t is type(None):  # noqa: E721
         return (-1, repr(t))
-    return (issubclass(t, collections.Container), repr(t))
+    if not isinstance(t, type):  # pragma: no cover
+        # Some generics in the typing module are not actually types in 3.7
+        return (2, repr(t))
+    return (int(issubclass(t, abc.Container)), repr(t))
 
 
-def try_issubclass(thing, maybe_superclass):
+def try_issubclass(thing, superclass):
+    thing = getattr(thing, '__origin__', None) or thing
+    superclass = getattr(superclass, '__origin__', None) or superclass
     try:
-        return issubclass(thing, maybe_superclass)
+        return issubclass(thing, superclass)
     except (AttributeError, TypeError):  # pragma: no cover
         # Some types can't be the subject or object of an instance or
         # subclass check under Python 3.5
         return False
 
 
+def is_a_type(thing):
+    """Return True if thing is a type or a generic type like thing."""
+    return isinstance(thing, type) or isinstance(thing, typing_root_type)
+
+
 def from_typing_type(thing):
     # We start with special-case support for Union and Tuple - the latter
     # isn't actually a generic type.  Support for Callable may be added to
@@ -65,22 +76,32 @@ def from_typing_type(thing):
         if not args:
             raise ResolutionFailed('Cannot resolve Union of no types.')
         return st.one_of([st.from_type(t) for t in args])
-    if isinstance(thing, typing.TupleMeta):
+    if getattr(thing, '__origin__', None) == tuple or \
+            isinstance(thing, getattr(typing, 'TupleMeta', ())):
         elem_types = getattr(thing, '__tuple_params__', None) or ()
         elem_types += getattr(thing, '__args__', None) or ()
         if getattr(thing, '__tuple_use_ellipsis__', False) or \
                 len(elem_types) == 2 and elem_types[-1] is Ellipsis:
             return st.lists(st.from_type(elem_types[0])).map(tuple)
         return st.tuples(*map(st.from_type, elem_types))
+    if isinstance(thing, typing.TypeVar):
+        if getattr(thing, '__bound__', None) is not None:
+            return st.from_type(thing.__bound__)
+        if getattr(thing, '__constraints__', None):
+            return st.shared(
+                st.sampled_from(thing.__constraints__),
+                key='typevar-with-constraint'
+            ).flatmap(st.from_type)
+        # Constraints may be None or () on various Python versions.
+        return st.text()  # An arbitrary type for the typevar
     # Now, confirm that we're dealing with a generic type as we expected
-    if not isinstance(thing, typing.GenericMeta):  # pragma: no cover
+    if not isinstance(thing, typing_root_type):  # pragma: no cover
         raise ResolutionFailed('Cannot resolve %s to a strategy' % (thing,))
     # Parametrised generic types have their __origin__ attribute set to the
     # un-parametrised version, which we need to use in the subclass checks.
     # e.g.:     typing.List[int].__origin__ == typing.List
     mapping = {k: v for k, v in _global_type_lookup.items()
-               if isinstance(k, typing.GenericMeta) and
-               try_issubclass(k, getattr(thing, '__origin__', None) or thing)}
+               if isinstance(k, typing_root_type) and try_issubclass(k, thing)}
     if typing.Dict in mapping:
         # The subtype relationships between generic and concrete View types
         # are sometimes inconsistent under Python 3.5, so we pop them out to
@@ -206,10 +227,10 @@ def resolve_Type(thing):
             args = args[0].__args__
         elif hasattr(args[0], '__union_params__'):  # pragma: no cover
             args = args[0].__union_params__
-        if isinstance(typing._ForwardRef, type):  # pragma: no cover
+        if isinstance(ForwardRef, type):  # pragma: no cover
             # Duplicate check from from_type here - only paying when needed.
             for a in args:
-                if type(a) == typing._ForwardRef:
+                if type(a) == ForwardRef:
                     raise ResolutionFailed(
                         'thing=%s cannot be resolved.  Upgrading to '
                         'python>=3.6 may fix this problem via improvements '
diff --git a/hypothesis-python/src/hypothesis/strategies.py b/hypothesis-python/src/hypothesis/strategies.py
--- a/hypothesis-python/src/hypothesis/strategies.py
+++ b/hypothesis-python/src/hypothesis/strategies.py
@@ -38,7 +38,8 @@
 from hypothesis.internal.cache import LRUReusedCache
 from hypothesis.searchstrategy import SearchStrategy, check_strategy
 from hypothesis.internal.compat import gcd, ceil, floor, hrange, \
-    string_types, get_type_hints, getfullargspec, implements_iterator
+    string_types, get_type_hints, getfullargspec, typing_root_type, \
+    implements_iterator
 from hypothesis.internal.floats import next_up, next_down, is_negative, \
     float_to_int, int_to_float, count_between_floats
 from hypothesis.internal.charmap import as_general_categories
@@ -1189,7 +1190,6 @@ def from_type(thing):
     from hypothesis.searchstrategy import types
 
     if typing is not None:  # pragma: no branch
-        fr = typing._ForwardRef
         if not isinstance(thing, type):
             # At runtime, `typing.NewType` returns an identity function rather
             # than an actual type, but we can check that for a possible match
@@ -1206,12 +1206,13 @@ def from_type(thing):
                 return one_of([from_type(t) for t in args])
         # We can't resolve forward references, and under Python 3.5 (only)
         # a forward reference is an instance of type.  Hence, explicit check:
-        elif type(thing) == fr:  # pragma: no cover
+        elif hasattr(typing, '_ForwardRef') and \
+                type(thing) == typing._ForwardRef:  # pragma: no cover
             raise ResolutionFailed(
                 'thing=%s cannot be resolved.  Upgrading to python>=3.6 may '
                 'fix this problem via improvements to the typing module.'
                 % (thing,))
-    if not isinstance(thing, type):
+    if not types.is_a_type(thing):
         raise InvalidArgument('thing=%s must be a type' % (thing,))
     # Now that we know `thing` is a type, the first step is to check for an
     # explicitly registered strategy.  This is the best (and hopefully most
@@ -1232,7 +1233,7 @@ def from_type(thing):
     # because there are several special cases that don't play well with
     # subclass and instance checks.
     if typing is not None:  # pragma: no branch
-        if isinstance(thing, typing.TypingMeta):
+        if isinstance(thing, typing_root_type):
             return types.from_typing_type(thing)
     # If it's not from the typing module, we get all registered types that are
     # a subclass of `thing` and are not themselves a subtype of any other such
@@ -1241,7 +1242,7 @@ def from_type(thing):
     strategies = [
         v if isinstance(v, SearchStrategy) else v(thing)
         for k, v in types._global_type_lookup.items()
-        if issubclass(k, thing) and sum(
+        if isinstance(k, type) and issubclass(k, thing) and sum(
             types.try_issubclass(k, typ) for typ in types._global_type_lookup
         ) == 1
     ]
@@ -2072,7 +2073,7 @@ def register_type_strategy(custom_type, strategy):
     # TODO: We would like to move this to the top level, but pending some major
     # refactoring it's hard to do without creating circular imports.
     from hypothesis.searchstrategy import types
-    if not isinstance(custom_type, type):
+    if not types.is_a_type(custom_type):
         raise InvalidArgument('custom_type=%r must be a type')
     elif not (isinstance(strategy, SearchStrategy) or callable(strategy)):
         raise InvalidArgument(


Test patch:
diff --git a/hypothesis-python/tests/py3/test_asyncio.py b/hypothesis-python/tests/py3/test_asyncio.py
--- a/hypothesis-python/tests/py3/test_asyncio.py
+++ b/hypothesis-python/tests/py3/test_asyncio.py
@@ -55,6 +55,7 @@ def g():
             raise error
 
     @given(st.text())
+    @asyncio.coroutine
     def test_foo(self, x):
         assume(x)
         yield from asyncio.sleep(0.001)
diff --git a/hypothesis-python/tests/py3/test_lookup.py b/hypothesis-python/tests/py3/test_lookup.py
--- a/hypothesis-python/tests/py3/test_lookup.py
+++ b/hypothesis-python/tests/py3/test_lookup.py
@@ -26,20 +26,24 @@
 import pytest
 
 import hypothesis.strategies as st
-from hypothesis import find, given, infer, assume
+from hypothesis import HealthCheck, find, given, infer, assume, settings
 from hypothesis.errors import NoExamples, InvalidArgument, ResolutionFailed
 from hypothesis.strategies import from_type
 from hypothesis.searchstrategy import types
-from hypothesis.internal.compat import integer_types, get_type_hints
+from hypothesis.internal.compat import ForwardRef, integer_types, \
+    get_type_hints, typing_root_type
 
 typing = pytest.importorskip('typing')
 sentinel = object()
 generics = sorted((t for t in types._global_type_lookup
-                   if isinstance(t, typing.GenericMeta)), key=str)
+                   if isinstance(t, typing_root_type)), key=str)
 
 
 @pytest.mark.parametrize('typ', generics)
 def test_resolve_typing_module(typ):
+    @settings(suppress_health_check=[
+        HealthCheck.too_slow, HealthCheck.filter_too_much
+    ])
     @given(from_type(typ))
     def inner(ex):
         if typ in (typing.BinaryIO, typing.TextIO):
@@ -49,6 +53,8 @@ def inner(ex):
             assert ex == ()
         elif isinstance(typ, typing._ProtocolMeta):
             pass
+        elif typ is typing.Type and not isinstance(typing.Type, type):
+            assert isinstance(ex, typing.TypeVar)
         else:
             try:
                 assert isinstance(ex, typ)
@@ -227,6 +233,21 @@ def test_resolves_weird_types(typ):
     from_type(typ).example()
 
 
+@pytest.mark.parametrize('var,expected', [
+    (typing.TypeVar('V'), object),
+    (typing.TypeVar('V', bound=int), int),
+    (typing.TypeVar('V', int, str), (int, str)),
+])
+@given(data=st.data())
+def test_typevar_type_is_consistent(data, var, expected):
+    strat = st.from_type(var)
+    v1 = data.draw(strat)
+    v2 = data.draw(strat)
+    assume(v1 != v2)  # Values may vary, just not types
+    assert type(v1) == type(v2)
+    assert isinstance(v1, expected)
+
+
 def annotated_func(a: int, b: int=2, *, c: int, d: int=4):
     return a + b + c + d
 
@@ -391,7 +412,7 @@ def test_override_args_for_namedtuple(thing):
 def test_cannot_resolve_bare_forward_reference(thing):
     with pytest.raises(InvalidArgument):
         t = thing['int']
-        if type(getattr(t, '__args__', [None])[0]) != typing._ForwardRef:
+        if type(getattr(t, '__args__', [None])[0]) != ForwardRef:
             assert sys.version_info[:2] == (3, 5)
             pytest.xfail('python 3.5 typing module is really weird')
         st.from_type(t).example()
diff --git a/hypothesis-python/tests/pytest/test_capture.py b/hypothesis-python/tests/pytest/test_capture.py
--- a/hypothesis-python/tests/pytest/test_capture.py
+++ b/hypothesis-python/tests/pytest/test_capture.py
@@ -81,7 +81,8 @@ def test_output_emitting_unicode(testdir, monkeypatch):
         script, '--verbose', '--capture=no')
     out = '\n'.join(result.stdout.lines)
     assert 'test_emits_unicode' in out
-    assert escape_unicode_characters(hunichr(1001)) in out
+    assert hunichr(1001) in out or \
+        escape_unicode_characters(hunichr(1001)) in out
     assert result.ret == 0