Repository: HypothesisWorks/hypothesis

Issue:
the 3.64.x release has problems on py3.5
The builds on conda forge for 3.64.x fail internal tests on all platforms for python 3.5. 
Build logs:
Linux: https://circleci.com/gh/jochym/hypothesis-feedstock/284
OSX (3.64.0): https://travis-ci.org/conda-forge/hypothesis-feedstock/jobs/397619120
OSX (3.64.2): https://travis-ci.org/conda-forge/hypothesis-feedstock/jobs/397836319 (queued)
Windows: https://ci.appveyor.com/project/conda-forge/hypothesis-feedstock/build/1.0.223/job/2sen81r9mc9215ni

Is 3.5 still supported?
py3 tests are only run in coverage job
Based on some investigation in response to #1366 it looks like the tests in `tests/py3` are actually only running under coverage, and not under other versions of Python 3. Ideally they should be running on all of our `check-py3x` jobs.

Worse (except in the sense that we seem to have got away with it) is that as far as I can tell the py2 tests aren't running *anywhere*.
Run Python version specific tests as part of basic-test.sh
Fixes #1367.

NB the build for this should fail until #1366 is resolved. @Zac-HD if you want to just build your work on top of this branch we can close this without merging, but I figured it would be useful to see what the build for this is like independently.


Discussion:
Yup, 3.5 is still supported. Looks like the reason we didn't catch this is that a lot of these tests are in our `py3` folder which cleverly looks like we only run it as part of the coverage tests (which run on 3.6). 😢 

@Zac-HD this failure looks like it's in the new attrs inference work. Can you take a look? I'll take on sorting out the build side of things.
Yep, I'll take this one on (the feature is my puppy now...).

From a first glance this looks like a problem getting the return type annotation of a function.  Given the general state of static typing on 3.5, and our best-effort / latest only approach, I might just split it out into a separate test that can be skipped before Python 3.6
> Yep, I'll take this one on (the feature is my puppy now...).

I *may* have had the exact same framing in my head when I asked you to look into it, but decided not to voice it. 😉 
You might want to check out #863 at the same time, since it seems related.


Resolving patch:
diff --git a/tooling/src/hypothesistooling/__main__.py b/tooling/src/hypothesistooling/__main__.py
--- a/tooling/src/hypothesistooling/__main__.py
+++ b/tooling/src/hypothesistooling/__main__.py
@@ -352,7 +352,11 @@ def install_core():
     ALIASES[n] = 'python%s.%s' % (major, minor)
 
 
-python_tests = task(if_changed=(hp.PYTHON_SRC, hp.PYTHON_TESTS))
+python_tests = task(if_changed=(
+    hp.PYTHON_SRC,
+    hp.PYTHON_TESTS,
+    os.path.join(hp.HYPOTHESIS_PYTHON, 'scripts'),
+))
 
 
 @python_tests


Test patch:
diff --git a/hypothesis-python/tests/py3/test_annotations.py b/hypothesis-python/tests/py3/test_annotations.py
--- a/hypothesis-python/tests/py3/test_annotations.py
+++ b/hypothesis-python/tests/py3/test_annotations.py
@@ -17,6 +17,8 @@
 
 from __future__ import division, print_function, absolute_import
 
+import sys
+
 import attr
 import pytest
 
@@ -133,11 +135,15 @@ class Inferrables(object):
     annot_converter = attr.ib(converter=a_converter)
 
 
+@pytest.mark.skipif(sys.version_info[:2] <= (3, 5),
+                    reason="Too-old typing module can't get return value hint")
 @given(st.builds(Inferrables))
 def test_attrs_inference_builds(c):
     pass
 
 
+@pytest.mark.skipif(sys.version_info[:2] <= (3, 5),
+                    reason="Too-old typing module can't get return value hint")
 @given(st.from_type(Inferrables))
 def test_attrs_inference_from_type(c):
     pass