Repository: HypothesisWorks/hypothesis

Issue:
We're running the wrong pytest version in builds
See e.g. [here](https://travis-ci.org/HypothesisWorks/hypothesis-python/jobs/275856651#L477) where the pytest28 job is running 3.2.2

I think some of these might be versions of pytest we no longer actually support anyway, but we should figure out which are our supported versions, make sure we're running those correctly, and then drop the tasks for the rest.


Discussion:
Recent pytest release history:

* 2.8.0 – 18 Sep 2015
* 2.9.0 – 1 Mar 2016
* 3.0.0 – 19 Aug 2016
* 3.1.0 – 22 May 2017
* 3.2.0 – 1 Aug 2017

We dropped pytest 2.7 in #435, and the supported version in `setup.py` is `pytest>=2.8.0`.
> We dropped pytest 2.7 in #435, and the supported version in setup.py is pytest>=2.8.0.

I'd be perfectly happy to drop anything older than 3.0

I think we should probably clarify our supported versions policy in general. The [pandas extra](https://hypothesis.readthedocs.io/en/latest/numpy.html#supported-versions) is pretty explicit about this. I think something along the following lines might be reasonable:

* We only ever support the latest patch release of any minor release
* We always support the latest release
* Everything else is on a "best effort" basis and support for it may be dropped in any minor release.
* We probably don't support any releases that are more than a year old unless there is a good reason to
* Even with a very good reason to, we don't support anything two years old. They might work, but you're on your own if they don't.

(nb. Pandas 0.18.1 which we do currently support is in that between one and two year grey area)

Anyway, regardless of which versions of pytest we decide to support, we should definitely make sure we're actually testing them. :-)

Resolving patch:
diff --git a/hypothesis-python/setup.py b/hypothesis-python/setup.py
--- a/hypothesis-python/setup.py
+++ b/hypothesis-python/setup.py
@@ -58,7 +58,7 @@ def local_file(name):
     'dateutil': ['python-dateutil'],
     'fakefactory': ['Faker>=0.7'],
     'numpy': ['numpy>=1.9.0'],
-    'pytest': ['pytest>=2.8.0'],
+    'pytest': ['pytest>=3.0'],
     # We only support Django versions with upstream support - see
     # https://www.djangoproject.com/download/#supported-versions
     'django': ['pytz', 'django>=1.11'],
@@ -110,6 +110,7 @@ def local_file(name):
         'Programming Language :: Python :: 3.4',
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.7',
         'Programming Language :: Python :: Implementation :: CPython',
         'Programming Language :: Python :: Implementation :: PyPy',
         'Topic :: Software Development :: Testing',
diff --git a/tooling/src/hypothesistooling/__main__.py b/tooling/src/hypothesistooling/__main__.py
--- a/tooling/src/hypothesistooling/__main__.py
+++ b/tooling/src/hypothesistooling/__main__.py
@@ -405,7 +405,7 @@ def standard_tox_task(name):
 
 
 standard_tox_task('nose')
-standard_tox_task('pytest28')
+standard_tox_task('pytest30')
 standard_tox_task('faker070')
 standard_tox_task('faker-latest')
 standard_tox_task('django20')


Test patch:
diff --git a/hypothesis-python/tests/pytest/test_statistics.py b/hypothesis-python/tests/pytest/test_statistics.py
--- a/hypothesis-python/tests/pytest/test_statistics.py
+++ b/hypothesis-python/tests/pytest/test_statistics.py
@@ -64,7 +64,7 @@ def test_prints_statistics_given_option(testdir):
     assert 'Hypothesis Statistics' in out
     assert 'timeout=0.2' in out
     assert 'max_examples=100' in out
-    assert 'HypothesisDeprecationWarning' in out
+    assert '< 10% of examples satisfied assumptions' in out
 
 
 UNITTEST_TESTSUITE = """