Repository: numpy/numpy

Issue:
Newer versions of pytest are failing because of multiple hypothesis imports.
The CI tests fail for newer pytest on account of a pytest warning, see https://travis-ci.org/github/numpy/numpy/jobs/730868650 and https://github.com/pytest-dev/pytest/issues/7810. If I comment out hypothesis related code in `numpy/_pytesttester.py` the warning goes away. The other place hypothesis is imported is `numpy/configtest.py`. I seems we need to get rid of one of the imports.
<!-- Please describe the issue in detail here, and fill in the fields below -->

### Reproducing code example:

<!-- A short code example that reproduces the problem/missing feature. It should be
self-contained, i.e., possible to run as-is via 'python myproblem.py' -->

```python
import numpy as np
<< your code here >>
```

### Error message:

<!-- If you are reporting a segfault please include a GDB traceback, which you
can generate by following
https://github.com/numpy/numpy/blob/master/doc/source/dev/development_environment.rst#debugging -->

<!-- Full error message, if any (starting from line Traceback: ...) -->

### NumPy/Python version information:

<!-- Output from 'import sys, numpy; print(numpy.__version__, sys.version)' -->




Discussion:
@Zac-HD Thoughts?
The reason we have this configuration split over two places is, per #16879, that we want `numpy.test()` to be deterministic, but running the tests in development mode to be randomised and save failing examples.

Given this issue, the "right way to do that" is probably to move *all* the config into `conftest.py`, and then decide which settings profile to load based on whether or not we're running via `numpy.test()`.  I don't know how to detect that, but happy to write the PR to do everything else - and include that if someone can advise me.
I think moving all the config into one place is desirable.

"running the tests in development mode" [imports](https://github.com/numpy/numpy/blob/v1.19.2/runtests.py#L283) and [runs](https://github.com/numpy/numpy/blob/v1.19.2/runtests.py#L309) `numpy.test()`. The only way I can think of to do what you ask is to add an extra argument to the call to set the mode: either deterministic or development. I am not so good with names: whether to have `deterministc` flag that defaults to True and is set to False in `runtests.py` or a `development` flag that defaults to False and is set True in `runtests.py`
IIRC, we now detect "development" by the presence of `pytest.ini`.
@Zac-HD any progress on that PR?
Apologies for the delay; it's been bumped to the top of my OSS todo list and I expect to have something up over the weekend :slightly_smiling_face: 

Resolving patch:
diff --git a/numpy/_pytesttester.py b/numpy/_pytesttester.py
--- a/numpy/_pytesttester.py
+++ b/numpy/_pytesttester.py
@@ -140,9 +140,6 @@ def __call__(self, label='fast', verbose=1, extra_argv=None,
         import pytest
         import warnings
 
-        # Imported after pytest to enable assertion rewriting
-        import hypothesis
-
         module = sys.modules[self.module_name]
         module_path = os.path.abspath(module.__path__[0])
 
@@ -205,15 +202,6 @@ def __call__(self, label='fast', verbose=1, extra_argv=None,
 
         pytest_args += ["--pyargs"] + list(tests)
 
-        # This configuration is picked up by numpy.conftest, and ensures that
-        # running `np.test()` is deterministic and does not write any files.
-        # See https://hypothesis.readthedocs.io/en/latest/settings.html
-        hypothesis.settings.register_profile(
-            name="np.test() profile",
-            deadline=None, print_blob=True, database=None, derandomize=True,
-            suppress_health_check=hypothesis.HealthCheck.all(),
-        )
-
         # run tests.
         _show_numpy_info()
 
diff --git a/numpy/conftest.py b/numpy/conftest.py
--- a/numpy/conftest.py
+++ b/numpy/conftest.py
@@ -19,18 +19,26 @@
 hypothesis.configuration.set_hypothesis_home_dir(
     os.path.join(tempfile.gettempdir(), ".hypothesis")
 )
-# See https://hypothesis.readthedocs.io/en/latest/settings.html
+
+# We register two custom profiles for Numpy - for details see
+# https://hypothesis.readthedocs.io/en/latest/settings.html
+# The first is designed for our own CI runs; the latter also 
+# forces determinism and is designed for use via np.test()
 hypothesis.settings.register_profile(
     name="numpy-profile", deadline=None, print_blob=True,
 )
-# We try loading the profile defined by np.test(), which disables the
-# database and forces determinism, and fall back to the profile defined
-# above if we're running pytest directly.  The odd dance is required
-# because np.test() executes this file *after* its own setup code.
-try:
-    hypothesis.settings.load_profile("np.test() profile")
-except hypothesis.errors.InvalidArgument:  # profile not found
-    hypothesis.settings.load_profile("numpy-profile")
+hypothesis.settings.register_profile(
+    name="np.test() profile",
+    deadline=None, print_blob=True, database=None, derandomize=True,
+    suppress_health_check=hypothesis.HealthCheck.all(),
+)
+# Note that the default profile is chosen based on the presence 
+# of pytest.ini, but can be overriden by passing the 
+# --hypothesis-profile=NAME argument to pytest.
+_pytest_ini = os.path.join(os.path.dirname(__file__), "..", "pytest.ini")
+hypothesis.settings.load_profile(
+    "numpy-profile" if os.path.isfile(_pytest_ini) else "np.test() profile"
+)
 
 
 def pytest_configure(config):


Test patch: