Repository: HypothesisWorks/hypothesis

Issue:
Our pytest plugin loads all Hypothesis entry points, which may have undesirable side effects
If I have Hypothesis installed and run pytest, it always creates `.hypothesis/unicode_data/13.0.0/charmap.json.gz`,  whether the project uses Hypothesis or not.

This then shows up as an untracked file, because the non-Hypotheis project has no reason to put `.hypothesis` in `.gitignore`.

Would it be feasible to avoid creating the file, or creating it in a non-local directory?

# Alternative

If this is non-trivial, I can easily enough put `.hypothesis` in a global `.gitignore` file.


# Minimal demo

Python 3.10
pytest 
hypothesis 6.24.2

Running pytest in an empty directory without hypothesis installed:

```console
$ mkdir /tmp/123
$ cd /tmp/123
$ pytest
======================================================= test session starts ========================================================
platform darwin -- Python 3.10.0, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /private/tmp/123
plugins: respx-0.18.2, xdist-2.4.0, anyio-3.3.4, timeout-1.4.2, forked-1.3.0, lazy-fixture-0.6.3, cov-3.0.0
collected 0 items

====================================================== no tests ran in 0.02s =======================================================
$ l
total 0
drwxr-xr-x   3 hugo  wheel    96B  8 Nov 10:53 .
drwxrwxrwt  43 root  wheel   1.3K  8 Nov 10:53 ..
drwxr-xr-x   6 hugo  wheel   192B  8 Nov 10:53 .pytest_cache
```

Installing hypothesis and running pytest:

```console
$ python -m pip install hypothesis
Collecting hypothesis
  Using cached hypothesis-6.24.2-py3-none-any.whl (381 kB)
Requirement already satisfied: sortedcontainers<3.0.0,>=2.1.0 in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages (from hypothesis) (2.4.0)
Requirement already satisfied: attrs>=19.2.0 in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages (from hypothesis) (21.2.0)
Installing collected packages: hypothesis
Successfully installed hypothesis-6.24.2
$ l
total 0
drwxr-xr-x   3 hugo  wheel    96B  8 Nov 10:53 .
drwxrwxrwt  43 root  wheel   1.3K  8 Nov 10:55 ..
drwxr-xr-x   6 hugo  wheel   192B  8 Nov 10:53 .pytest_cache
$ pytest
======================================================= test session starts ========================================================
platform darwin -- Python 3.10.0, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /private/tmp/123
plugins: respx-0.18.2, hypothesis-6.24.2, xdist-2.4.0, anyio-3.3.4, timeout-1.4.2, forked-1.3.0, lazy-fixture-0.6.3, cov-3.0.0
collected 0 items

====================================================== no tests ran in 0.03s =======================================================
$ l
total 0
drwxr-xr-x   4 hugo  wheel   128B  8 Nov 10:55 .
drwxrwxrwt  43 root  wheel   1.3K  8 Nov 10:55 ..
drwxr-xr-x   3 hugo  wheel    96B  8 Nov 10:55 .hypothesis
drwxr-xr-x   6 hugo  wheel   192B  8 Nov 10:53 .pytest_cache
$ l .hypothesis/unicode_data/13.0.0/
total 48
drwxr-xr-x  3 hugo  wheel    96B  8 Nov 10:55 .
drwxr-xr-x  3 hugo  wheel    96B  8 Nov 10:55 ..
-rw-------  1 hugo  wheel    20K  8 Nov 10:55 charmap.json.gz
```

# Bisecting

Bisecting releases, this didn't happen with 5.48.0 and started in 5.49.0.

* https://github.com/HypothesisWorks/hypothesis/compare/hypothesis-python-5.48.0...hypothesis-python-5.49.0

Bisecting commits, this started in 763e545d7958d8d4ab4d55b71cf3f2c7f6e2ea07 from PR https://github.com/HypothesisWorks/hypothesis/pull/2738.

(Due to the frequency of Hypothesis releases, it was really easy to bisect this!)

Thank you!
Calling st.register_type_strategy imports numpy
This is the "offending" block:

https://github.com/HypothesisWorks/hypothesis/blob/65b4b2102b94463af95a94a9994e77cfaf12f275/hypothesis-python/src/hypothesis/strategies/_internal/types.py#L493-L501

This test

https://github.com/HypothesisWorks/hypothesis/blob/65b4b2102b94463af95a94a9994e77cfaf12f275/hypothesis-python/tests/numpy/test_import.py#L16-L29

can be made to fail by modifying the src-string to be:

```
SHOULD_NOT_IMPORT_NUMPY = """
import sys
from hypothesis import given, strategies as st

@given(st.integers() | st.floats() | st.sampled_from(["a", "b"]) | st.just(st.register_type_strategy(int, st.integers())))
def test_no_numpy_import(x):
    assert "numpy" not in sys.modules
"""
```

This was causing a headache for me when figuring out why numpy was being imported the instant I ran pytest... this was because I happened to have installed numpy + hypothesis + pydantic (which uses `st.register_type_strategy` via our entrypoint).

This could be a big headache for testers over at, say, PyTorch, who want to test that they don't cause numpy to be imported.

Improve the error message for `HealthCheck.function_scoped_fixture`
*Seeing #3017 reminded me that I had previously done some partial work on improving the error message for `HealthCheck.function_scoped_fixture`, but never got around to actually finishing it. So here's at least an initial write-up.*

---

On a whim I searched GitHub and Google for “HealthCheck.function_scoped_fixture”, and found a number of cases where users did not seem to end up with a good understanding of how to deal with the problem.

For example, I saw:
- People jumping through hoops to get rid of a function-scoped fixture, when the fixture in question was totally fine to reuse and they should have just suppressed the check.
- People rightly identifying that they should just suppress the check in their particular case, but then feeling vaguely bad about it.

I think our current error message has a pitfalls:
- It assumes that the user wrote the fixture and is free to modify it; in practice it's also common to use existing fixtures written by someone else (e.g. built-in pytest fixtures).
- It assumes that using function-scoped fixtures with Hypothesis is almost certainly bad; in practice there seems to be a large subset of function-scoped fixtures that are already safely reusable across examples, typically because they set up some read-only resource. In these cases, we need to be do a better job of reassuring users that suppressing the error is sometimes totally fine.
- It assumes that widening the scope of the fixture is a reasonable solution; in practice there seem to be a lot of cases where the fixture is both reusable *and* lightweight, so widening the scope isn't inherently better than just suppressing the health check.


Discussion:
Thanks for the report!  The weird thing here is that we've *already* designed Hypothesis to avoid creating the `.hypothesis` directory until we're actually using it, and try as I might I haven't been able to reproduce this - I even wrote a test in #3141 without success.

I *think* this could happen because you have some package (e.g. Pydantic) with a [Hypothesis entry point](https://hypothesis.readthedocs.io/en/latest/strategies.html#registering-strategies-via-setuptools-entry-points) installed (e.g. Pydantic), and then when `pytest` loads our pytest-plugin we in turn load that hypothesis-plugin and that's what creates the charmap file.  If so, we'll want to defer loading Hypothesis entry points, so that they're triggered on user interaction rather than just running pytest.

The only problem is that this doesn't fit at all with your bisection!  I am seriously confused.
Aha, it is pydantic! Uninstalling pydantic prevents it from happening, re-installing pydantic reproduces it.

(Before this, I uninstalled all `pytest-*` and those listed by `pytest -VV`.)
Looks like we might need to refactor our pytest plugin for lazy-loading of Hypothesis then... it'll be a little annoying, but could be worse!
If I expected this to require a nasty patch - i.e. worse than our numpy special case in `register_random` - I wouldn't consider it worth fixing.

That's because once we make our pytest plugin lazily load Hypothesis (#3140), this will only trigger if you deliberately `import hypothesis` *and* somehow call `st.register_type_strategy()` *in the test that Numpy is not imported*, which usually happens in a fresh process.

Other libraries will never get this special treatment, but Numpy is already exceptional in our code and the ecosystem.
Current error message for reference:

https://github.com/HypothesisWorks/hypothesis/blob/1023a9b6b1e78387523273c61628e340889b8878/hypothesis-python/src/hypothesis/extra/pytestplugin.py#L177-L188
This is what one of my draft improvements ended up looking like:

```python
msg = """Function-scoped fixture {1!r} used by {0}

Function-scoped fixtures are not reset between examples generated by
`@given(...)`, which is often surprising and can cause subtle test bugs.

If you were expecting the fixture to run separately for each generated example,
then unfortunately you will need to find a different way to achieve your goal
(e.g. using a similar context manager instead of a fixture).

If you are confident that your test will work correctly even though the
fixture is not reset between generated examples, you can suppress this health
check to assure Hypothesis that you understand what you are doing.
"""
```

I'm not entirely happy with it, but I like the general direction of where I was going. The three sentences highlight three key pieces of information:
1. Short explanation of the problem
2. Advice for people who are surprised by the error
3. Reassurance that sometimes the error is bogus and it's OK to suppress it in those cases

I also like the idea of inserting hard line breaks, because relying on one auto-wrapped line ends up making things harder to read in some contexts.
An alternate take on the final paragraph that tries to be a bit more practical about when to suppress the check:

```
If you know that the fixture is safe to reuse across multiple generated
examples, or your test manually resets the fixture state, you can suppress
this health check to assure Hypothesis that you understand what you are doing.
```

Resolving patch:
diff --git a/hypothesis-python/setup.py b/hypothesis-python/setup.py
--- a/hypothesis-python/setup.py
+++ b/hypothesis-python/setup.py
@@ -129,8 +129,9 @@ def local_file(name):
         "Topic :: Software Development :: Testing",
         "Typing :: Typed",
     ],
+    py_modules=["_hypothesis_pytestplugin"],
     entry_points={
-        "pytest11": ["hypothesispytest = hypothesis.extra.pytestplugin"],
+        "pytest11": ["hypothesispytest = _hypothesis_pytestplugin"],
         "console_scripts": ["hypothesis = hypothesis.extra.cli:main"],
     },
     long_description=open(README).read(),
diff --git a/hypothesis-python/src/_hypothesis_pytestplugin.py b/hypothesis-python/src/_hypothesis_pytestplugin.py
new file mode 100644
--- /dev/null
+++ b/hypothesis-python/src/_hypothesis_pytestplugin.py
@@ -0,0 +1,335 @@
+# This file is part of Hypothesis, which may be found at
+# https://github.com/HypothesisWorks/hypothesis/
+#
+# Most of this work is copyright (C) 2013-2021 David R. MacIver
+# (david@drmaciver.com), but it contains contributions by others. See
+# CONTRIBUTING.rst for a full list of people who may hold copyright, and
+# consult the git log if you need to determine who owns an individual
+# contribution.
+#
+# This Source Code Form is subject to the terms of the Mozilla Public License,
+# v. 2.0. If a copy of the MPL was not distributed with this file, You can
+# obtain one at https://mozilla.org/MPL/2.0/.
+#
+# END HEADER
+
+"""
+The pytest plugin for Hypothesis.
+
+We move this from the old location at `hypothesis.extra.pytestplugin` so that it
+can be loaded by Pytest without importing Hypothesis.  In turn, this means that
+Hypothesis will not load our own third-party plugins (with associated side-effects)
+unless and until the user explicitly runs `import hypothesis`.
+
+See https://github.com/HypothesisWorks/hypothesis/issues/3140 for details.
+"""
+
+import base64
+import sys
+from inspect import signature
+
+import pytest
+
+LOAD_PROFILE_OPTION = "--hypothesis-profile"
+VERBOSITY_OPTION = "--hypothesis-verbosity"
+PRINT_STATISTICS_OPTION = "--hypothesis-show-statistics"
+SEED_OPTION = "--hypothesis-seed"
+EXPLAIN_OPTION = "--hypothesis-explain"
+
+_VERBOSITY_NAMES = ["quiet", "normal", "verbose", "debug"]
+_ALL_OPTIONS = [
+    LOAD_PROFILE_OPTION,
+    VERBOSITY_OPTION,
+    PRINT_STATISTICS_OPTION,
+    SEED_OPTION,
+    EXPLAIN_OPTION,
+]
+_FIXTURE_MSG = """Function-scoped fixture {0!r} used by {1!r}
+
+Function-scoped fixtures are not reset between examples generated by
+`@given(...)`, which is often surprising and can cause subtle test bugs.
+
+If you were expecting the fixture to run separately for each generated example,
+then unfortunately you will need to find a different way to achieve your goal
+(e.g. using a similar context manager instead of a fixture).
+
+If you are confident that your test will work correctly even though the
+fixture is not reset between generated examples, you can suppress this health
+check to assure Hypothesis that you understand what you are doing.
+"""
+
+
+class StoringReporter:
+    def __init__(self, config):
+        assert "hypothesis" in sys.modules
+        from hypothesis.reporting import default
+
+        self.report = default
+        self.config = config
+        self.results = []
+
+    def __call__(self, msg):
+        if self.config.getoption("capture", "fd") == "no":
+            self.report(msg)
+        if not isinstance(msg, str):
+            msg = repr(msg)
+        self.results.append(msg)
+
+
+# Avoiding distutils.version.LooseVersion due to
+# https://github.com/HypothesisWorks/hypothesis/issues/2490
+if tuple(map(int, pytest.__version__.split(".")[:2])) < (4, 6):  # pragma: no cover
+    import warnings
+
+    PYTEST_TOO_OLD_MESSAGE = """
+        You are using pytest version %s. Hypothesis tests work with any test
+        runner, but our pytest plugin requires pytest 4.6 or newer.
+        Note that the pytest developers no longer support your version either!
+        Disabling the Hypothesis pytest plugin...
+    """
+    warnings.warn(PYTEST_TOO_OLD_MESSAGE % (pytest.__version__,))
+
+else:
+
+    def pytest_addoption(parser):
+        group = parser.getgroup("hypothesis", "Hypothesis")
+        group.addoption(
+            LOAD_PROFILE_OPTION,
+            action="store",
+            help="Load in a registered hypothesis.settings profile",
+        )
+        group.addoption(
+            VERBOSITY_OPTION,
+            action="store",
+            choices=_VERBOSITY_NAMES,
+            help="Override profile with verbosity setting specified",
+        )
+        group.addoption(
+            PRINT_STATISTICS_OPTION,
+            action="store_true",
+            help="Configure when statistics are printed",
+            default=False,
+        )
+        group.addoption(
+            SEED_OPTION,
+            action="store",
+            help="Set a seed to use for all Hypothesis tests",
+        )
+        group.addoption(
+            EXPLAIN_OPTION,
+            action="store_true",
+            help="Enable the `explain` phase for failing Hypothesis tests",
+            default=False,
+        )
+
+    def _any_hypothesis_option(config):
+        return bool(any(config.getoption(opt) for opt in _ALL_OPTIONS))
+
+    def pytest_report_header(config):
+        if not (
+            config.option.verbose >= 1
+            or "hypothesis" in sys.modules
+            or _any_hypothesis_option(config)
+        ):
+            return None
+
+        from hypothesis import Verbosity, settings
+
+        if config.option.verbose < 1 and settings.default.verbosity < Verbosity.verbose:
+            return None
+        settings_str = settings.default.show_changed()
+        if settings_str != "":
+            settings_str = f" -> {settings_str}"
+        return f"hypothesis profile {settings._current_profile!r}{settings_str}"
+
+    def pytest_configure(config):
+        config.addinivalue_line("markers", "hypothesis: Tests which use hypothesis.")
+        if not _any_hypothesis_option(config):
+            return
+        from hypothesis import Phase, Verbosity, core, settings
+
+        profile = config.getoption(LOAD_PROFILE_OPTION)
+        if profile:
+            settings.load_profile(profile)
+        verbosity_name = config.getoption(VERBOSITY_OPTION)
+        if verbosity_name and verbosity_name != settings.default.verbosity.name:
+            verbosity_value = Verbosity[verbosity_name]
+            name = f"{settings._current_profile}-with-{verbosity_name}-verbosity"
+            # register_profile creates a new profile, exactly like the current one,
+            # with the extra values given (in this case 'verbosity')
+            settings.register_profile(name, verbosity=verbosity_value)
+            settings.load_profile(name)
+        if (
+            config.getoption(EXPLAIN_OPTION)
+            and Phase.explain not in settings.default.phases
+        ):
+            name = f"{settings._current_profile}-with-explain-phase"
+            phases = settings.default.phases + (Phase.explain,)
+            settings.register_profile(name, phases=phases)
+            settings.load_profile(name)
+
+        seed = config.getoption(SEED_OPTION)
+        if seed is not None:
+            try:
+                seed = int(seed)
+            except ValueError:
+                pass
+            core.global_force_seed = seed
+
+    @pytest.hookimpl(hookwrapper=True)
+    def pytest_runtest_call(item):
+        if not (hasattr(item, "obj") and "hypothesis" in sys.modules):
+            yield
+            return
+
+        from hypothesis import core
+        from hypothesis.internal.detection import is_hypothesis_test
+
+        core.running_under_pytest = True
+
+        if not is_hypothesis_test(item.obj):
+            # If @given was not applied, check whether other hypothesis
+            # decorators were applied, and raise an error if they were.
+            if getattr(item.obj, "is_hypothesis_strategy_function", False):
+                from hypothesis.errors import InvalidArgument
+
+                raise InvalidArgument(
+                    f"{item.nodeid} is a function that returns a Hypothesis strategy, "
+                    "but pytest has collected it as a test function.  This is useless "
+                    "as the function body will never be executed.  To define a test "
+                    "function, use @given instead of @composite."
+                )
+            message = "Using `@%s` on a test without `@given` is completely pointless."
+            for name, attribute in [
+                ("example", "hypothesis_explicit_examples"),
+                ("seed", "_hypothesis_internal_use_seed"),
+                ("settings", "_hypothesis_internal_settings_applied"),
+                ("reproduce_example", "_hypothesis_internal_use_reproduce_failure"),
+            ]:
+                if hasattr(item.obj, attribute):
+                    from hypothesis.errors import InvalidArgument
+
+                    raise InvalidArgument(message % (name,))
+            yield
+        else:
+            from hypothesis import HealthCheck, settings
+            from hypothesis.internal.escalation import current_pytest_item
+            from hypothesis.internal.healthcheck import fail_health_check
+            from hypothesis.reporting import with_reporter
+            from hypothesis.statistics import collector, describe_statistics
+
+            # Retrieve the settings for this test from the test object, which
+            # is normally a Hypothesis wrapped_test wrapper. If this doesn't
+            # work, the test object is probably something weird
+            # (e.g a stateful test wrapper), so we skip the function-scoped
+            # fixture check.
+            settings = getattr(item.obj, "_hypothesis_internal_use_settings", None)
+
+            # Check for suspicious use of function-scoped fixtures, but only
+            # if the corresponding health check is not suppressed.
+            if (
+                settings is not None
+                and HealthCheck.function_scoped_fixture
+                not in settings.suppress_health_check
+            ):
+                # Warn about function-scoped fixtures, excluding autouse fixtures because
+                # the advice is probably not actionable and the status quo seems OK...
+                # See https://github.com/HypothesisWorks/hypothesis/issues/377 for detail.
+                argnames = None
+                for fx_defs in item._request._fixturemanager.getfixtureinfo(
+                    node=item, func=item.function, cls=None
+                ).name2fixturedefs.values():
+                    if argnames is None:
+                        argnames = frozenset(signature(item.function).parameters)
+                    for fx in fx_defs:
+                        if fx.argname in argnames:
+                            active_fx = item._request._get_active_fixturedef(fx.argname)
+                            if active_fx.scope == "function":
+                                fail_health_check(
+                                    settings,
+                                    _FIXTURE_MSG.format(fx.argname, item.nodeid),
+                                    HealthCheck.function_scoped_fixture,
+                                )
+
+            if item.get_closest_marker("parametrize") is not None:
+                # Give every parametrized test invocation a unique database key
+                key = item.nodeid.encode()
+                item.obj.hypothesis.inner_test._hypothesis_internal_add_digest = key
+
+            store = StoringReporter(item.config)
+
+            def note_statistics(stats):
+                stats["nodeid"] = item.nodeid
+                item.hypothesis_statistics = base64.b64encode(
+                    describe_statistics(stats).encode()
+                ).decode()
+
+            with collector.with_value(note_statistics):
+                with with_reporter(store):
+                    with current_pytest_item.with_value(item):
+                        yield
+            if store.results:
+                item.hypothesis_report_information = list(store.results)
+
+    @pytest.hookimpl(hookwrapper=True)
+    def pytest_runtest_makereport(item, call):
+        report = (yield).get_result()
+        if hasattr(item, "hypothesis_report_information"):
+            report.sections.append(
+                ("Hypothesis", "\n".join(item.hypothesis_report_information))
+            )
+        if hasattr(item, "hypothesis_statistics") and report.when == "teardown":
+            name = "hypothesis-statistics-" + item.nodeid
+            try:
+                item.config._xml.add_global_property(name, item.hypothesis_statistics)
+            except AttributeError:
+                # --junitxml not passed, or Pytest 4.5 (before add_global_property)
+                # We'll fail xunit2 xml schema checks, upgrade pytest if you care.
+                report.user_properties.append((name, item.hypothesis_statistics))
+            # If there's an HTML report, include our summary stats for each test
+            stats = base64.b64decode(item.hypothesis_statistics.encode()).decode()
+            pytest_html = item.config.pluginmanager.getplugin("html")
+            if pytest_html is not None:  # pragma: no cover
+                report.extra = getattr(report, "extra", []) + [
+                    pytest_html.extras.text(stats, name="Hypothesis stats")
+                ]
+
+    def pytest_terminal_summary(terminalreporter):
+        if not terminalreporter.config.getoption(PRINT_STATISTICS_OPTION):
+            return
+        terminalreporter.section("Hypothesis Statistics")
+
+        def report(properties):
+            for name, value in properties:
+                if name.startswith("hypothesis-statistics-"):
+                    if hasattr(value, "uniobj"):
+                        # Under old versions of pytest, `value` was a `py.xml.raw`
+                        # rather than a string, so we get the (unicode) string off it.
+                        value = value.uniobj
+                    line = base64.b64decode(value.encode()).decode() + "\n\n"
+                    terminalreporter.write_line(line)
+
+        try:
+            global_properties = terminalreporter.config._xml.global_properties
+        except AttributeError:
+            # terminalreporter.stats is a dict, where the empty string appears to
+            # always be the key for a list of _pytest.reports.TestReport objects
+            for test_report in terminalreporter.stats.get("", []):
+                if test_report.when == "teardown":
+                    report(test_report.user_properties)
+        else:
+            report(global_properties)
+
+    def pytest_collection_modifyitems(items):
+        if "hypothesis" not in sys.modules:
+            return
+
+        from hypothesis.internal.detection import is_hypothesis_test
+
+        for item in items:
+            if isinstance(item, pytest.Function) and is_hypothesis_test(item.obj):
+                item.add_marker("hypothesis")
+
+
+def load():
+    """Required for `pluggy` to load a plugin from setuptools entrypoints."""
diff --git a/hypothesis-python/src/hypothesis/extra/pytestplugin.py b/hypothesis-python/src/hypothesis/extra/pytestplugin.py
--- a/hypothesis-python/src/hypothesis/extra/pytestplugin.py
+++ b/hypothesis-python/src/hypothesis/extra/pytestplugin.py
@@ -13,270 +13,12 @@
 #
 # END HEADER
 
-import base64
-from inspect import signature
+"""
+Stub for users who manually load our pytest plugin.
 
-import pytest
+The plugin implementation is now located in a top-level module outside the main
+hypothesis tree, so that Pytest can load the plugin without thereby triggering
+the import of Hypothesis itself (and thus loading our own plugins).
+"""
 
-from hypothesis import HealthCheck, Phase, Verbosity, core, settings
-from hypothesis.errors import InvalidArgument
-from hypothesis.internal.detection import is_hypothesis_test
-from hypothesis.internal.escalation import current_pytest_item
-from hypothesis.internal.healthcheck import fail_health_check
-from hypothesis.reporting import default as default_reporter, with_reporter
-from hypothesis.statistics import collector, describe_statistics
-
-LOAD_PROFILE_OPTION = "--hypothesis-profile"
-VERBOSITY_OPTION = "--hypothesis-verbosity"
-PRINT_STATISTICS_OPTION = "--hypothesis-show-statistics"
-SEED_OPTION = "--hypothesis-seed"
-EXPLAIN_OPTION = "--hypothesis-explain"
-
-
-class StoringReporter:
-    def __init__(self, config):
-        self.config = config
-        self.results = []
-
-    def __call__(self, msg):
-        if self.config.getoption("capture", "fd") == "no":
-            default_reporter(msg)
-        if not isinstance(msg, str):
-            msg = repr(msg)
-        self.results.append(msg)
-
-
-# Avoiding distutils.version.LooseVersion due to
-# https://github.com/HypothesisWorks/hypothesis/issues/2490
-if tuple(map(int, pytest.__version__.split(".")[:2])) < (4, 6):  # pragma: no cover
-    import warnings
-
-    from hypothesis.errors import HypothesisWarning
-
-    PYTEST_TOO_OLD_MESSAGE = """
-        You are using pytest version %s. Hypothesis tests work with any test
-        runner, but our pytest plugin requires pytest 4.6 or newer.
-        Note that the pytest developers no longer support your version either!
-        Disabling the Hypothesis pytest plugin...
-    """
-    warnings.warn(PYTEST_TOO_OLD_MESSAGE % (pytest.__version__,), HypothesisWarning)
-
-else:
-
-    def pytest_addoption(parser):
-        group = parser.getgroup("hypothesis", "Hypothesis")
-        group.addoption(
-            LOAD_PROFILE_OPTION,
-            action="store",
-            help="Load in a registered hypothesis.settings profile",
-        )
-        group.addoption(
-            VERBOSITY_OPTION,
-            action="store",
-            choices=[opt.name for opt in Verbosity],
-            help="Override profile with verbosity setting specified",
-        )
-        group.addoption(
-            PRINT_STATISTICS_OPTION,
-            action="store_true",
-            help="Configure when statistics are printed",
-            default=False,
-        )
-        group.addoption(
-            SEED_OPTION,
-            action="store",
-            help="Set a seed to use for all Hypothesis tests",
-        )
-        group.addoption(
-            EXPLAIN_OPTION,
-            action="store_true",
-            help="Enable the `explain` phase for failing Hypothesis tests",
-            default=False,
-        )
-
-    def pytest_report_header(config):
-        if config.option.verbose < 1 and settings.default.verbosity < Verbosity.verbose:
-            return None
-        profile = config.getoption(LOAD_PROFILE_OPTION)
-        if not profile:
-            profile = settings._current_profile
-        settings_str = settings.get_profile(profile).show_changed()
-        if settings_str != "":
-            settings_str = f" -> {settings_str}"
-        return f"hypothesis profile {profile!r}{settings_str}"
-
-    def pytest_configure(config):
-        core.running_under_pytest = True
-        profile = config.getoption(LOAD_PROFILE_OPTION)
-        if profile:
-            settings.load_profile(profile)
-        verbosity_name = config.getoption(VERBOSITY_OPTION)
-        if verbosity_name and verbosity_name != settings.default.verbosity.name:
-            verbosity_value = Verbosity[verbosity_name]
-            name = f"{settings._current_profile}-with-{verbosity_name}-verbosity"
-            # register_profile creates a new profile, exactly like the current one,
-            # with the extra values given (in this case 'verbosity')
-            settings.register_profile(name, verbosity=verbosity_value)
-            settings.load_profile(name)
-        if (
-            config.getoption(EXPLAIN_OPTION)
-            and Phase.explain not in settings.default.phases
-        ):
-            name = f"{settings._current_profile}-with-explain-phase"
-            phases = settings.default.phases + (Phase.explain,)
-            settings.register_profile(name, phases=phases)
-            settings.load_profile(name)
-
-        seed = config.getoption(SEED_OPTION)
-        if seed is not None:
-            try:
-                seed = int(seed)
-            except ValueError:
-                pass
-            core.global_force_seed = seed
-        config.addinivalue_line("markers", "hypothesis: Tests which use hypothesis.")
-
-    @pytest.hookimpl(hookwrapper=True)
-    def pytest_runtest_call(item):
-        if not hasattr(item, "obj"):
-            yield
-        elif not is_hypothesis_test(item.obj):
-            # If @given was not applied, check whether other hypothesis
-            # decorators were applied, and raise an error if they were.
-            if getattr(item.obj, "is_hypothesis_strategy_function", False):
-                raise InvalidArgument(
-                    f"{item.nodeid} is a function that returns a Hypothesis strategy, "
-                    "but pytest has collected it as a test function.  This is useless "
-                    "as the function body will never be executed.  To define a test "
-                    "function, use @given instead of @composite."
-                )
-            message = "Using `@%s` on a test without `@given` is completely pointless."
-            for name, attribute in [
-                ("example", "hypothesis_explicit_examples"),
-                ("seed", "_hypothesis_internal_use_seed"),
-                ("settings", "_hypothesis_internal_settings_applied"),
-                ("reproduce_example", "_hypothesis_internal_use_reproduce_failure"),
-            ]:
-                if hasattr(item.obj, attribute):
-                    raise InvalidArgument(message % (name,))
-            yield
-        else:
-            # Retrieve the settings for this test from the test object, which
-            # is normally a Hypothesis wrapped_test wrapper. If this doesn't
-            # work, the test object is probably something weird
-            # (e.g a stateful test wrapper), so we skip the function-scoped
-            # fixture check.
-            settings = getattr(item.obj, "_hypothesis_internal_use_settings", None)
-
-            # Check for suspicious use of function-scoped fixtures, but only
-            # if the corresponding health check is not suppressed.
-            if (
-                settings is not None
-                and HealthCheck.function_scoped_fixture
-                not in settings.suppress_health_check
-            ):
-                # Warn about function-scoped fixtures, excluding autouse fixtures because
-                # the advice is probably not actionable and the status quo seems OK...
-                # See https://github.com/HypothesisWorks/hypothesis/issues/377 for detail.
-                msg = (
-                    "%s uses the %r fixture, which is reset between function calls but not "
-                    "between test cases generated by `@given(...)`.  You can change it to "
-                    "a module- or session-scoped fixture if it is safe to reuse; if not "
-                    "we recommend using a context manager inside your test function.  See "
-                    "https://docs.pytest.org/en/latest/how-to/fixtures.html"
-                    "#scope-sharing-fixtures-across-classes-modules-packages-or-session "
-                    "for details on fixture scope."
-                )
-                argnames = None
-                for fx_defs in item._request._fixturemanager.getfixtureinfo(
-                    node=item, func=item.function, cls=None
-                ).name2fixturedefs.values():
-                    if argnames is None:
-                        argnames = frozenset(signature(item.function).parameters)
-                    for fx in fx_defs:
-                        if fx.argname in argnames:
-                            active_fx = item._request._get_active_fixturedef(fx.argname)
-                            if active_fx.scope == "function":
-                                fail_health_check(
-                                    settings,
-                                    msg % (item.nodeid, fx.argname),
-                                    HealthCheck.function_scoped_fixture,
-                                )
-
-            if item.get_closest_marker("parametrize") is not None:
-                # Give every parametrized test invocation a unique database key
-                key = item.nodeid.encode()
-                item.obj.hypothesis.inner_test._hypothesis_internal_add_digest = key
-
-            store = StoringReporter(item.config)
-
-            def note_statistics(stats):
-                stats["nodeid"] = item.nodeid
-                item.hypothesis_statistics = base64.b64encode(
-                    describe_statistics(stats).encode()
-                ).decode()
-
-            with collector.with_value(note_statistics):
-                with with_reporter(store):
-                    with current_pytest_item.with_value(item):
-                        yield
-            if store.results:
-                item.hypothesis_report_information = list(store.results)
-
-    @pytest.hookimpl(hookwrapper=True)
-    def pytest_runtest_makereport(item, call):
-        report = (yield).get_result()
-        if hasattr(item, "hypothesis_report_information"):
-            report.sections.append(
-                ("Hypothesis", "\n".join(item.hypothesis_report_information))
-            )
-        if hasattr(item, "hypothesis_statistics") and report.when == "teardown":
-            name = "hypothesis-statistics-" + item.nodeid
-            try:
-                item.config._xml.add_global_property(name, item.hypothesis_statistics)
-            except AttributeError:
-                # --junitxml not passed, or Pytest 4.5 (before add_global_property)
-                # We'll fail xunit2 xml schema checks, upgrade pytest if you care.
-                report.user_properties.append((name, item.hypothesis_statistics))
-            # If there's an HTML report, include our summary stats for each test
-            stats = base64.b64decode(item.hypothesis_statistics.encode()).decode()
-            pytest_html = item.config.pluginmanager.getplugin("html")
-            if pytest_html is not None:  # pragma: no cover
-                report.extra = getattr(report, "extra", []) + [
-                    pytest_html.extras.text(stats, name="Hypothesis stats")
-                ]
-
-    def pytest_terminal_summary(terminalreporter):
-        if not terminalreporter.config.getoption(PRINT_STATISTICS_OPTION):
-            return
-        terminalreporter.section("Hypothesis Statistics")
-
-        def report(properties):
-            for name, value in properties:
-                if name.startswith("hypothesis-statistics-"):
-                    if hasattr(value, "uniobj"):
-                        # Under old versions of pytest, `value` was a `py.xml.raw`
-                        # rather than a string, so we get the (unicode) string off it.
-                        value = value.uniobj
-                    line = base64.b64decode(value.encode()).decode() + "\n\n"
-                    terminalreporter.write_line(line)
-
-        try:
-            global_properties = terminalreporter.config._xml.global_properties
-        except AttributeError:
-            # terminalreporter.stats is a dict, where the empty string appears to
-            # always be the key for a list of _pytest.reports.TestReport objects
-            for test_report in terminalreporter.stats.get("", []):
-                if test_report.when == "teardown":
-                    report(test_report.user_properties)
-        else:
-            report(global_properties)
-
-    def pytest_collection_modifyitems(items):
-        for item in items:
-            if isinstance(item, pytest.Function) and is_hypothesis_test(item.obj):
-                item.add_marker("hypothesis")
-
-
-def load():
-    """Required for `pluggy` to load a plugin from setuptools entrypoints."""
+from _hypothesis_pytestplugin import *  # noqa


Test patch:
diff --git a/hypothesis-python/tests/pytest/test_profiles.py b/hypothesis-python/tests/pytest/test_profiles.py
--- a/hypothesis-python/tests/pytest/test_profiles.py
+++ b/hypothesis-python/tests/pytest/test_profiles.py
@@ -14,8 +14,8 @@
 # END HEADER
 
 import pytest
+from _hypothesis_pytestplugin import LOAD_PROFILE_OPTION
 
-from hypothesis.extra.pytestplugin import LOAD_PROFILE_OPTION
 from hypothesis.version import __version__
 
 pytest_plugins = "pytester"
diff --git a/hypothesis-python/tests/pytest/test_pytest_detection.py b/hypothesis-python/tests/pytest/test_pytest_detection.py
--- a/hypothesis-python/tests/pytest/test_pytest_detection.py
+++ b/hypothesis-python/tests/pytest/test_pytest_detection.py
@@ -33,3 +33,16 @@ def test_is_not_running_under_pytest(tmpdir):
     pyfile = tmpdir.join("test.py")
     pyfile.write(FILE_TO_RUN)
     subprocess.check_call([sys.executable, str(pyfile)])
+
+
+DOES_NOT_IMPORT_HYPOTHESIS = """
+import sys
+
+def test_pytest_plugin_does_not_import_hypothesis():
+    assert "hypothesis" not in sys.modules
+"""
+
+
+def test_plugin_does_not_import_pytest(testdir):
+    testdir.makepyfile(DOES_NOT_IMPORT_HYPOTHESIS)
+    testdir.runpytest_subprocess().assert_outcomes(passed=1)
diff --git a/hypothesis-python/tests/pytest/test_statistics.py b/hypothesis-python/tests/pytest/test_statistics.py
--- a/hypothesis-python/tests/pytest/test_statistics.py
+++ b/hypothesis-python/tests/pytest/test_statistics.py
@@ -16,8 +16,7 @@
 from distutils.version import LooseVersion
 
 import pytest
-
-from hypothesis.extra.pytestplugin import PRINT_STATISTICS_OPTION
+from _hypothesis_pytestplugin import PRINT_STATISTICS_OPTION
 
 pytest_plugins = "pytester"