Repository: HypothesisWorks/hypothesis

Issue:
"You can reproduce this example..." doesn't respect Verbosity.quiet
Since 3.44.0 we have the message "You can reproduce this example by temporarily adding `@reproduce_failure(...)" for failing tests, which is nice.

What's less nice is that it shows up even if you have `verbosity=Verbosity.quiet` in your settings, but this should suppress all output from Hypothesis.

`Verbosity.quiet` is [supposed to be enforced by report](https://github.com/HypothesisWorks/hypothesis/blob/master/hypothesis-python/src/hypothesis/reporting.py#L72) so my guess is that we're running this line outside of the current settings for the test.


Discussion:
Happy to have a look at this.

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/_settings.py b/hypothesis-python/src/hypothesis/_settings.py
--- a/hypothesis-python/src/hypothesis/_settings.py
+++ b/hypothesis-python/src/hypothesis/_settings.py
@@ -802,11 +802,12 @@ class PrintSettings(Enum):
     """Make an educated guess as to whether it would be appropriate to print
     the blob.
 
-    The current rules are that this will print if both:
+    The current rules are that this will print if:
 
     1. The output from Hypothesis appears to be unsuitable for use with
-       :func:`~hypothesis.example`.
-    2. The output is not too long."""
+       :func:`~hypothesis.example`, and
+    2. The output is not too long, and
+    3. Verbosity is at least normal."""
 
     ALWAYS = 2
     """Always print a blob on failure."""
diff --git a/hypothesis-python/src/hypothesis/core.py b/hypothesis-python/src/hypothesis/core.py
--- a/hypothesis-python/src/hypothesis/core.py
+++ b/hypothesis-python/src/hypothesis/core.py
@@ -858,6 +858,7 @@ def run(self):
                     if (
                         self.settings.print_blob is PrintSettings.ALWAYS or (
                             self.settings.print_blob is PrintSettings.INFER and
+                            self.settings.verbosity >= Verbosity.normal and
                             not can_use_repr and
                             len(failure_blob) < 200
                         )


Test patch:
diff --git a/hypothesis-python/tests/cover/test_reproduce_failure.py b/hypothesis-python/tests/cover/test_reproduce_failure.py
--- a/hypothesis-python/tests/cover/test_reproduce_failure.py
+++ b/hypothesis-python/tests/cover/test_reproduce_failure.py
@@ -24,8 +24,8 @@
 import pytest
 
 import hypothesis.strategies as st
-from hypothesis import PrintSettings, given, reject, example, settings, \
-    __version__, reproduce_failure
+from hypothesis import Verbosity, PrintSettings, given, reject, example, \
+    settings, __version__, reproduce_failure
 from hypothesis.core import decode_failure, encode_failure
 from hypothesis.errors import DidNotReproduce, InvalidArgument, \
     UnsatisfiedAssumption
@@ -210,3 +210,16 @@ def test(x):
 
     with pytest.raises(InvalidArgument):
         test()
+
+
+def test_does_not_print_reproduction_if_verbosity_set_to_quiet():
+    @given(st.data())
+    @settings(verbosity=Verbosity.quiet)
+    def test_always_fails(data):
+        assert data.draw(st.just(False))
+
+    with capture_out() as out:
+        with pytest.raises(AssertionError):
+            test_always_fails()
+
+    assert '@reproduce_failure' not in out.getvalue()