Repository: HypothesisWorks/hypothesis

Issue:
Avoid "impossible" ZeroDivisionError we keep seeing
https://github.com/HypothesisWorks/hypothesis/blob/592bc7cd2c642bf273113083457444b4823021c9/hypothesis-python/src/hypothesis/statistics.py#L88-L95

This snippet occasionally leads to problems, when `total_drawtime` is nonzero but `total_runtime` *is* zero, causing test failures [like this one](https://ci.appveyor.com/project/DRMacIver/hypothesis/build/1.0.458/job/hl2d74810901ttf5#L926).  I'm not sure why this happens - it's been observed on Python 2 (without) and Python 3 (with) a monotonic clock, so there might be multiple causes - but the solution is pretty simple.  All you need to do to fix this is to change the first conditional to

```python
if total_drawtime <= 0.0 or total_runtime <= 0.0:
```

Then write up a short changelog (see guides/documentation.rst), add your name to the contributors list (in - and read! - contributing.rst), and open a pull request!  If you have any questions, just let me know and I'd be happy to help out.


Discussion:
My inclination would be not to fix this until we have a reproducible test case, or we’re just hiding the error.

Not sure if that disqualifies it for [good-first-issue].
It's easy (or at least straightforward) to reproduce by mocking the global clock, but otherwise it only comes up *often* on shared VMs such as free CI where the time is screwy.

Given that the logic is OK and the measurements are weird, I'm OK with just displaying the "we're not sure" value of `~ 0%` when things go wrong - it's at least as accurate as any other number we could display.

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/statistics.py b/hypothesis-python/src/hypothesis/statistics.py
--- a/hypothesis-python/src/hypothesis/statistics.py
+++ b/hypothesis-python/src/hypothesis/statistics.py
@@ -87,8 +87,15 @@ def __init__(self, engine):
         total_runtime = math.fsum(engine.all_runtimes)
         total_drawtime = math.fsum(engine.all_drawtimes)
 
-        if total_drawtime == 0.0:
+        if total_drawtime == 0.0 and total_runtime >= 0.0:
             self.draw_time_percentage = '~ 0%'
+        elif total_drawtime < 0.0 or total_runtime <= 0.0:
+            # This weird condition is possible in two ways:
+            # 1.  drawtime and/or runtime are negative, due to clock changes
+            #     on Python 2 or old OSs (we use monotonic() where available)
+            # 2.  floating-point issues *very rarely* cause math.fsum to be
+            #     off by the lowest bit, so drawtime==0 and runtime!=0, eek!
+            self.draw_time_percentage = 'NaN'
         else:
             draw_time_percentage = 100.0 * min(
                 1, total_drawtime / total_runtime)


Test patch:
diff --git a/hypothesis-python/tests/cover/test_statistical_events.py b/hypothesis-python/tests/cover/test_statistical_events.py
--- a/hypothesis-python/tests/cover/test_statistical_events.py
+++ b/hypothesis-python/tests/cover/test_statistical_events.py
@@ -25,7 +25,10 @@
 
 from hypothesis import HealthCheck, event, given, assume, example, settings
 from hypothesis import strategies as st
-from hypothesis.statistics import collector
+from hypothesis.statistics import Statistics, collector
+from hypothesis.internal.conjecture.data import Status
+from hypothesis.internal.conjecture.engine import ExitReason, \
+    ConjectureRunner
 
 
 def call_for_statistics(test_function):
@@ -193,3 +196,20 @@ def test(n):
 
     stats = call_for_statistics(test)
     assert 'shrunk example' in stats.exit_reason
+
+
+@pytest.mark.parametrize('drawtime,runtime', [
+    (1, 0), (-1, 0), (0, -1), (-1, -1),
+])
+def test_weird_drawtime_issues(drawtime, runtime):
+    # Regression test for #1346, where we don't have the expected relationship
+    # 0<=drawtime<= runtime due to changing clocks or floating-point issues.
+    engine = ConjectureRunner(lambda: None)
+    engine.exit_reason = ExitReason.finished
+    engine.status_runtimes[Status.VALID] = [0]
+
+    engine.all_drawtimes.append(drawtime)
+    engine.all_runtimes.extend([0, runtime])
+
+    stats = Statistics(engine)
+    assert stats.draw_time_percentage == 'NaN'