Repository: HypothesisWorks/hypothesis

Issue:
Replace usage of `getrandbits` with calls to `draw_bits`
In `hypothesis.internal.conjecture.utils` we have a `getrandbits` method which takes a `ConjectureData` and a number of bits to draw. On `ConjectureData` we have a method `draw_bits` which takes a number of bits to draw.

You might reasonably assume that these functions do the same thing. Certainly I did. However, you would be wrong! 

For reasons that I am currently unaware of, if you make the following replacement:

```python
def getrandbits(data, n):
    return data.draw_bits(n)
```

then the endpoint tests in `tests/cover/test_float_nastiness.py` start failing.

The source of this difference is the setting of the mask in `ConjectureData`. If you comment out the line `self.masked_indices[self.index] = mask` then this starts working.

Why? Beats me.

There are basically two possibilities:

1. There's a really subtle bug somewhere.
2. This is basically just working by coincidence at the moment, as a result of some interaction between the strategies and our weird mutator.

My hunch is that it's the latter.


Discussion:
A large factor seems to be the code that splices the novel prefix over part of the mutator's output:

https://github.com/HypothesisWorks/hypothesis/blob/35b2c3cbad2e7eafb7bd3e18271d383cb3624698/hypothesis-python/src/hypothesis/internal/conjecture/engine.py#L479-L490

I see lots of cases where `draw_max` produces bytes that *should* find the desired endpoint (lots of `0xff`), but then the prefix bytes get spliced over the top and the opportunity is lost.

I haven't fully determined why this isn't also hurting with the existing `getrandbits` implementation, but it's plausible that the lack of normalization gives the novel-prefix generator more chances to stumble across a suitable prefix while `draw_max` is in use.
The prefixer and the mutator are both doing their respective jobs, so the underlying problem looks like a design hurdle rather than an implementation slip-up.

I suspect the long-term solution is to integrate prefix-generation and mutation, so that we first consult the mutator, and only override it if it produces something that has already been exhausted.

That sounds like a hassle, but maybe it will be easier once the block-based tree is in place.
I can reproduce this effect in endpoint tests for `st.integers(0, 2 ** n - 1)` for values of `n` in the high 10s or low 20s.

(And since `integers` doesn't use `getrandbits`, this affects current versions of Hypothesis.)
Possibly related to #909?  I thought that was fixed, but maybe it just got harder to trigger...
> I suspect the long-term solution is to integrate prefix-generation and mutation, so that we first consult the mutator, and only override it if it produces something that has already been exhausted.

I have a plan, outlined [here](https://notebook.drmaciver.com/posts/2019-02-15-14:37.html) that would let us do away with the prefixer entirely I think, by allowing us to cache interactions against arbitrary `ConjectureData` objects instead of just buffers, which will mean we can write fairly complex mutators and still cache them without worrying about the prefixes.

All that being said, the current mutator code is basically trash. The only reasons it still exists are:

* It works *slightly* better than not having it in some cases (including our test suite, unsurprisingly).
* Any attempt to replace it should start with a serious think about how to evaluate the difference, and we/I haven't done that.

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/internal/conjecture/utils.py b/hypothesis-python/src/hypothesis/internal/conjecture/utils.py
--- a/hypothesis-python/src/hypothesis/internal/conjecture/utils.py
+++ b/hypothesis-python/src/hypothesis/internal/conjecture/utils.py
@@ -154,25 +154,12 @@ def choice(data, values):
     return values[integer_range(data, 0, len(values) - 1)]
 
 
-def getrandbits(data, n):
-    # This method is equivalent to data.draw_bits(n) except that it fails
-    # to set the mask. This method should die but is currently maintaining
-    # bugwards compatibility with some oddities of behaviour that we've
-    # not yet fully debugged. See
-    # https://github.com/HypothesisWorks/hypothesis/issues/1827
-    # for details.
-    n_bytes = n // 8
-    if n % 8 != 0:
-        n_bytes += 1
-    return data.draw_bits(n_bytes * 8) & ((1 << n) - 1)
-
-
 FLOAT_PREFIX = 0b1111111111 << 52
 FULL_FLOAT = int_to_float(FLOAT_PREFIX | ((2 << 53) - 1)) - 1
 
 
 def fractional_float(data):
-    return (int_to_float(FLOAT_PREFIX | getrandbits(data, 52)) - 1) / FULL_FLOAT
+    return (int_to_float(FLOAT_PREFIX | data.draw_bits(52)) - 1) / FULL_FLOAT
 
 
 def boolean(data):


Test patch:
diff --git a/hypothesis-python/tests/cover/test_conjecture_utils.py b/hypothesis-python/tests/cover/test_conjecture_utils.py
--- a/hypothesis-python/tests/cover/test_conjecture_utils.py
+++ b/hypothesis-python/tests/cover/test_conjecture_utils.py
@@ -26,7 +26,6 @@
 from hypothesis.internal.compat import hbytes, hrange
 from hypothesis.internal.conjecture.data import ConjectureData
 from hypothesis.internal.coverage import IN_COVERAGE_TESTS
-from tests.cover.test_conjecture_engine import run_to_buffer
 
 
 def test_does_draw_data_for_empty_range():
@@ -147,12 +146,3 @@ def test_sampler_distribution(weights):
         calculated[base] += (1 - p_alternate) / n
         calculated[alternate] += p_alternate / n
     assert probabilities == calculated
-
-
-def test_get_randbits_can_set_high_bit():
-    @run_to_buffer
-    def x(data):
-        if cu.getrandbits(data, 8) >> 7:
-            data.mark_interesting()
-
-    assert x == hbytes([1 << 7])