Repository: HypothesisWorks/hypothesis (Python)

Issue:
Shrinker.explain() raises internal assertion
I'm experiencing internal assertion errors from the shrinker. I am using hypothesis 6.130.8 on Linux installed from PYPI.

```
[2025-04-02T17:10:43.319Z]                 # Turns out this was a variable-length part, so grab the infix...
[2025-04-02T17:10:43.319Z]                 if result.status is Status.OVERRUN:
[2025-04-02T17:10:43.319Z]                     continue  # pragma: no cover  # flakily covered
[2025-04-02T17:10:43.319Z]                 result = cast(ConjectureResult, result)
[2025-04-02T17:10:43.319Z]                 if not (
[2025-04-02T17:10:43.319Z]                     len(attempt) == len(result.choices)
[2025-04-02T17:10:43.319Z]                     and endswith(result.nodes, nodes[end:])
[2025-04-02T17:10:43.319Z]                 ):
[2025-04-02T17:10:43.319Z]                     for ex, res in zip(shrink_target.spans, result.spans):
[2025-04-02T17:10:43.319Z]                         assert ex.start == res.start
[2025-04-02T17:10:43.319Z]                         assert ex.start <= start
[2025-04-02T17:10:43.319Z] >                       assert ex.label == res.label
[2025-04-02T17:10:43.319Z] E                       assert 160697414095697247 == 8768992231400091199
[2025-04-02T17:10:43.319Z] E                        +  where 160697414095697247 = spans[4].label
[2025-04-02T17:10:43.319Z] E                        +  and   8768992231400091199 = spans[4].label
[2025-04-02T17:10:43.319Z] 
[2025-04-02T17:10:43.319Z] ..python3.11/site-packages/hypothesis/internal/conjecture/shrinker.py:552: AssertionError
```

Any hints about what situations might be causing this would be helpful. So far, I've got a test which always raises an error which results in this assertion instead of an explanation that the problem was irreducible and what the error is that hypothesis is trying to reduce.

```python
import collections

from hypothesis import given
from hypothesis.strategies import (
    one_of,
    none,
    sampled_from,
    fixed_dictionaries,
    composite,
)

MatmulInputs = collections.namedtuple(
    "MatmulInputs",
    [
        "m",
    ],
)


@composite
def matrix_multiply_arrays(draw):
    m = draw(one_of(none(), sampled_from([1])))
    return MatmulInputs(m=m)


options_blocking_values = [True, "auto"]


@given(
    input_arrays=matrix_multiply_arrays(),
    options=one_of(
        none(),
        fixed_dictionaries(
            {
                "blocking": sampled_from(options_blocking_values),
            }
        ),
    ),
)
def test_matmul(input_arrays, options):
    """Call nvmath.linalg.advanced.matmul() with valid inputs."""
    raise RuntimeError("MOCK error that always happens")

```



Discussion: So far, I have been able to reduce my reproducer script of this error (😆 I'm the shrinker finding an error in the shrinker) to the above. My current guess is that the error is caused by use of the `one_of(none(), ...)`.
Hmm, that's not reproducing for me.

(but I'm pretty sure that's happening during [the `explain` phase](https://hypothesis.readthedocs.io/en/latest/reference/api.html#phases), so if you want to work around it you could disable that)
This doesn't reproduce for me either. Does this still reproduce for you with `@settings(database=None)`? I wonder if you have a local cached failing example which the explain phase is erroring on. (in which case, zipping and uploading your `.hypothesis` would be helpful to reproduce).
I wasn't able to reproduce either once I moved this script to a new project folder, so there's something else that I haven't captured in the reproducer that is contributing. I'll retry this week.
```python
import collections
import datetime
import hypothesis
from hypothesis import given
from hypothesis.strategies import (
    one_of,
    none,
    floats,
    integers,
    sampled_from,
    fixed_dictionaries,
    composite,
)
import pytest

hypothesis.settings.register_profile(
    "nvmath",
    derandomize=True,
    deadline=datetime.timedelta(minutes=5),
    verbosity=hypothesis.Verbosity.normal,
    print_blob=True,
)
hypothesis.settings.load_profile("nvmath")


problem_size_mnk = integers(min_value=1, max_value=256)

options_blocking_values = [True, "auto"]

MatmulInputs = collections.namedtuple(
    "MatmulInputs",
    [
        "a",
        "b",
        "c",
        "m",
        "n",
        "k",
        "ab_type",
        "bias",
        "beta",
        "alpha",
        "epilogs",
    ],
)


@composite
def matrix_multiply_arrays(draw):
    m = draw(one_of(none(), problem_size_mnk))
    n = draw(one_of(none(), problem_size_mnk))
    k = draw(problem_size_mnk)
    ab_type = None
    a = None
    b = None
    c = None
    beta = None
    alpha = None
    epilogs = None
    bias = None
    return MatmulInputs(
        a=a,
        b=b,
        c=c,
        m=m,
        n=n,
        k=k,
        ab_type=ab_type,
        bias=bias,
        beta=beta,
        alpha=alpha,
        epilogs=epilogs,
    )


@given(
    input_arrays=matrix_multiply_arrays(),
    order=sampled_from(["F", "C"]),
    options=one_of(
        none(),
        fixed_dictionaries(
            {
                "blocking": sampled_from(options_blocking_values),
            }
        ),
    ),
    preferences=one_of(
        none(),
        fixed_dictionaries(
            {
                "max_waves_count": one_of(floats(min_value=0, max_value=100, width=32)),
            }
        ),
    ),
)
def test_matmul(input_arrays, order, options, preferences):
    """Call nvmath.linalg.advanced.matmul() with valid inputs."""
    raise RuntimeError("This error always occurs.")


if __name__ == "__main__":
    pytest.main()
```

What about this one? I moved the script into a new folder/environment and set `derandomize=True`, so the seed should be based on the function signature. I created a new environment with `python==3.12 pytest==8.3.5 hypothesis==6.130.10`.
That one does it, thanks.



Fix (patch):
diff --git a/hypothesis-python/RELEASE.rst b/hypothesis-python/RELEASE.rst
new file mode 100644
index 0000000000..e48ecb614a
--- /dev/null
+++ b/hypothesis-python/RELEASE.rst
@@ -0,0 +1,3 @@
+RELEASE_TYPE: patch
+
+Fix the caching behavior of |st.sampled_from|, which in rare cases led to failing an internal assertion (:issue:`4339`).
diff --git a/hypothesis-python/src/hypothesis/internal/conjecture/shrinker.py b/hypothesis-python/src/hypothesis/internal/conjecture/shrinker.py
index 1802034110..0aac436012 100644
--- a/hypothesis-python/src/hypothesis/internal/conjecture/shrinker.py
+++ b/hypothesis-python/src/hypothesis/internal/conjecture/shrinker.py
@@ -540,7 +540,6 @@ def explain(self) -> None:
                 attempt = choices[:start] + tuple(replacement) + choices[end:]
                 result = self.engine.cached_test_function(attempt, extend="full")
 
-                # Turns out this was a variable-length part, so grab the infix...
                 if result.status is Status.OVERRUN:
                     continue  # pragma: no cover  # flakily covered
                 result = cast(ConjectureResult, result)
@@ -548,20 +547,23 @@ def explain(self) -> None:
                     len(attempt) == len(result.choices)
                     and endswith(result.nodes, nodes[end:])
                 ):
-                    for ex, res in zip(shrink_target.spans, result.spans):
-                        assert ex.start == res.start
-                        assert ex.start <= start
-                        assert ex.label == res.label
-                        if start == ex.start and end == ex.end:
-                            res_end = res.end
+                    # Turns out this was a variable-length part, so grab the infix...
+                    for span1, span2 in zip(shrink_target.spans, result.spans):
+                        assert span1.start == span2.start
+                        assert span1.start <= start
+                        assert span1.label == span2.label
+                        if span1.start == start and span1.end == end:
+                            result_end = span2.end
                             break
                     else:
                         raise NotImplementedError("Expected matching prefixes")
 
                     attempt = (
-                        choices[:start] + result.choices[start:res_end] + choices[end:]
+                        choices[:start]
+                        + result.choices[start:result_end]
+                        + choices[end:]
                     )
-                    chunks[(start, end)].append(result.choices[start:res_end])
+                    chunks[(start, end)].append(result.choices[start:result_end])
                     result = self.engine.cached_test_function(attempt)
 
                     if result.status is Status.OVERRUN:
diff --git a/hypothesis-python/src/hypothesis/strategies/_internal/lazy.py b/hypothesis-python/src/hypothesis/strategies/_internal/lazy.py
index ce202a338c..b398c165e2 100644
--- a/hypothesis-python/src/hypothesis/strategies/_internal/lazy.py
+++ b/hypothesis-python/src/hypothesis/strategies/_internal/lazy.py
@@ -104,6 +104,9 @@ def calc_is_cacheable(self, recur: RecurT) -> bool:
                     return False
         return True
 
+    def calc_label(self) -> int:
+        return self.wrapped_strategy.label
+
     @property
     def wrapped_strategy(self) -> SearchStrategy[Ex]:
         if self.__wrapped_strategy is None:
@@ -176,7 +179,3 @@ def __repr__(self) -> str:
 
     def do_draw(self, data: ConjectureData) -> Ex:
         return data.draw(self.wrapped_strategy)
-
-    @property
-    def label(self) -> int:
-        return self.wrapped_strategy.label
diff --git a/hypothesis-python/src/hypothesis/strategies/_internal/strategies.py b/hypothesis-python/src/hypothesis/strategies/_internal/strategies.py
index 8585c72c3f..58cb66b2f1 100644
--- a/hypothesis-python/src/hypothesis/strategies/_internal/strategies.py
+++ b/hypothesis-python/src/hypothesis/strategies/_internal/strategies.py
@@ -564,14 +564,53 @@ def __repr__(self) -> str:
         )
 
     def calc_label(self) -> int:
-        return combine_labels(
-            self.class_label,
-            *(
-                (calc_label_from_hash(self.elements),)
-                if is_hashable(self.elements)
-                else ()
-            ),
-        )
+        # strategy.label is effectively an under-approximation of structural
+        # equality (i.e., some strategies may have the same label when they are not
+        # structurally identical). More importantly for calculating the
+        # SampledFromStrategy label, we might have hash(s1) != hash(s2) even
+        # when s1 and s2 are structurally identical. For instance:
+        #
+        #   s1 = st.sampled_from([st.none()])
+        #   s2 = st.sampled_from([st.none()])
+        #   assert hash(s1) != hash(s2)
+        #
+        # (see also test cases in test_labels.py).
+        #
+        # We therefore use the labels of any component strategies when calculating
+        # our label, and only use the hash if it is not a strategy.
+        #
+        # That's the ideal, anyway. In reality the logic is more complicated than
+        # necessary in order to be efficient in the presence of (very) large sequences:
+        # * add an unabashed special case for range, to avoid iteration over an
+        #   enormous range when we know it is entirely integers.
+        # * if there is at least one strategy in self.elements, use strategy label,
+        #   and the element hash otherwise.
+        # * if there are no strategies in self.elements, take the hash of the
+        #   entire sequence. This prevents worst-case performance of hashing each
+        #   element when a hash of the entire sequence would have sufficed.
+        #
+        # The worst case performance of this scheme is
+        # itertools.chain(range(2**100), [st.none()]), where it degrades to
+        # hashing every int in the range.
+
+        if isinstance(self.elements, range) or (
+            is_hashable(self.elements)
+            and not any(isinstance(e, SearchStrategy) for e in self.elements)
+        ):
+            return combine_labels(self.class_label, calc_label_from_hash(self.elements))
+
+        labels = [self.class_label]
+        for element in self.elements:
+            if not is_hashable(element):
+                continue
+
+            labels.append(
+                element.label
+                if isinstance(element, SearchStrategy)
+                else calc_label_from_hash(element)
+            )
+
+        return combine_labels(*labels)
 
     def calc_has_reusable_values(self, recur: RecurT) -> bool:
         # Because our custom .map/.filter implementations skip the normal