Repository: HypothesisWorks/hypothesis

Issue:
Add Numpy strategy for `multiple_shapes` which are mutually broadcastable
The `broadcastable_shapes(shape)` strategy is great, but sometimes you need *several* shapes, all of which are broadcastable against any of the others - for example to describe the [loop dimensions of generalised ufuncs](https://docs.scipy.org/doc/numpy/reference/c-api.generalized-ufuncs.html).  This may not be needed often, but be critical when it is!

I therefore propose to add the following strategy:

```python
Shape = Tuple[int, ...]

def multiple_shapes(
    inputs,
    base_shape=(),
    min_dims=0,
    max_dims=None,
    min_side=1,
    max_side=None,
):
    # type: (int, Shape, int, int, int, int) -> Tuple[Tuple[Shape, ...], Shape]
    ...
    # Returns a namedtuple of (input_shapes, result_shape)
```

This strategy will be defined in the first release as generating an `len(inputs)`-tuple of mutually-broadcastable array shapes, and the expected result shape as the second element of a named tuple.

In future releases, `inputs` may be a gufunc signature string describing core dimensions, with the other arguments interpreted as describing the loop dimensions (if any).  This plan replaces #1784.


Discussion:
FYI we have something like this in TFP:
https://github.com/tensorflow/probability/blob/master/tensorflow_probability/python/internal/hypothesis_testlib.py#L489
Oh, cool!  I didn't know that Google was using Hypothesis... though I'd strongly recommend upgrading to a newer version, beyond the usual bugfixes and new features we have a couple of perfomance improvements along the lines of "things that were quadratic are now n-log-n or linear" which are *very nice*.  

(reconfiguring some of those strategies for data locality would probably help shrink time a lot too if that's been a problem for you, but updating Hypothesis would be an easier win.  feel free to contact me to discuss)

We're probably going to take a slightly different approach here to support future generation based on ufunc signatures, where the shape-to-broadcast-to is bounded but not fixed.  Plus some use and abuse of internal APIs to get as close to ideal shrinking as we can, I hope!
I've been working on the upgrade. :) I admit I'm curious how you knew our
version was out of date. Is there a timeout= arg in there somewhere? :)

Brian Patton | Software Engineer | bjp@google.com



On Sat, Aug 10, 2019 at 1:42 AM Zac Hatfield-Dodds <notifications@github.com>
wrote:

> Oh, cool! I didn't know that Google was using Hypothesis... though I'd
> strongly recommend upgrading to a newer version, beyond the usual bugfixes
> and new features we have a couple of perfomance improvements along the
> lines of "things that were quadratic are now n-log-n or linear" which are *very
> nice*.
>
> (reconfiguring some of those strategies for data locality would probably
> help shrink time a lot too if that's been a problem for you, but updating
> Hypothesis would be an easier win. feel free to contact me to discuss)
>
> We're probably going to take a slightly different approach here to support
> future generation based on ufunc signatures, where the
> shape-to-broadcast-to is bounded but not fixed. Plus some use and abuse of
> internal APIs to get as close to ideal shrinking as we can, I hope!
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/HypothesisWorks/hypothesis/issues/1969?email_source=notifications&email_token=AFJFSI2ZTCQGFCRGJWTBKYDQDZPMPA5CNFSM4HLMYBUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4AHIUA#issuecomment-520123472>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AFJFSIYZGXKO3DTRWKZYB2LQDZPMPANCNFSM4HLMYBUA>
> .
>

Ooh, exciting to hear that there'll be an update 😄 

I didn't *know* so much as have a very likely guess: I know roughly how the Google monorepo works, and that someone was experimenting with Hypothesis a year or so ago, so I was comfortable betting that "update please" is good advice 😉... and the file you linked has some comments up near the settings config which I didn't think were out of date.

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/extra/numpy.py b/hypothesis-python/src/hypothesis/extra/numpy.py
--- a/hypothesis-python/src/hypothesis/extra/numpy.py
+++ b/hypothesis-python/src/hypothesis/extra/numpy.py
@@ -18,6 +18,7 @@
 from __future__ import absolute_import, division, print_function
 
 import math
+from collections import namedtuple
 
 import numpy as np
 
@@ -34,11 +35,23 @@
 from hypothesis.searchstrategy import SearchStrategy
 from hypothesis.utils.conventions import not_set
 
+if PY2:
+    BroadcastableShapes = namedtuple(
+        "BroadcastableShapes", ["input_shapes", "result_shape"]
+    )
+else:
+    from typing import NamedTuple, Tuple
+
+    Shape = Tuple[int, ...]  # noqa
+    BroadcastableShapes = NamedTuple(
+        "BroadcastableShapes",
+        [("input_shapes", Tuple[Shape, ...]), ("result_shape", Shape)],
+    )
+
 if False:
-    from typing import Any, Union, Sequence, Tuple, Optional  # noqa
+    from typing import Any, Union, Sequence, Tuple  # noqa
     from hypothesis.searchstrategy.strategies import T  # noqa
 
-    Shape = Tuple[int, ...]  # noqa
     BasicIndex = Tuple[Union[int, slice, ellipsis, np.newaxis], ...]  # noqa
 
 TIME_RESOLUTIONS = tuple("Y  M  D  h  m  s  ms  us  ns  ps  fs  as".split())
@@ -740,52 +753,6 @@ def valid_tuple_axes(ndim, min_size=0, max_size=None):
     return st.lists(axes, min_size, max_size, unique_by=lambda x: x % ndim).map(tuple)
 
 
-class BroadcastShapeStrategy(SearchStrategy):
-    def __init__(self, shape, min_dims, max_dims, min_side, max_side):
-        assert 0 <= min_side <= max_side
-        assert 0 <= min_dims <= max_dims <= 32
-        SearchStrategy.__init__(self)
-        self.shape = shape
-        self.side_strat = st.integers(min_side, max_side)
-        self.min_dims = min_dims
-        self.max_dims = max_dims
-        self.min_side = min_side
-        self.max_side = max_side
-
-    def do_draw(self, data):
-        elements = cu.many(
-            data,
-            min_size=self.min_dims,
-            max_size=self.max_dims,
-            average_size=min(
-                max(self.min_dims * 2, self.min_dims + 5),
-                0.5 * (self.min_dims + self.max_dims),
-            ),
-        )
-        result = []
-        reversed_shape = tuple(self.shape[::-1])
-        while elements.more():
-            if len(result) < len(self.shape):
-                # Shrinks towards original shape
-                if reversed_shape[len(result)] == 1:
-                    if self.min_side <= 1 and not data.draw(st.booleans()):
-                        side = 1
-                    else:
-                        side = data.draw(self.side_strat)
-                elif self.max_side >= reversed_shape[len(result)] and (
-                    not self.min_side <= 1 <= self.max_side or data.draw(st.booleans())
-                ):
-                    side = reversed_shape[len(result)]
-                else:
-                    side = 1
-            else:
-                side = data.draw(self.side_strat)
-            result.append(side)
-        assert self.min_dims <= len(result) <= self.max_dims
-        assert all(self.min_side <= s <= self.max_side for s in result)
-        return tuple(reversed(result))
-
-
 @st.defines_strategy
 def broadcastable_shapes(shape, min_dims=0, max_dims=None, min_side=1, max_side=None):
     # type: (Shape, int, int, int, int) -> st.SearchStrategy[Shape]
@@ -863,8 +830,216 @@ def broadcastable_shapes(shape, min_dims=0, max_dims=None, min_side=1, max_side=
                 max_dims = n
                 break
 
-    return BroadcastShapeStrategy(
-        shape,
+    return MutuallyBroadcastableShapesStrategy(
+        num_shapes=1,
+        base_shape=shape,
+        min_dims=min_dims,
+        max_dims=max_dims,
+        min_side=min_side,
+        max_side=max_side,
+    ).map(lambda x: x.input_shapes[0])
+
+
+class MutuallyBroadcastableShapesStrategy(SearchStrategy):
+    def __init__(
+        self,
+        num_shapes,
+        base_shape=(),
+        min_dims=0,
+        max_dims=None,
+        min_side=1,
+        max_side=None,
+    ):
+        assert 0 <= min_side <= max_side
+        assert 0 <= min_dims <= max_dims <= 32
+        SearchStrategy.__init__(self)
+        self.base_shape = base_shape
+        self.side_strat = st.integers(min_side, max_side)
+        self.num_shapes = num_shapes
+        self.min_dims = min_dims
+        self.max_dims = max_dims
+        self.min_side = min_side
+        self.max_side = max_side
+
+        self.size_one_allowed = self.min_side <= 1 <= self.max_side
+
+    def do_draw(self, data):
+        # All shapes are handled in column-major order; i.e. they are reversed
+        base_shape = self.base_shape[::-1]
+        result_shape = list(base_shape)
+        shapes = [[] for _ in range(self.num_shapes)]
+        use = [True for _ in range(self.num_shapes)]
+
+        for dim_count in range(1, self.max_dims + 1):
+            dim = dim_count - 1
+
+            # We begin by drawing a valid dimension-size for the given
+            # dimension. This restricts the variability across the shapes
+            # at this dimension such that they can only choose between
+            # this size and a singleton dimension.
+            if len(base_shape) < dim_count or base_shape[dim] == 1:
+                # dim is unrestricted by the base-shape: shrink to min_side
+                dim_side = data.draw(self.side_strat)
+            elif base_shape[dim] <= self.max_side:
+                # dim is aligned with non-singleton base-dim
+                dim_side = base_shape[dim]
+            else:
+                # only a singleton is valid in alignment with the base-dim
+                dim_side = 1
+
+            for shape_id, shape in enumerate(shapes):
+                # Populating this dimension-size for each shape, either
+                # the drawn size is used or, if permitted, a singleton
+                # dimension.
+                if dim_count <= len(base_shape) and self.size_one_allowed:
+                    # aligned: shrink towards size 1
+                    side = data.draw(st.sampled_from([1, dim_side]))
+                else:
+                    side = dim_side
+
+                # Use a trick where where a biased coin is queried to see
+                # if the given shape-tuple will continue to be grown. All
+                # of the relevant draws will still be made for the given
+                # shape-tuple even if it is no longer being added to.
+                # This helps to ensure more stable shrinking behavior.
+                if self.min_dims < dim_count:
+                    use[shape_id] &= cu.biased_coin(
+                        data, 1 - 1 / (1 + self.max_dims - dim)
+                    )
+
+                if use[shape_id]:
+                    shape.append(side)
+                    if len(result_shape) < len(shape):
+                        result_shape.append(shape[-1])
+                    elif shape[-1] != 1 and result_shape[dim] == 1:
+                        result_shape[dim] = shape[-1]
+            if not any(use):
+                break
+
+        result_shape = result_shape[: max(map(len, [self.base_shape] + shapes))]
+
+        assert len(shapes) == self.num_shapes
+        assert all(self.min_dims <= len(s) <= self.max_dims for s in shapes)
+        assert all(self.min_side <= s <= self.max_side for side in shapes for s in side)
+
+        return BroadcastableShapes(
+            input_shapes=tuple(tuple(reversed(shape)) for shape in shapes),
+            result_shape=tuple(reversed(result_shape)),
+        )
+
+
+@st.defines_strategy
+@reserved_means_kwonly_star
+def mutually_broadcastable_shapes(
+    __reserved=not_set,
+    num_shapes=not_set,
+    base_shape=(),
+    min_dims=0,
+    max_dims=None,
+    min_side=1,
+    max_side=None,
+):
+    # type: (Any, Any, Shape , int, int, int, int) -> st.SearchStrategy[BroadcastableShapes]
+    """Return a strategy for generating a specified number of shapes, N, that are
+    mutually-broadcastable with one another and with the provided "base-shape".
+
+    The strategy will generate a named-tuple of:
+     * input_shapes: the N generated shapes
+     * result_shape: the resulting shape, produced by broadcasting the
+       N shapes with the base-shape
+
+    Each shape produced from this strategy shrinks towards a shape with length
+    ``min_dims``. The size of an aligned dimension shrinks towards being having
+    a size of 1. The size of an unaligned dimension shrink towards ``min_side``.
+
+    * ``num_shapes`` The number of mutually broadcast-compatible shapes to generate.
+    * ``base-shape`` The shape against which all generated shapes can broadcast.
+       The default shape is empty, which corresponds to a scalar and thus does not
+       constrain broadcasting at all.
+    * ``min_dims`` The smallest length that any generated shape can possess.
+    * ``max_dims`` The largest length that any generated shape can possess.
+      It cannot exceed 32, which is the greatest supported dimensionality for a
+      numpy array. The default-value for ``max_dims`` is
+      ``2 + max(len(shape), min_dims)`` - capped at 32.
+    * ``min_side`` The smallest size that an unaligned dimension can possess.
+    * ``max_side`` The largest size that an unaligned dimension can possess.
+      The default value is 2 + 'size-of-largest-aligned-dimension'.
+
+    The following are some examples drawn from this strategy.
+
+    .. code-block:: pycon
+
+        >>> # Each example will draw three shapes,
+        ... # and each shape is broadcast-compatible
+        ... # with `(2, 3)`
+        >>> [mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3)).example() for i in range(5)]
+        [BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3)),
+         BroadcastableShapes(input_shapes=((3,), (1,), (2, 1)), result_shape=(2, 3)),
+         BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3)),
+         BroadcastableShapes(input_shapes=((), (), ()), result_shape=(2, 3)),
+         BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(2, 3))]
+    """
+    if __reserved is not not_set:
+        raise InvalidArgument("Do not pass the __reserved argument.")
+
+    check_type(integer_types, num_shapes, "num_shapes")
+    if num_shapes < 1:
+        raise InvalidArgument("num_shapes=%s must be at least 1." % (num_shapes,))
+
+    check_type(tuple, base_shape, "base_shape")
+    strict_check = max_dims is not None
+    check_type(integer_types, min_side, "min_side")
+    check_type(integer_types, min_dims, "min_dims")
+
+    if max_dims is None:
+        max_dims = min(32, max(len(base_shape), min_dims) + 2)
+    else:
+        check_type(integer_types, max_dims, "max_dims")
+
+    if max_side is None:
+        max_side = max(tuple(base_shape[-max_dims:]) + (min_side,)) + 2
+    else:
+        check_type(integer_types, max_side, "max_side")
+
+    order_check("dims", 0, min_dims, max_dims)
+    order_check("side", 0, min_side, max_side)
+
+    if 32 < max_dims:
+        raise InvalidArgument("max_dims cannot exceed 32")
+
+    dims, bnd_name = (max_dims, "max_dims") if strict_check else (min_dims, "min_dims")
+
+    # check for unsatisfiable min_side
+    if not all(min_side <= s for s in base_shape[::-1][:dims] if s != 1):
+        raise InvalidArgument(
+            "Given base_shape=%r, there are no broadcast-compatible "
+            "shapes that satisfy: %s=%s and min_side=%s"
+            % (base_shape, bnd_name, dims, min_side)
+        )
+
+    # check for unsatisfiable [min_side, max_side]
+    if not (
+        min_side <= 1 <= max_side or all(s <= max_side for s in base_shape[::-1][:dims])
+    ):
+        raise InvalidArgument(
+            "Given base_shape=%r, there are no broadcast-compatible shapes "
+            "that satisfy all of %s=%s, min_side=%s, and max_side=%s"
+            % (base_shape, bnd_name, dims, min_side, max_side)
+        )
+
+    if not strict_check:
+        # reduce max_dims to exclude unsatisfiable dimensions
+        for n, s in zip(range(max_dims), reversed(base_shape)):
+            if s < min_side and s != 1:
+                max_dims = n
+                break
+            elif not (min_side <= 1 <= max_side or s <= max_side):
+                max_dims = n
+                break
+
+    return MutuallyBroadcastableShapesStrategy(
+        num_shapes=num_shapes,
+        base_shape=base_shape,
         min_dims=min_dims,
         max_dims=max_dims,
         min_side=min_side,


Test patch:
diff --git a/hypothesis-python/tests/cover/test_statistical_events.py b/hypothesis-python/tests/cover/test_statistical_events.py
--- a/hypothesis-python/tests/cover/test_statistical_events.py
+++ b/hypothesis-python/tests/cover/test_statistical_events.py
@@ -126,7 +126,7 @@ def test(i):
         time.sleep(0.5)
 
     stats = call_for_statistics(test)
-    assert re.match(r"~ 5\d\dms", stats.runtimes)
+    assert re.match(r"~ [56]\d\dms", stats.runtimes) is not None, stats.runtimes
 
 
 def test_apparently_instantaneous_tests():
diff --git a/hypothesis-python/tests/numpy/test_argument_validation.py b/hypothesis-python/tests/numpy/test_argument_validation.py
--- a/hypothesis-python/tests/numpy/test_argument_validation.py
+++ b/hypothesis-python/tests/numpy/test_argument_validation.py
@@ -117,6 +117,109 @@ def e(a, **kwargs):
             min_side=2,
             max_side=3,
         ),
+        e(nps.mutually_broadcastable_shapes, num_shapes=0),
+        e(nps.mutually_broadcastable_shapes, num_shapes="a"),
+        e(nps.mutually_broadcastable_shapes, num_shapes=2, base_shape="a"),
+        e(
+            nps.mutually_broadcastable_shapes,  # min_side is invalid type
+            num_shapes=2,
+            base_shape=(2, 2),
+            min_side="a",
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # min_dims is invalid type
+            num_shapes=2,
+            base_shape=(2, 2),
+            min_dims="a",
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # max_side is invalid type
+            num_shapes=2,
+            base_shape=(2, 2),
+            max_side="a",
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # max_side is invalid type
+            num_shapes=2,
+            base_shape=(2, 2),
+            max_dims="a",
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # min_side is out of domain
+            num_shapes=2,
+            base_shape=(2, 2),
+            min_side=-1,
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # min_dims is out of domain
+            num_shapes=2,
+            base_shape=(2, 2),
+            min_dims=-1,
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # min_dims is out of domain
+            num_shapes=2,
+            base_shape=(2, 2),
+            min_dims=33,
+            max_dims=None,
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # max_dims is out of domain
+            num_shapes=2,
+            base_shape=(2, 2),
+            min_dims=1,
+            max_dims=33,
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # max_side < min_side
+            num_shapes=2,
+            base_shape=(2, 2),
+            min_side=1,
+            max_side=0,
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # max_dims < min_dims
+            num_shapes=2,
+            base_shape=(2, 2),
+            min_dims=1,
+            max_dims=0,
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # max_side too small
+            num_shapes=2,
+            base_shape=(5, 1),
+            min_dims=2,
+            max_dims=4,
+            min_side=2,
+            max_side=3,
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # min_side too large
+            num_shapes=2,
+            base_shape=(0, 1),
+            min_dims=2,
+            max_dims=4,
+            min_side=2,
+            max_side=3,
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # user-specified max_dims unsatisfiable
+            num_shapes=1,
+            base_shape=(5, 3, 2, 1),
+            min_dims=3,
+            max_dims=4,
+            min_side=2,
+            max_side=3,
+        ),
+        e(
+            nps.mutually_broadcastable_shapes,  # user-specified max_dims unsatisfiable
+            num_shapes=2,
+            base_shape=(0, 3, 2, 1),
+            min_dims=3,
+            max_dims=4,
+            min_side=2,
+            max_side=3,
+        ),
         e(nps.basic_indices, shape=0),
         e(nps.basic_indices, shape=("1", "2")),
         e(nps.basic_indices, shape=(0, -1)),
@@ -175,3 +278,12 @@ def test_test_basic_indices_kwonly_emulation():
         nps.basic_indices((), 0, 1).validate()
     with pytest.raises(TypeError):
         nps.basic_indices((), __reserved=None).validate()
+
+
+@pytest.mark.parametrize("args", ({}, (1,), dict(num_shapes=1, __reserved=None)))
+def test_test_mutually_broadcastable_shapes_kwonly_emulation(args):
+    with pytest.raises(TypeError):
+        if isinstance(args, dict):
+            nps.mutually_broadcastable_shapes(**args).validate()
+        else:
+            nps.mutually_broadcastable_shapes(*args).validate()
diff --git a/hypothesis-python/tests/numpy/test_gen_data.py b/hypothesis-python/tests/numpy/test_gen_data.py
--- a/hypothesis-python/tests/numpy/test_gen_data.py
+++ b/hypothesis-python/tests/numpy/test_gen_data.py
@@ -547,8 +547,8 @@ def test_minimize_negative_tuple_axes(ndim, data):
 @settings(deadline=None, suppress_health_check=[HealthCheck.too_slow])
 @given(shape=ANY_SHAPE, data=st.data())
 def test_broadcastable_shape_bounds_are_satisfied(shape, data):
-    min_dim = data.draw(st.integers(0, 32), label="min_dims")
-    max_dim = data.draw(st.none() | st.integers(min_dim, 32), label="max_dims")
+    min_dims = data.draw(st.integers(0, 32), label="min_dims")
+    max_dims = data.draw(st.none() | st.integers(min_dims, 32), label="max_dims")
     min_side = data.draw(st.integers(0, 3), label="min_side")
     max_side = data.draw(st.none() | st.integers(min_side, 6), label="max_side")
     try:
@@ -557,8 +557,8 @@ def test_broadcastable_shape_bounds_are_satisfied(shape, data):
                 shape,
                 min_side=min_side,
                 max_side=max_side,
-                min_dims=min_dim,
-                max_dims=max_dim,
+                min_dims=min_dims,
+                max_dims=max_dims,
             ),
             label="bshape",
         )
@@ -566,29 +566,74 @@ def test_broadcastable_shape_bounds_are_satisfied(shape, data):
         assume(False)
         assert False, "unreachable"
 
-    if max_dim is None:
-        max_dim = max(len(shape), min_dim) + 2
+    if max_dims is None:
+        max_dims = max(len(shape), min_dims) + 2
 
     if max_side is None:
-        max_side = max(tuple(shape[::-1][:max_dim]) + (min_side,)) + 2
+        max_side = max(tuple(shape[::-1][:max_dims]) + (min_side,)) + 2
 
     assert isinstance(bshape, tuple) and all(isinstance(s, int) for s in bshape)
-    assert min_dim <= len(bshape) <= max_dim
+    assert min_dims <= len(bshape) <= max_dims
     assert all(min_side <= s <= max_side for s in bshape)
 
 
-def _draw_valid_bounds(data, shape, max_dim, permit_none=True):
-    if max_dim == 0 or not shape:
+@settings(deadline=None)
+@given(num_shapes=st.integers(1, 4), base_shape=ANY_SHAPE, data=st.data())
+def test_mutually_broadcastable_shape_bounds_are_satisfied(
+    num_shapes, base_shape, data
+):
+    min_dims = data.draw(st.integers(0, 32), label="min_dims")
+    max_dims = data.draw(
+        st.one_of(st.none(), st.integers(min_dims, 32)), label="max_dims"
+    )
+    min_side = data.draw(st.integers(0, 3), label="min_side")
+    max_side = data.draw(
+        st.one_of(st.none(), st.integers(min_side, 6)), label="max_side"
+    )
+    try:
+        shapes, result = data.draw(
+            nps.mutually_broadcastable_shapes(
+                num_shapes=num_shapes,
+                base_shape=base_shape,
+                min_side=min_side,
+                max_side=max_side,
+                min_dims=min_dims,
+                max_dims=max_dims,
+            ),
+            label="shapes, result",
+        )
+    except InvalidArgument:
+        assume(False)
+        assert False, "unreachable"
+
+    if max_dims is None:
+        max_dims = max(len(base_shape), min_dims) + 2
+
+    if max_side is None:
+        max_side = max(tuple(base_shape[::-1][:max_dims]) + (min_side,)) + 2
+
+    assert isinstance(shapes, tuple)
+    assert isinstance(result, tuple)
+    assert all(isinstance(s, int) for s in result)
+
+    for bshape in shapes:
+        assert isinstance(bshape, tuple) and all(isinstance(s, int) for s in bshape)
+        assert min_dims <= len(bshape) <= max_dims
+        assert all(min_side <= s <= max_side for s in bshape)
+
+
+def _draw_valid_bounds(data, shape, max_dims, permit_none=True):
+    if max_dims == 0 or not shape:
         return 0, None
 
-    smallest_side = min(shape[::-1][:max_dim])
+    smallest_side = min(shape[::-1][:max_dims])
     min_strat = (
         st.sampled_from([1, smallest_side])
         if smallest_side > 1
         else st.just(smallest_side)
     )
     min_side = data.draw(min_strat, label="min_side")
-    largest_side = max(max(shape[::-1][:max_dim]), min_side)
+    largest_side = max(max(shape[::-1][:max_dims]), min_side)
     if permit_none:
         max_strat = st.one_of(st.none(), st.integers(largest_side, largest_side + 2))
     else:
@@ -655,18 +700,33 @@ def test_broadcastable_shape_has_good_default_values(shape, data):
     _broadcast_shapes(shape, broadcastable_shape)
 
 
+@settings(deadline=None, max_examples=200)
+@given(base_shape=ANY_SHAPE, num_shapes=st.integers(1, 10), data=st.data())
+def test_mutually_broadcastableshapes_has_good_default_values(
+    num_shapes, base_shape, data
+):
+    # This test ensures that default parameters can always produce broadcast-compatible shapes
+    shapes, result = data.draw(
+        nps.mutually_broadcastable_shapes(num_shapes=num_shapes, base_shape=base_shape),
+        label="shapes, result",
+    )
+    assert len(shapes) == num_shapes
+    # raises if shapes are not mutually-compatible
+    assert result == _broadcast_shapes(base_shape, *shapes)
+
+
 @settings(deadline=None)
-@given(min_dim=st.integers(0, 32), shape=ANY_SHAPE, data=st.data())
-def test_broadcastable_shape_can_broadcast(min_dim, shape, data):
-    max_dim = data.draw(st.none() | st.integers(min_dim, 32), label="max_dim")
-    min_side, max_side = _draw_valid_bounds(data, shape, max_dim)
+@given(min_dims=st.integers(0, 32), shape=ANY_SHAPE, data=st.data())
+def test_broadcastable_shape_can_broadcast(min_dims, shape, data):
+    max_dims = data.draw(st.none() | st.integers(min_dims, 32), label="max_dims")
+    min_side, max_side = _draw_valid_bounds(data, shape, max_dims)
     broadcastable_shape = data.draw(
         nps.broadcastable_shapes(
             shape,
             min_side=min_side,
             max_side=max_side,
-            min_dims=min_dim,
-            max_dims=max_dim,
+            min_dims=min_dims,
+            max_dims=max_dims,
         ),
         label="broadcastable_shapes",
     )
@@ -674,19 +734,47 @@ def test_broadcastable_shape_can_broadcast(min_dim, shape, data):
     _broadcast_shapes(shape, broadcastable_shape)
 
 
+@settings(deadline=None)
+@given(
+    num_shapes=st.integers(1, 10),
+    min_dims=st.integers(0, 32),
+    base_shape=ANY_SHAPE,
+    data=st.data(),
+)
+def test_mutually_broadcastable_shape_can_broadcast(
+    num_shapes, min_dims, base_shape, data
+):
+    max_dims = data.draw(st.none() | st.integers(min_dims, 32), label="max_dims")
+    min_side, max_side = _draw_valid_bounds(data, base_shape, max_dims)
+    shapes, result = data.draw(
+        nps.mutually_broadcastable_shapes(
+            num_shapes=num_shapes,
+            base_shape=base_shape,
+            min_side=min_side,
+            max_side=max_side,
+            min_dims=min_dims,
+            max_dims=max_dims,
+        ),
+        label="shapes, result",
+    )
+
+    # error if drawn shapes are not mutually broadcast-compatible
+    assert result == _broadcast_shapes(base_shape, *shapes)
+
+
 @settings(deadline=None, max_examples=10)
-@given(min_dim=st.integers(0, 32), shape=ANY_SHAPE, data=st.data())
-def test_minimize_broadcastable_shape(min_dim, shape, data):
-    # Ensure aligned dimensions of broadcastable shape minimizes to `(1,) * min_dim`
-    max_dim = data.draw(st.none() | st.integers(min_dim, 32), label="max_dim")
-    min_side, max_side = _draw_valid_bounds(data, shape, max_dim, permit_none=False)
+@given(min_dims=st.integers(0, 32), shape=ANY_SHAPE, data=st.data())
+def test_minimize_broadcastable_shape(min_dims, shape, data):
+    # Ensure aligned dimensions of broadcastable shape minimizes to `(1,) * min_dims`
+    max_dims = data.draw(st.none() | st.integers(min_dims, 32), label="max_dims")
+    min_side, max_side = _draw_valid_bounds(data, shape, max_dims, permit_none=False)
     smallest = minimal(
         nps.broadcastable_shapes(
             shape,
             min_side=min_side,
             max_side=max_side,
-            min_dims=min_dim,
-            max_dims=max_dim,
+            min_dims=min_dims,
+            max_dims=max_dims,
         )
     )
     note("(smallest): {}".format(smallest))
@@ -698,15 +786,57 @@ def test_minimize_broadcastable_shape(min_dim, shape, data):
     assert tuple(expected) == smallest
 
 
+@settings(deadline=None, max_examples=50)
+@given(
+    num_shapes=st.integers(1, 3),
+    min_dims=st.integers(0, 5),
+    base_shape=nps.array_shapes(min_dims=0, max_dims=3, min_side=0, max_side=5),
+    data=st.data(),
+)
+def test_minimize_mutually_broadcastable_shape(num_shapes, min_dims, base_shape, data):
+    # Ensure aligned dimensions of broadcastable shape minimizes to `(1,) * min_dims`
+    max_dims = data.draw(st.none() | st.integers(min_dims, 5), label="max_dims")
+    min_side, max_side = _draw_valid_bounds(
+        data, base_shape, max_dims, permit_none=False
+    )
+
+    if num_shapes > 1:
+        # shrinking gets a little bit hairy when we have empty axes
+        # and multiple num_shapes
+        assume(min_side > 0)
+    note("(min_side, max_side): {}".format((min_side, max_side)))
+    smallest_shapes, result = minimal(
+        nps.mutually_broadcastable_shapes(
+            num_shapes=num_shapes,
+            base_shape=base_shape,
+            min_side=min_side,
+            max_side=max_side,
+            min_dims=min_dims,
+            max_dims=max_dims,
+        )
+    )
+    note("(smallest_shapes, result): {}".format((smallest_shapes, result)))
+    assert len(smallest_shapes) == num_shapes
+    assert result == _broadcast_shapes(base_shape, *smallest_shapes)
+    for smallest in smallest_shapes:
+        n_leading = max(len(smallest) - len(base_shape), 0)
+        n_aligned = max(len(smallest) - n_leading, 0)
+        expected = [min_side] * n_leading + [
+            1 if min_side <= 1 <= max_side else i
+            for i in base_shape[len(base_shape) - n_aligned :]
+        ]
+        assert tuple(expected) == smallest
+
+
 @settings(deadline=None)
-@given(max_dim=st.integers(4, 6), data=st.data())
-def test_broadcastable_shape_adjusts_max_dim_with_explicit_bounds(max_dim, data):
+@given(max_dims=st.integers(4, 6), data=st.data())
+def test_broadcastable_shape_adjusts_max_dim_with_explicit_bounds(max_dims, data):
     # Ensures that `broadcastable_shapes` limits itself to satisfiable dimensions
     # Broadcastable values can only be drawn for dims 0-3 for these shapes
     shape = data.draw(st.sampled_from([(5, 3, 2, 1), (0, 3, 2, 1)]), label="shape")
     broadcastable_shape = data.draw(
         nps.broadcastable_shapes(
-            shape, min_side=2, max_side=3, min_dims=3, max_dims=max_dim
+            shape, min_side=2, max_side=3, min_dims=3, max_dims=max_dims
         ),
         label="broadcastable_shapes",
     )
@@ -715,12 +845,54 @@ def test_broadcastable_shape_adjusts_max_dim_with_explicit_bounds(max_dim, data)
     _broadcast_shapes(shape, broadcastable_shape)
 
 
+@settings(deadline=None)
+@given(
+    max_side=st.sampled_from([3, None]),
+    min_dims=st.integers(0, 4),
+    num_shapes=st.integers(1, 3),
+    data=st.data(),
+)
+def test_mutually_broadcastable_shape_adjusts_max_dim_with_default_bounds(
+    max_side, min_dims, num_shapes, data
+):
+    # Ensures that `mutually_broadcastable_shapes` limits itself to satisfiable dimensions
+    # when a default `max_dims` is derived.
+    base_shape = data.draw(
+        st.sampled_from([(5, 3, 2, 1), (0, 3, 2, 1)]), label="base_shape"
+    )
+
+    try:
+        shapes, result = data.draw(
+            nps.mutually_broadcastable_shapes(
+                num_shapes=num_shapes,
+                base_shape=base_shape,
+                min_side=2,
+                max_side=max_side,
+                min_dims=min_dims,
+            ),
+            label="shapes, result",
+        )
+    except InvalidArgument:
+        # There is no satisfiable `max_dims` for us to tune
+        assert min_dims == 4 and (max_side == 3 or base_shape[0] == 0)
+        return
+
+    if max_side == 3 or base_shape[0] == 0:
+        assert all(len(s) <= 3 for s in shapes)
+    elif min_dims == 4:
+        assert all(4 <= len(s) for s in shapes)
+
+    # error if drawn shape for b is not broadcast-compatible
+    assert len(shapes) == num_shapes
+    assert result == _broadcast_shapes(base_shape, *shapes)
+
+
 @settings(deadline=None, max_examples=10)
-@given(min_dim=st.integers(0, 32), min_side=st.integers(2, 3), data=st.data())
+@given(min_dims=st.integers(0, 32), min_side=st.integers(2, 3), data=st.data())
 def test_broadcastable_shape_shrinking_with_singleton_out_of_bounds(
-    min_dim, min_side, data
+    min_dims, min_side, data
 ):
-    max_dim = data.draw(st.none() | st.integers(min_dim, 32), label="max_dim")
+    max_dims = data.draw(st.none() | st.integers(min_dims, 32), label="max_dims")
     max_side = data.draw(st.none() | st.integers(min_side, 6), label="max_side")
     shape = data.draw(st.integers(1, 4).map(lambda n: n * (1,)), label="shape")
     smallest = minimal(
@@ -728,11 +900,79 @@ def test_broadcastable_shape_shrinking_with_singleton_out_of_bounds(
             shape,
             min_side=min_side,
             max_side=max_side,
-            min_dims=min_dim,
-            max_dims=max_dim,
+            min_dims=min_dims,
+            max_dims=max_dims,
+        )
+    )
+    assert smallest == (min_side,) * min_dims
+
+
+@settings(deadline=None, max_examples=50)
+@given(
+    num_shapes=st.integers(1, 4),
+    min_dims=st.integers(0, 4),
+    min_side=st.integers(2, 3),
+    data=st.data(),
+)
+def test_mutually_broadcastable_shapes_shrinking_with_singleton_out_of_bounds(
+    num_shapes, min_dims, min_side, data
+):
+    """Ensures that shapes minimize to `(min_side,) * min_dims` when singleton dimensions
+    are disallowed."""
+    max_dims = data.draw(st.none() | st.integers(min_dims, 4), label="max_dims")
+    max_side = data.draw(
+        st.one_of(st.none(), st.integers(min_side, 6)), label="max_side"
+    )
+    ndims = data.draw(st.integers(1, 4), label="ndim")
+    base_shape = (1,) * ndims
+    smallest_shapes, result = minimal(
+        nps.mutually_broadcastable_shapes(
+            num_shapes=num_shapes,
+            base_shape=base_shape,
+            min_side=min_side,
+            max_side=max_side,
+            min_dims=min_dims,
+            max_dims=max_dims,
         )
     )
-    assert smallest == (min_side,) * min_dim
+    note("(smallest_shapes, result): {}".format((smallest_shapes, result)))
+    assert len(smallest_shapes) == num_shapes
+    assert result == _broadcast_shapes(base_shape, *smallest_shapes)
+    for smallest in smallest_shapes:
+        assert smallest == (min_side,) * min_dims
+
+
+@given(
+    num_shapes=st.integers(1, 4),
+    min_dims=st.integers(1, 32),
+    max_side=st.integers(1, 6),
+    data=st.data(),
+)
+def test_mutually_broadcastable_shapes_only_singleton_is_valid(
+    num_shapes, min_dims, max_side, data
+):
+    """Ensures that, when all aligned base-shape dim sizes are larger
+    than ``max_side``, only singletons can be drawn"""
+    max_dims = data.draw(st.integers(min_dims, 32), label="max_dims")
+    base_shape = data.draw(
+        nps.array_shapes(min_side=max_side + 1, min_dims=1), label="base_shape"
+    )
+    input_shapes, result = data.draw(
+        nps.mutually_broadcastable_shapes(
+            num_shapes=num_shapes,
+            base_shape=base_shape,
+            min_side=1,
+            max_side=max_side,
+            min_dims=min_dims,
+            max_dims=max_dims,
+        ),
+        label="input_shapes, result",
+    )
+
+    assert len(input_shapes) == num_shapes
+    assert result == _broadcast_shapes(base_shape, *input_shapes)
+    for shape in input_shapes:
+        assert all([i == 1 for i in shape[-len(base_shape) :]])
 
 
 @settings(deadline=None)
@@ -757,6 +997,40 @@ def test_broadcastable_shape_can_generate_arbitrary_ndims(shape, max_dims, data)
     )
 
 
+@settings(deadline=None)
+@given(
+    num_shapes=st.integers(1, 3),
+    base_shape=nps.array_shapes(min_dims=0, max_dims=3, min_side=0, max_side=5),
+    max_dims=st.integers(0, 4),
+    data=st.data(),
+)
+def test_mutually_broadcastable_shapes_can_generate_arbitrary_ndims(
+    num_shapes, base_shape, max_dims, data
+):
+    # ensures that each generated shape can possess any length in [min_dims, max_dims]
+    desired_ndims = data.draw(
+        st.lists(st.integers(0, max_dims), min_size=num_shapes, max_size=num_shapes),
+        label="desired_ndims",
+    )
+    min_dims = data.draw(
+        st.one_of(st.none(), st.integers(0, min(desired_ndims))), label="min_dims"
+    )
+    args = (
+        dict(min_dims=min_dims) if min_dims is not None else {}
+    )  # check default arg behavior too
+    find_any(
+        nps.mutually_broadcastable_shapes(
+            num_shapes=num_shapes,
+            base_shape=base_shape,
+            min_side=0,
+            max_dims=max_dims,
+            **args
+        ),
+        lambda x: {len(s) for s in x.input_shapes} == set(desired_ndims),
+        settings(max_examples=10 ** 6),
+    )
+
+
 @settings(deadline=None)
 @given(
     shape=nps.array_shapes(min_dims=1, min_side=1),