Repository: HypothesisWorks/hypothesis

Issue:
`st.functions()` returning consistent (or pure) functions
Currently, `st.functions()` creates functions that are not-pure (in a sence that they don't return the same value for the same arguments).

My use-case is that I am testing monad laws (see https://github.com/dry-python/returns/issues/533 and https://github.com/dry-python/returns/pull/543 for more information). Here's one of the definitions:

```python
def associativity_law(
    mappable: MappableN[_FirstType, _SecondType, _ThirdType],
    first: Callable[[_FirstType], _NewType1],
    second: Callable[[_NewType1], _NewType2],
) -> None:
    assert mappable.map(first).map(second) == mappable.map(
        compose(first, second),
    )
``` 

And it never passes! Because each time `first(x)` and `second(y)` returns a different result.

My work-around for now is this custom strategy:

```python
_func_template = """
lambda {0}: ...
"""

def functions(data: st.DataObject):
        def factory(typ):
            *arg_types, return_type = typ.__args__
            return_value = data.draw(st.from_type(return_type))
            args = []
            for i, arg in enumerate(arg_types):
                args.append('arg{0}'.format(i + 1))

            func = eval(_func_template.format(
                ', '.join(args),
                # return_type.__name__,
            ))
            
            return st.functions(
                like=func,
                returns=st.just(return_value),
            )
        return factory
```

And then I register this callback to handle `Callable`, like so: `st.register_type_strategy(Callable, custom_strategies.functions(data))`

So, is there any chance to see pure functions in the core of `st.functions()`?


Discussion:
Generating pure functions is a little tricky - I think in general we could only support it based on hashing the arguments, and mutable return values wouldn't be regenerated.  If you think it's useful though, there's a demo at https://github.com/HypothesisWorks/hypothesis/compare/master...Zac-HD:pure-functions and it seems to work.  Needs much more testing before it could be merged though :sweat_smile: 
@Zac-HD is there anything I can help you with?
I guess `returns` is a great place to test pure function generation! 

P.S. does my solution look viable for now? 
Actually, yes - if you want to copy that branch and add tests + release notes, I'd be happy to merge it.  All the tests can go in `hypothesis-python/tests/cover/test_functions.py`:

- raises InvalidArgument if `pure` is not a bool
- with `pure=True`, identical arguments -> same returned value
- can generate non-identical return value for non-identical arguments
- different generated functions `like` the same base function can generate separate return values for same args
- passing kwargs in different orders, or an arg positionally vs by keyword, does not affect the caching (unlike `lru_cache()`)

Your workaround looks OK to me, with the caveats that I think it fails for `Callable` or `Callable[..., X]` on some Python versions (maybe only 3.5?); and that it only generates a single example ever rather than maybe varying if arguments vary.
I am on it! 👍 

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/strategies/_internal/core.py b/hypothesis-python/src/hypothesis/strategies/_internal/core.py
--- a/hypothesis-python/src/hypothesis/strategies/_internal/core.py
+++ b/hypothesis-python/src/hypothesis/strategies/_internal/core.py
@@ -2158,7 +2158,10 @@ def emails() -> SearchStrategy[str]:
 @defines_strategy
 @deprecated_posargs
 def functions(
-    *, like: Callable[..., Any] = lambda: None, returns: SearchStrategy[Any] = None
+    *,
+    like: Callable[..., Any] = lambda: None,
+    returns: SearchStrategy[Any] = None,
+    pure: bool = False
 ) -> SearchStrategy[Callable[..., Any]]:
     # The proper type signature of `functions()` would have T instead of Any, but mypy
     # disallows default args for generics: https://github.com/python/mypy/issues/3737
@@ -2171,12 +2174,17 @@ def functions(
     for the function is drawn from the ``returns`` argument, which must be a
     strategy.
 
-    Note that the generated functions do not validate their arguments, and
+    If ``pure=True``, all arguments passed to the generated function must be
+    hashable, and if passed identical arguments the original return value will
+    be returned again - *not* regenerated, so beware mutable values.
+
+    If ``pure=False``, generated functions do not validate their arguments, and
     may return a different value if called again with the same arguments.
 
     Generated functions can only be called within the scope of the ``@given``
     which created them.  This strategy does not support ``.example()``.
     """
+    check_type(bool, pure, "pure")
     if not callable(like):
         raise InvalidArgument(
             "The first argument to functions() must be a callable to imitate, "
@@ -2188,7 +2196,7 @@ def functions(
         returns = from_type(hints.get("return", type(None)))
 
     check_strategy(returns, "returns")
-    return FunctionStrategy(like, returns)
+    return FunctionStrategy(like, returns, pure)
 
 
 @composite
diff --git a/hypothesis-python/src/hypothesis/strategies/_internal/functions.py b/hypothesis-python/src/hypothesis/strategies/_internal/functions.py
--- a/hypothesis-python/src/hypothesis/strategies/_internal/functions.py
+++ b/hypothesis-python/src/hypothesis/strategies/_internal/functions.py
@@ -15,17 +15,24 @@
 
 from hypothesis.control import note
 from hypothesis.errors import InvalidState
-from hypothesis.internal.reflection import arg_string, nicerepr, proxies
+from hypothesis.internal.reflection import (
+    arg_string,
+    convert_positional_arguments,
+    nicerepr,
+    proxies,
+)
+from hypothesis.strategies._internal.shared import SharedStrategy
 from hypothesis.strategies._internal.strategies import SearchStrategy
 
 
 class FunctionStrategy(SearchStrategy):
     supports_find = False
 
-    def __init__(self, like, returns):
+    def __init__(self, like, returns, pure):
         super().__init__()
         self.like = like
         self.returns = returns
+        self.pure = pure
 
     def calc_is_empty(self, recur):
         return recur(self.returns)
@@ -38,7 +45,12 @@ def inner(*args, **kwargs):
                     "This generated %s function can only be called within the "
                     "scope of the @given that created it." % (nicerepr(self.like),)
                 )
-            val = data.draw(self.returns)
+            if self.pure:
+                args, kwargs = convert_positional_arguments(self.like, args, kwargs)
+                key = (inner, args, frozenset(kwargs.items()))
+                val = data.draw(SharedStrategy(base=self.returns, key=key))
+            else:
+                val = data.draw(self.returns)
             note(
                 "Called function: %s(%s) -> %r"
                 % (nicerepr(self.like), arg_string(self.like, args, kwargs), val)


Test patch:
diff --git a/hypothesis-python/tests/cover/test_functions.py b/hypothesis-python/tests/cover/test_functions.py
--- a/hypothesis-python/tests/cover/test_functions.py
+++ b/hypothesis-python/tests/cover/test_functions.py
@@ -19,7 +19,7 @@
 
 from hypothesis import assume, given
 from hypothesis.errors import InvalidArgument, InvalidState
-from hypothesis.strategies import booleans, functions
+from hypothesis.strategies import booleans, functions, integers
 
 
 def func_a():
@@ -76,11 +76,16 @@ def test_functions_lambda_with_arg(f):
 
 
 @pytest.mark.parametrize(
-    "like,returns", [(None, booleans()), (lambda: None, "not a strategy")]
+    "like,returns,pure",
+    [
+        (None, booleans(), False),
+        (lambda: None, "not a strategy", True),
+        (lambda: None, booleans(), None),
+    ],
 )
-def test_invalid_arguments(like, returns):
+def test_invalid_arguments(like, returns, pure):
     with pytest.raises(InvalidArgument):
-        functions(like=like, returns=returns).example()
+        functions(like=like, returns=returns, pure=pure).example()
 
 
 def func_returns_str() -> str:
@@ -125,3 +130,57 @@ def test_functions_strategy_with_kwonly_args(f):
         f(1, 2)
     f(1, kwonly_arg=2)
     f(kwonly_arg=2, arg=1)
+
+
+def pure_func(arg1, arg2):
+    pass
+
+
+@given(
+    f=functions(like=pure_func, returns=integers(), pure=True),
+    arg1=integers(),
+    arg2=integers(),
+)
+def test_functions_pure_with_same_args(f, arg1, arg2):
+    # Same regardless of calling convention, unlike functools.lru_cache()
+    expected = f(arg1, arg2)
+    assert f(arg1, arg2) == expected
+    assert f(arg1, arg2=arg2) == expected
+    assert f(arg1=arg1, arg2=arg2) == expected
+    assert f(arg2=arg2, arg1=arg1) == expected
+
+
+@given(
+    f=functions(like=pure_func, returns=integers(), pure=True),
+    arg1=integers(),
+    arg2=integers(),
+)
+def test_functions_pure_with_different_args(f, arg1, arg2):
+    r1 = f(arg1, arg2)
+    r2 = f(arg2, arg1)
+    assume(r1 != r2)
+    # If this is never true, the test will fail with Unsatisfiable
+
+
+@given(
+    f1=functions(like=pure_func, returns=integers(), pure=True),
+    f2=functions(like=pure_func, returns=integers(), pure=True),
+)
+def test_functions_pure_two_functions_different_args_different_result(f1, f2):
+    r1 = f1(1, 2)
+    r2 = f2(3, 4)
+    assume(r1 != r2)
+    # If this is never true, the test will fail with Unsatisfiable
+
+
+@given(
+    f1=functions(like=pure_func, returns=integers(), pure=True),
+    f2=functions(like=pure_func, returns=integers(), pure=True),
+    arg1=integers(),
+    arg2=integers(),
+)
+def test_functions_pure_two_functions_same_args_different_result(f1, f2, arg1, arg2):
+    r1 = f1(arg1, arg2)
+    r2 = f2(arg1, arg2)
+    assume(r1 != r2)
+    # If this is never true, the test will fail with Unsatisfiable