Repository: HypothesisWorks/hypothesis

Issue:
Argument copying technique in LazyStrategy does not preserve custom list subtypes
I encountered an unexpected behavior when attempting to test a third-party library which makes use of custom `list` subtypes. Here's an example which reproduces the behavior:

Consider a class which inherits from the built-in Python `list` type and adds additional methods:

```python
class MyList(list):
    def custom_method(self):
        return "result"
    
    def __copy__(self):
        return MyList(list(self))
    
    def __deepcopy__(self, table):
        return self.__copy__()
```

Because I've overridden both `__copy__` and `__deepcopy__`, copying this list preserves its type:

```python
>>> from copy import copy, deepcopy
>>> my_list = MyList([])
>>> type(copy(my_list))
<class '__main__.MyList'>
>>> type(deepcopy(my_list))
<class '__main__.MyList'>
```

Let's say that I want to have a strategy which is parameterized by an instance of this class. This works as expected for some strategies:

```python
>>> from hypothesis import strategies as st
>>> type(st.just(my_list).example())
<class '__main__.MyList'>
```

However, I noticed that the argument type is not preserved when I use a composite strategy:

```python
>>> @st.composite
... def my_strategy(draw, my_list):
...     return my_list
...
>>> type(my_strategy(my_list).example())
<type 'tuple'>
```

I believe that this behavior is due to how arguments are copied in `LazyStrategy`: 

https://github.com/HypothesisWorks/hypothesis-python/blob/3ee500943938d60a8a97b7d3d948522d65f23e84/src/hypothesis/searchstrategy/lazy.py#L88

Each argument is being copied by `tupelize()`, which is defined as

```python
def tupleize(x):
    if isinstance(x, (tuple, list)):
        return tuple(x)
    else:
        return x
```

I'm not sure whether it would be safe to replace `tupelize` with `copy` here: converting lists to tuples here guards against mutation from both code inside of the composite strategy's body as well as the code which calls / constructs the strategy, so safely using `copy` here might also require additional `copy` calls elsewhere to guard against mutation during/after invocation.

I'm able to work around this behavior by wrapping my argument in an outer list (e.g. `[my_list]`) and unpacking the argument in my composite strategy.

I'm therefore not blocked by this behavior but I found it confusing and figured it might be worth reporting / documenting.


Discussion:
Thanks for the report Josh - it's fantastic to get so much detail.

I'll need to have a closer look when I have more time, but I *think* this is working as intended to avoid the mutable-default-argument problem - having arguments mutate between examples would break example shrinking, along with other things.  If that's the case we should probably disallow mutable arguments instead of changing them though!

Can you show a larger example of where this has come up?  I think a concrete example would help me understand what you want to do 😄 
No I don't think this is intended behaviour. I can't remember what the argument handling logic in lazy strategy does exactly, but as far as I remember it's meant mostly as an optimization with the intention of unwrapping nested strategies. It shouldn't have user visible behaviour other than that 
For more context on how this somewhat convoluted-looking example cropped up: I am in the process of incrementally porting a legacy third-party domain-specific randomized testing harness to use Hypothesis for test case data generation so that I can benefit from Hypothesis's example database and example shrinking (the port is going well so far and has already uncovered a new bug in the system under test).

The existing test framework being ported has several internal data structures which have this pattern where they extend from `list` to add several list filtering methods (if this existing library was written in Scala then I suspect they would have used implicits to factor out these common list filtering methods and would just pass around immutable Seqs). A slightly more concrete example is something like this (in psuedocode):

```python
@st.composite
def build_typed_thing(draw, types_list):
    // [...]
    condition = ...
    type_of_thing = draw(st.sampled_from(types_list.valid_types(condition)))
    // [... other generator calls ...]
    return thing(type_of_thing, ...)
```

where `types_list` is some custom `list` subtype whose sole purpose is to add the `.valid_types(condition)` filtering method.

I'm not actually a fan of this pattern and my planned workaround is to refactor these wrapper classes so that they don't directly extend from `list` (I'm pretty sure they can simply _contain_ a list and can simply extend the `collections.Sequence` abstract base class).

I'm reporting this mostly just to aid in searchability (keywords: `tuple`, `copy`, `type`) in case anyone else hits the same issue because this was slightly tricky to debug.

Regarding the mutable-default-argument problem: `tupelize` helps here, but it's only guarding against mutation of `list`s. For more general safety, it seems like you'd also want to `copy` other mutable arguments here.
Digging into the history of `lazy.py`, it looks like this is the remnant of a Hypothesis-1-era attempt to fix broken reprs of elements of tuples in argument signatures.  I think we can therefore just delete it!

Resolving patch:
diff --git a/src/hypothesis/searchstrategy/lazy.py b/src/hypothesis/searchstrategy/lazy.py
--- a/src/hypothesis/searchstrategy/lazy.py
+++ b/src/hypothesis/searchstrategy/lazy.py
@@ -22,14 +22,6 @@
     convert_keyword_arguments, convert_positional_arguments
 from hypothesis.searchstrategy.strategies import SearchStrategy
 
-
-def tupleize(x):
-    if isinstance(x, (tuple, list)):
-        return tuple(x)
-    else:
-        return x
-
-
 unwrap_cache = {}
 unwrap_depth = 0
 
@@ -85,10 +77,8 @@ def __init__(self, function, args, kwargs):
         self.__wrapped_strategy = None
         self.__representation = None
         self.__function = function
-        self.__args = tuple(map(tupleize, args))
-        self.__kwargs = dict(
-            (k, tupleize(v)) for k, v in kwargs.items()
-        )
+        self.__args = args
+        self.__kwargs = kwargs
 
     @property
     def supports_find(self):


Test patch:
diff --git a/tests/cover/test_composite.py b/tests/cover/test_composite.py
--- a/tests/cover/test_composite.py
+++ b/tests/cover/test_composite.py
@@ -129,3 +129,18 @@ def is_square(m):
     assert len(value) == 2
     assert len(value[0]) == 2
     assert sorted(value[0] + value[1]) == [0, 0, 0, 1]
+
+
+class MyList(list):
+    pass
+
+
+@given(st.data(), st.lists(st.integers()).map(MyList))
+def test_does_not_change_arguments(data, ls):
+    # regression test for issue #1017 or other argument mutation
+    @st.composite
+    def strat(draw, arg):
+        return arg
+
+    ex = data.draw(strat(ls))
+    assert ex is ls
diff --git a/tests/cover/test_custom_reprs.py b/tests/cover/test_custom_reprs.py
--- a/tests/cover/test_custom_reprs.py
+++ b/tests/cover/test_custom_reprs.py
@@ -37,8 +37,8 @@ def test_supports_positional_and_keyword_args_in_builds():
         'builds(hi, integers(), there=booleans())'
 
 
-def test_includes_a_trailing_comma_in_single_element_sampling():
-    assert repr(st.sampled_from([0])) == 'sampled_from((0,))'
+def test_preserves_sequence_type_of_argument():
+    assert repr(st.sampled_from([0])) == 'sampled_from([0])'
 
 
 class IHaveABadRepr(object):