Repository: HypothesisWorks/hypothesis

Issue:
Adding an iterable strategy
Hey there! First of all, thanks for the talk at FB London today. Re: the question I asked about iterables, this is the kind of examples I was thinking of that I said I see a lot (untested, but you get the idea):

``` python
def always_the_same(elements):
    '''Check whether every element in an iterable is the same'''
    if not elements:
        return True
    return all(elements[0] is x for x in elements)
```

In this case, we might have an unindexable type of iterable, so even though this usually works since most people will probably just pass in a list or tuple, it breaks on generators/other non-indexable things. People might still use Hypothesis just to generate a bunch of lists or tuples in this case thinking it is good enough, but if there was a distinct "iterable" class, it could be used except in cases where an indexable iterable was _really_ necessary.

``` python
def yield_twice(elements):
    for item in elements:
        yield item
    for item in elements:
        yield item
```

This is a particularly dumb example, but you get the idea. Mostly I see this when multiple for loops may be used to iterate over something, and the person writing the function didn't consider generator exhaustion.

I think it's worth having an "iterable" strategy just because I think that it's quite possible that when many people say some function "takes an iterable", they may not have considered the consequences of iterables that don't allow indexing, or can become exhausted (just like every Python programmer, I make this assumption without realising it every now and then).

I may take a look at how the scenario generator works and put some code to go with this request, assuming, of course, that my proposal makes sense to you and you think it adds value.

Thanks!



Discussion:
Ah, I see, I totally misunderstood what you meant by generator exhaustion, sorry!

This seems like a reasonable thing to have a strategy for. If you want to give it a try I'd be happy to give some guidance.

The approach I would likely take with something like this is that rather than writing a custom strategy I'd just write it as a union of strategies built up from existing ones using the normal combinators. I'd recommend starting there before descending into the guts of how strategies work.

Isn't this, in its simplest form, `def iterable(*a, **kw): return st.lists(*a, **kw).map(iter)`? This returns an iterator that is also an iterable (`iter` appears to be idempotent). It doesn't support indexing and stays exhausted, no matter how often you iterate over it.

```
>>> i = iter([1,2,3])
>>> next(i)
1
>>> next(i)
2
>>> next(i)
3
>>> next(i)
StopIteration
>>> i = iter(i)
>>> next(i)
StopIteration
>>> i[0]
TypeError
```


Resolving patch:
diff --git a/src/hypothesis/internal/compat.py b/src/hypothesis/internal/compat.py
--- a/src/hypothesis/internal/compat.py
+++ b/src/hypothesis/internal/compat.py
@@ -474,6 +474,13 @@ def get_stream_enc(stream, default=None):
     return getattr(stream, 'encoding', None) or default
 
 
+def implements_iterator(it):
+    """Turn things with a __next__ attribute into iterators on Python 2."""
+    if PY2 and not hasattr(it, 'next') and hasattr(it, '__next__'):
+        it.next = it.__next__
+    return it
+
+
 if PY3:
     FileNotFoundError = FileNotFoundError
 else:
diff --git a/src/hypothesis/strategies.py b/src/hypothesis/strategies.py
--- a/src/hypothesis/strategies.py
+++ b/src/hypothesis/strategies.py
@@ -26,7 +26,7 @@
 from hypothesis.control import assume
 from hypothesis.searchstrategy import SearchStrategy
 from hypothesis.internal.compat import ArgSpec, hrange, text_type, \
-    getargspec, integer_types, float_to_decimal
+    getargspec, integer_types, float_to_decimal, implements_iterator
 from hypothesis.internal.floats import is_negative, float_to_int, \
     int_to_float, count_between_floats
 from hypothesis.utils.conventions import not_set
@@ -41,7 +41,7 @@
     'booleans', 'integers', 'floats', 'complex_numbers', 'fractions',
     'decimals',
     'characters', 'text', 'binary', 'uuids',
-    'tuples', 'lists', 'sets', 'frozensets',
+    'tuples', 'lists', 'sets', 'frozensets', 'iterables',
     'dictionaries', 'fixed_dictionaries',
     'sampled_from', 'permutations',
     'builds',
@@ -541,6 +541,38 @@ def frozensets(elements=None, min_size=None, average_size=None, max_size=None):
     ).map(frozenset)
 
 
+@defines_strategy
+def iterables(elements=None, min_size=None, average_size=None, max_size=None,
+              unique_by=None, unique=False):
+    """This has the same behaviour as lists, but returns iterables instead.
+
+    Some iterables cannot be indexed (e.g. sets) and some do not have a
+    fixed length (e.g. generators). This strategy produces iterators,
+    which cannot be indexed and do not have a fixed length. This ensures
+    that you do not accidentally depend on sequence behaviour.
+
+    """
+    @implements_iterator
+    class PrettyIter(object):
+        def __init__(self, values):
+            self._values = values
+            self._iter = iter(self._values)
+
+        def __iter__(self):
+            return self._iter
+
+        def __next__(self):
+            return next(self._iter)
+
+        def __repr__(self):
+            return 'iter({!r})'.format(self._values)
+
+    return lists(
+        elements=elements, min_size=min_size, average_size=average_size,
+        max_size=max_size, unique_by=unique_by, unique=unique
+    ).map(PrettyIter)
+
+
 @defines_strategy
 def fixed_dictionaries(mapping):
     """Generate a dictionary of the same type as mapping with a fixed set of
diff --git a/src/hypothesis/version.py b/src/hypothesis/version.py
--- a/src/hypothesis/version.py
+++ b/src/hypothesis/version.py
@@ -17,5 +17,5 @@
 
 from __future__ import division, print_function, absolute_import
 
-__version_info__ = (3, 7, 4)
+__version_info__ = (3, 8, 0)
 __version__ = '.'.join(map(str, __version_info__))


Test patch:
diff --git a/tests/cover/test_custom_reprs.py b/tests/cover/test_custom_reprs.py
--- a/tests/cover/test_custom_reprs.py
+++ b/tests/cover/test_custom_reprs.py
@@ -20,6 +20,7 @@
 import pytest
 
 import hypothesis.strategies as st
+from hypothesis import given
 
 
 def test_includes_non_default_args_in_repr():
@@ -57,3 +58,9 @@ def test_errors_are_deferred_until_repr_is_calculated():
 
     with pytest.raises(ValueError):
         repr(s)
+
+
+@given(st.iterables(st.integers()))
+def test_iterables_repr_is_useful(it):
+    # fairly hard-coded but useful; also ensures _values are inexhaustible
+    assert repr(it) == 'iter({!r})'.format(it._values)
diff --git a/tests/cover/test_direct_strategies.py b/tests/cover/test_direct_strategies.py
--- a/tests/cover/test_direct_strategies.py
+++ b/tests/cover/test_direct_strategies.py
@@ -280,3 +280,15 @@ def test_none_lists_respect_max_size(ls):
 )
 def test_none_lists_respect_max_and_min_size(ls):
     assert 1 <= len(ls) <= 5
+
+
+@given(ds.iterables(ds.integers(), max_size=5, min_size=1))
+def test_iterables_are_exhaustible(it):
+    for _ in it:
+        pass
+    with pytest.raises(StopIteration):
+        next(it)
+
+
+def test_minimal_iterable():
+    assert list(find(ds.iterables(ds.integers()), lambda x: True)) == []