Repository: HypothesisWorks/hypothesis

Issue:
Check that the alphabet argument to text() is a sequence of unicode characters
The `alphabet` argument to [`text()`](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.text) may be None, a strategy for generating unicode characters (typically the `characters()` strategy), or a sequence of unicode characters (possibly provided as a string).

```pycon
>>> # This currently "works", and that's terrible.
>>> text(alphabet={1, lambda: None, ('ab', 'cd')}).example()
"('ab', 'cd')('ab', 'cd')111<function <lambda> at 0x00000073F8EE0730>"
```

However, there is very little validation of the sequence-of-characters form.  We should deprecate use of non-sequence collections (as for `sampled_from`, hash randomisation breaks reproducibility), deprecate non-length-one elements (ie `alphabet=['a', 'bc']` is bad), and error on non-string types instead of coercing them to text.  Python2 `str` should emit a deprecation warning, since this is probably a common misuse and usually works.

A fixed implementation might look like

```python
else:
    if not isinstance(alphabet, Sequence):
        note_deprecation(...)
        alphabet = list(alphabet)
    # Warning rather than error on Py2-str for now, to avoid a breaking change
    non_string = [c for c in alphabet if not isinstance(c, string_types)]
    if non_string:
        raise InvalidArgument(...)
    if PY2:
        non_unicode = [c for c in alphabet if not isinstance(c, text_type)]
        ...  # using note_deprecation
    not_one_char = [...]; if not_one_char: note_deprecation(...)
    char_strategy = sampled_from(alphabet)
```

Note that all error messages should show the repr of each 'bad' value, explain why it is bad, and usually suggest what the user should do instead.  Filling that out, plus tests, would be a fantastic contribution.

If you'd like to work on this, let me know - I can lend a hand and also ensure that only one person works on any issue at a given time :smile: 


Discussion:
@Zac-HD I would like to work on this issue. I guess I understand most of the things, except 
"non-length-one elements (ie alphabet=['a', 'bc'] is bad),"
how ['a', 'bc'] is non-length elements?
Could you pls explain more.
Because `'bc'` is a string of length two, rather than a single unicode character - and therefore it can violate the property that the length of a generated string will always be within the specified bounds.  Eg:

```pycon
>>> text(['abc'], min_size=2, max_size=2).example()
'abcabc'
>>> len(_)
6
```
@Zac-HD got it, so basically alphabet must be sequence of single characters.
@Zac-HD 
I was working on this and got some queries regarding this issue:
```python
if not isinstance(alphabet, Sequence):
        note_deprecation(...)
        alphabet = list(alphabet)
```
If I assume that above check is for catching things like like 
alphabet=None or alphabet=123, alphabet=True etc.
Then, alphabet = list(alphabet) is gonna give TypeError.
So if someone is passing non iterable, shouldn't we first raise InvalidArgument instead of note_deprecation? 

I have more or less created test cases for all scenario, except 
```python
 if PY2:
            non_unicode = [c for c in alphabet if not isinstance(c, text_type)]
            if non_unicode:
                note_deprecation
```
Shall I have conditional test case for PY2?
P.S: I could not find note_deprecation test cases in test/*, am I missing something?
The main purpose is actually to catch unordered collections, like `alphabet=set('abc')` - the problem being that because iteration order for e.g. sets is random, Hypothesis won't be able to reproduce its results or even shrink to the same value next time.  Note: the conversion to list should be *outside* the conditional block.  We'll then get a warning *and* the TypeError if you pass a non-collection, but that's OK as it's never worked.

You can put a Python2-specific test in `tests/py2/` - this is almost always better than a conditional test.

To test a deprecated thing, write a test that passes but ignores the deprecation, then add the `@checks_deprecated_behaviour` decorator.  The `note_deprecation` function only issues deprecations, so it's quite useless in tests (though short and worth reading).  `with pytest.warns(HypothesisDeprecationWarning)` might help though :smile:

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/strategies.py b/hypothesis-python/src/hypothesis/strategies.py
--- a/hypothesis-python/src/hypothesis/strategies.py
+++ b/hypothesis-python/src/hypothesis/strategies.py
@@ -28,6 +28,7 @@
 from inspect import isclass, isfunction
 from fractions import Fraction
 from functools import reduce
+from collections import Sequence
 
 import attr
 
@@ -37,7 +38,7 @@
 from hypothesis.internal.cache import LRUReusedCache
 from hypothesis.searchstrategy import SearchStrategy, check_strategy
 from hypothesis.internal.compat import gcd, ceil, floor, hrange, \
-    text_type, get_type_hints, getfullargspec, implements_iterator
+    string_types, get_type_hints, getfullargspec, implements_iterator
 from hypothesis.internal.floats import next_up, next_down, is_negative, \
     float_to_int, int_to_float, count_between_floats
 from hypothesis.internal.charmap import as_general_categories
@@ -926,7 +927,31 @@ def text(
     elif isinstance(alphabet, SearchStrategy):
         char_strategy = alphabet
     else:
-        char_strategy = sampled_from(list(map(text_type, alphabet)))
+        if not isinstance(alphabet, Sequence):
+            note_deprecation(
+                'alphabet must be an ordered sequence, or tests may be '
+                'flaky and shrinking weaker, but a %r is not a type of '
+                'sequence.  This will be an error in future.'
+                % (type(alphabet),)
+            )
+        alphabet = list(alphabet)
+        non_string = [c for c in alphabet if not isinstance(c, string_types)]
+        if non_string:
+            note_deprecation(
+                'The following elements in alphabet are not unicode '
+                'strings, which will be an error in future:  %r'
+                % (non_string,)
+            )
+            alphabet = [str(c) for c in alphabet]
+        not_one_char = [c for c in alphabet
+                        if isinstance(c, string_types) and len(c) != 1]
+        if not_one_char:
+            note_deprecation(
+                'The following elements in alphabet are not of length '
+                'one, which leads to violation of size constraints and '
+                'will be an error in future:  %r' % (not_one_char,)
+            )
+        char_strategy = sampled_from(alphabet)
     return StringStrategy(lists(
         char_strategy, min_size=min_size, max_size=max_size
     ))


Test patch:
diff --git a/hypothesis-python/tests/cover/test_simple_strings.py b/hypothesis-python/tests/cover/test_simple_strings.py
--- a/hypothesis-python/tests/cover/test_simple_strings.py
+++ b/hypothesis-python/tests/cover/test_simple_strings.py
@@ -23,6 +23,7 @@
 import pytest
 
 from hypothesis import find, given, settings
+from tests.common.utils import checks_deprecated_behaviour
 from hypothesis.strategies import text, binary, tuples, characters
 
 
@@ -127,3 +128,18 @@ def test_fixed_size_bytes_just_draw_bytes():
 @given(text(max_size=10**6))
 def test_can_set_max_size_large(s):
     pass
+
+
+@checks_deprecated_behaviour
+def test_alphabet_is_not_a_sequence():
+    text(alphabet=set('abc')).example()
+
+
+@checks_deprecated_behaviour
+def test_alphabet_breaking_size_limit():
+    text(['a', 'c', 'ed', 'b', 'abc']).example()
+
+
+@checks_deprecated_behaviour
+def test_alphabet_non_string():
+    text([1, 2, 3]).example()