Repository: HypothesisWorks/hypothesis

Issue:
Generate fixed point decimals
Add an optional _places_ parameter to the `decimals()` strategy to specify the number of decimal places needed in generated values.

Many applications use `Decimal `numbers to perform precise calculations to a specific number of decimal places. Dollar amounts often use two decimal places, for example. Mutual fund shares may be tracked to five or six decimal places. Manufacturing tolerances are often specified to thousandths of an inch—three decimal places.

The Hypothesis `decimals()` strategy currently generates decimal numbers from floats or fractions. These value typically have uncontrollably many decimal places.
```
>>> from hypothesis.strategies import decimals
>>> decimals().example()
Decimal('0.002472471796257515368506383841')
```
My proof of concept, attached as [test_decimals.py.txt](https://github.com/HypothesisWorks/hypothesis-python/files/694095/test_decimals.py.txt), adds a an optional _places_ parameter to the `decimals()` strategy.  When _places_ is specified, `decimals()` generates values with the given number of digits after the decimal point. When _places_ is not specified, the current unconstrained value generation is used.
```
>>> import test_decimals
>>> decimals = test_decimals.decimals
>>> decimals(places=3).example()
Decimal('5269885835422919568130741.438')
>>> decimals(places=2).example()
Decimal('27258100601131699143211881.42')
>>> decimals().example()
Decimal('0.2908554201733450571536239166')
```
The sample code generates fixed point decimals from the `integers()` strategy rather than the floats and fractions used currently. The Hypothesis generated integers map directly to decimal values. With three decimal places, for example, 1 maps to .001, 2 maps to .002, and so on. For fixed decimal place values, I think that the underlying integer shrink strategy is more appropriate than the float or fraction strategy used for unconstrained decimal place values.

Python has a global decimal context containing various configuration parameters for the `decimal` module. Applications may change the decimal context globally or locally. My proof of concept code uses the context's _precision_ value, which specifies the maximum number of digits allowed in a value. Hypothesis caching does not play well with global values changed between tests. My sample code does not cache values from `decimals()`, but does cache values from `fixed_point_decimals()`, which has precision as an explicit parameter. For example,
```
>>> from decimal import localcontext
>>> with localcontext() as context:
...     context.prec = 10
...     value = decimals(places=3).example()
...     print(value)
...
2609227.817
>>> with localcontext() as context:
...     context.prec = 20
...     value = decimals(places=3).example()
...     print(value)
...
17961664460289794.568
```






Discussion:
Note that PR #409 is sorta related.
The _allow_nan_ and _allow_infinity_ flags requested PR #409 and related PR #186 could be added to my proof of concept code.

As discussed in those other PRs, the _allow_infinity_ flag default value _True_ is somewhat at conflict with the _max_value_ parameter. My sample code always has some _max_value_, either explicitly passed as an argument or derived from the Decimal context precision.

Applications (and people) using fixed point decimals typically don't worry about NaN or infinite values. Instead, these applications effectively perform integer math on some small units and report with a decimal inserted at an appropriate position. Users often have names for the smallest unit: 0.01 dollars is a cent, .00000001 bitcoins is a Satoshi, and .001 inches is a thousandth.

I hope that the default values for the _allow_nan_ and _allow_infinity_ flags do not require every fixed decimal point request to include `allow_nan=False, allow_infinity=False`. Those should be the default values when the _places_ argument is provided.
I am somewhat dubious because ```places``` actually affects the value returned, which seems wrong and AFAICT unnecessary.

I agree with your examples as being the usual and probably best use for the Decimal type.

I dunno about your final point, I haven't looked at that related code in a while. But I don't think it would be a big deal, because the strategy could be wrapped to send the desired values of those parameters anyway. The real problems occur if there are bad ideas implemented within the strategy that actually have to be worked around.
@mulkieran wrote, 
> I am somewhat dubious because `places `actually affects the value returned, which seems wrong and AFAICT unnecessary.

Shouldn't parameters such as `min_value`, `allow_nan`, and the proposed `places` affect the values returned by the `decimals` strategy? These parameters control which of all possible Decimal values the `decimals` strategy will provide. Values below some threshold are prohibited by `min_value`. Values with more than some number of decimal places are prohibited by `places`.

I agree with avoiding bad ideas in strategies and the need to work around them. As a relatively new Hypothesis user, I cannot always distinguish good and bad ideas in strategies. I am much more familiar with ordinary unit testing; I demonstrated PythonUnit at the January 2000 Python Conference.

The wrapper approach seems preferable to special cases for the `allow_nan` and `allow_infinity` flags. What would such a wrapper look like for, say, dollars with two decimal places and with no NaN or inifinity values?
My worry is specifically about the effect of the ```places``` parameter. I am not an expert, but I think that the integers() strategy is somewhat tuned to return values like 0, 1, 3. You get an integer from the strategy and transform it by means of places, favoring 1/10^place, 3 / 10 ^ places, etc. I'm not sure that is really right, and the strategy should maybe be preferred to favor things like 1.0, 3.0, etc. My concern is about the use of ```places``` to generate the values also affecting their distribution, possibly unhelpfully.

My remark about wrappers means that it is best to choose the simplest and most mathematically valid internal implementation of the strategy. Then, if the best default for ```allow_nan``` by that rubric is ```True``` it is still easy for someone to wrap the strategy and overrride the default, passing ```False``` if they want to.

I'm idly wondering if you might be better off distributing a competing Decimals strategy independently. I established a registry for external strategies a month or so ago, and right now there's only one on the list:
https://github.com/HypothesisWorks/hypothesis-python/blob/master/docs/strategies.rst.

But, I'm not the boss of Hypothesis, that's @DRMacIver :) So, I'm going to bow out for a bit and see what he has to say.




Resolving patch:
diff --git a/src/hypothesis/strategies.py b/src/hypothesis/strategies.py
--- a/src/hypothesis/strategies.py
+++ b/src/hypothesis/strategies.py
@@ -18,7 +18,7 @@
 from __future__ import division, print_function, absolute_import
 
 import math
-from decimal import Decimal
+from decimal import Decimal, InvalidOperation
 from numbers import Rational
 from fractions import Fraction
 
@@ -26,7 +26,7 @@
 from hypothesis.control import assume
 from hypothesis.searchstrategy import SearchStrategy
 from hypothesis.internal.compat import ArgSpec, hrange, text_type, \
-    getargspec, integer_types, float_to_decimal, implements_iterator
+    getargspec, integer_types, implements_iterator
 from hypothesis.internal.floats import is_negative, float_to_int, \
     int_to_float, count_between_floats
 from hypothesis.utils.conventions import not_set
@@ -836,31 +836,85 @@ def dm_func(denom):
 
 @cacheable
 @defines_strategy
-def decimals(
-    min_value=None, max_value=None, allow_nan=None, allow_infinity=None
-):
-    """Generates instances of decimals.Decimal.
+def decimals(min_value=None, max_value=None,
+             allow_nan=None, allow_infinity=None, places=None):
+    """Generates instances of decimals.Decimal, which may be:
 
-    - If min_value is not None, all values will be >= min_value.
-    - If max_value is not None, all values will be <= max_value.
-    - If min_value or max_value is not None, it is an error to enable
-      allow_nan.
-    - If both min_value and max_value are not None, it is an error to enable
-      allow_infinity.
+    - A finite rational number, between ``min_value`` and ``max_value``.
+    - Not a Number, if ``allow_nan`` is True.  None means "allow NaN, unless
+      ``min__value`` and ``max_value`` are not None".
+    - Positive or negative infinity, if ``max_value`` and ``min_value``
+      respectively are None, and ``allow_infinity`` is not False.  None means
+      "allow infinity, unless excluded by the min and max values".
 
-    Where not explicitly ruled out by the bounds, all of infinity, -infinity
-    and NaN are possible values generated by this strategy.
+    Note that where floats have one `NaN` value, Decimals have four: signed,
+    and either *quiet* or *signalling*.  See `the decimal module docs
+    <https://docs.python.org/3/library/decimal.html#special-values>`_ for
+    more information on special values.
+
+    If ``places`` is not None, all finite values drawn from the strategy will
+    have that number of digits after the decimal place.
 
     """
-    floats_strategy = floats(
-        min_value=min_value, max_value=max_value,
-        allow_nan=allow_nan, allow_infinity=allow_infinity)
-    fractions_strategy = fractions(min_value=min_value, max_value=max_value)
-
-    return (
-        floats_strategy.map(float_to_decimal) |
-        fractions_strategy.map(lambda f: Decimal(f.numerator) / f.denominator)
-    )
+    # Convert min_value and max_value to Decimal values, and validate args
+    check_valid_integer(places)
+    if places is not None and places < 0:
+        raise InvalidArgument('places=%r may not be negative' % places)
+
+    if min_value is not None:
+        min_value = Decimal(min_value)
+        if min_value.is_infinite() and min_value < 0:
+            if not (allow_infinity or allow_infinity is None):
+                raise InvalidArgument('allow_infinity=%r, but min_value=%r'
+                                      % (allow_infinity, min_value))
+            min_value = None
+        elif not min_value.is_finite():
+            # This could be positive infinity, quiet NaN, or signalling NaN
+            raise InvalidArgument(u'Invalid min_value=%r' % min_value)
+    if max_value is not None:
+        max_value = Decimal(max_value)
+        if max_value.is_infinite() and max_value > 0:
+            if not (allow_infinity or allow_infinity is None):
+                raise InvalidArgument('allow_infinity=%r, but max_value=%r'
+                                      % (allow_infinity, max_value))
+            max_value = None
+        elif not max_value.is_finite():
+            raise InvalidArgument(u'Invalid max_value=%r' % max_value)
+    check_valid_bound(min_value, 'min_value')
+    check_valid_bound(max_value, 'max_value')
+    check_valid_interval(min_value, max_value, 'min_value', 'max_value')
+
+    if allow_infinity and (None not in (min_value, max_value)):
+        raise InvalidArgument('Cannot allow infinity between finite bounds')
+    # Set up a strategy for finite decimals
+    if places is not None:
+        # Fixed-point decimals are basically integers with a scale factor
+
+        def try_quantize(d):
+            try:
+                return d.quantize(factor)
+            except InvalidOperation:  # pragma: no cover
+                return None
+        factor = Decimal(10) ** -places
+        max_num = max_value / factor if max_value is not None else None
+        min_num = min_value / factor if min_value is not None else None
+        strat = integers(min_value=min_num, max_value=max_num)\
+            .map(lambda d: try_quantize(d * factor))\
+            .filter(lambda d: d is not None)
+    else:
+        # Otherwise, they're like fractions featuring a power of ten
+        strat = fractions(
+            min_value=min_value, max_value=max_value
+        ).map(lambda f: Decimal(f.numerator) / f.denominator)
+    # Compose with sampled_from for infinities and NaNs as appropriate
+    special = []
+    if allow_nan or (allow_nan is None and (None in (min_value, max_value))):
+        special.extend(map(Decimal, ('NaN', '-NaN', 'sNaN', '-sNaN')))
+    if allow_infinity or (allow_infinity is max_value is None):
+        special.append(Decimal('Infinity'))
+    if allow_infinity or (allow_infinity is min_value is None):
+        special.append(Decimal('-Infinity'))
+    return strat | sampled_from(special)
 
 
 def recursive(base, extend, max_leaves=100):
diff --git a/src/hypothesis/version.py b/src/hypothesis/version.py
--- a/src/hypothesis/version.py
+++ b/src/hypothesis/version.py
@@ -17,5 +17,5 @@
 
 from __future__ import division, print_function, absolute_import
 
-__version_info__ = (3, 8, 5)
+__version_info__ = (3, 9, 0)
 __version__ = '.'.join(map(str, __version_info__))


Test patch:
diff --git a/tests/cover/test_direct_strategies.py b/tests/cover/test_direct_strategies.py
--- a/tests/cover/test_direct_strategies.py
+++ b/tests/cover/test_direct_strategies.py
@@ -26,6 +26,7 @@
 import hypothesis.strategies as ds
 from hypothesis import find, given, settings
 from hypothesis.errors import InvalidArgument
+from hypothesis.internal.compat import float_to_decimal
 from hypothesis.internal.reflection import nicerepr
 
 
@@ -59,11 +60,15 @@ def fn_ktest(*fnkwargs):
     (ds.integers, {'min_value': 2, 'max_value': 1}),
     (ds.decimals, {'min_value': float('nan')}),
     (ds.decimals, {'min_value': 2, 'max_value': 1}),
+    (ds.decimals, {'max_value': '-snan'}),
+    (ds.decimals, {'places': -1}),
     (ds.decimals, {'max_value': 0.0, 'min_value': 1.0}),
-    (ds.decimals, {'min_value': 0.0, 'allow_nan': True}),
-    (ds.decimals, {'max_value': 0.0, 'allow_nan': True}),
     (ds.decimals, {
         'min_value': 0.0, 'max_value': 1.0, 'allow_infinity': True}),
+    (ds.decimals, {'min_value': 'inf'}),
+    (ds.decimals, {'max_value': '-inf'}),
+    (ds.decimals, {'min_value': '-inf', 'allow_infinity': False}),
+    (ds.decimals, {'max_value': 'inf', 'allow_infinity': False}),
     (ds.fractions, {'min_value': float('nan')}),
     (ds.fractions, {'min_value': 2, 'max_value': 1}),
     (ds.fractions, {'max_denominator': 0}),
@@ -104,15 +109,16 @@ def test_validates_keyword_arguments(fn, kwargs):
     (ds.integers, {'min_value': 11, 'max_value': 100}),
     (ds.integers, {'max_value': 0}),
     (ds.integers, {'min_value': decimal.Decimal('1.5')}),
-    (ds.integers, {'min_value': fractions.Fraction(1, 2)}),
     (ds.decimals, {'min_value': 1.0, 'max_value': 1.5}),
+    (ds.decimals, {'min_value': '1.0', 'max_value': '1.5'}),
     (ds.decimals, {'min_value': decimal.Decimal('1.5')}),
-    (ds.decimals, {'min_value': fractions.Fraction(1, 3)}),
     (ds.decimals, {
         'max_value': 1.0, 'min_value': -1.0, 'allow_infinity': False}),
     (ds.decimals, {'min_value': 1.0, 'allow_nan': False}),
     (ds.decimals, {'max_value': 1.0, 'allow_nan': False}),
     (ds.decimals, {'max_value': 1.0, 'min_value': -1.0, 'allow_nan': False}),
+    (ds.decimals, {'min_value': '-inf'}),
+    (ds.decimals, {'max_value': 'inf'}),
     (ds.fractions, {
         'min_value': -1, 'max_value': 1, 'max_denominator': 1000}),
     (ds.fractions, {'min_value': 1.0}),
@@ -212,7 +218,7 @@ def test_fraction_lt_negative(x):
     assert x <= fractions.Fraction(-1, 2)
 
 
-@given(ds.decimals(min_value=-1.5, max_value=1.5))
+@given(ds.decimals(min_value=-1.5, max_value=1.5, allow_nan=False))
 def test_decimal_is_in_bounds(x):
     # decimal.Decimal("-1.5") == -1.5 (not explicitly testable in py2.6)
     assert decimal.Decimal('-1.5') <= x <= decimal.Decimal('1.5')
@@ -246,7 +252,7 @@ def test_decimals():
 
 
 def test_non_float_decimal():
-    find(ds.decimals(), lambda d: ds.float_to_decimal(float(d)) != d)
+    find(ds.decimals(), lambda d: float_to_decimal(float(d)) != d)
 
 
 def test_produces_dictionaries_of_at_least_minimum_size():
diff --git a/tests/cover/test_numerics.py b/tests/cover/test_numerics.py
--- a/tests/cover/test_numerics.py
+++ b/tests/cover/test_numerics.py
@@ -18,10 +18,14 @@
 from __future__ import division, print_function, absolute_import
 
 import math
+import decimal
+
+import pytest
 
 from hypothesis import given, assume
-from tests.common.utils import fails, fails_with
-from hypothesis.strategies import decimals, fractions, float_to_decimal
+from tests.common.utils import fails
+from hypothesis.strategies import decimals, fractions
+from hypothesis.internal.compat import float_to_decimal
 
 
 @fails
@@ -36,15 +40,16 @@ def test_fraction_addition_is_well_behaved(x, y, z):
     assert x + y + z == y + x + z
 
 
-@fails_with(AssertionError)
+@fails
 @given(decimals())
 def test_decimals_include_nan(x):
     assert not math.isnan(x)
 
 
-@fails_with(AssertionError)
+@fails
 @given(decimals())
 def test_decimals_include_inf(x):
+    assume(not x.is_snan())
     assert not math.isinf(x)
 
 
@@ -55,4 +60,18 @@ def test_decimals_can_disallow_nan(x):
 
 @given(decimals(allow_infinity=False))
 def test_decimals_can_disallow_inf(x):
+    assume(not x.is_snan())
     assert not math.isinf(x)
+
+
+@pytest.mark.parametrize('places', range(10))
+def test_decimals_have_correct_places(places):
+    @given(decimals(0, 10, allow_nan=False, places=places))
+    def inner_tst(n):
+        assert n.as_tuple().exponent == -places
+    inner_tst()
+
+
+@given(decimals(min_value='0.1', max_value='0.2', allow_nan=False, places=1))
+def test_works_with_few_values(dec):
+    assert dec in (decimal.Decimal('0.1'), decimal.Decimal('0.2'))