Repository: lisa-lab/pylearn2

Issue:
Add a YAML implicit resolver for 1e3, 2E-6, -4e99, etc.
The form <optional sign><int><e or E><optional sign><int> for floats doesn't work in YAML since #812. What #812 removed was overkill, but it'd be nice to have this back at some point.

This ticket is to add a resolver for these types.

A regular expression to get you started is `r'\-?\d+[eE]\-?\d+'` (note the r for [raw string](http://stackoverflow.com/a/2081708/50462)). You might replace the two occurrences of `\-?` with `[\-\+]?` if you're feeling fancy.

Bonus: add resolvers for floats that include a + for either the mantissa or the exponent, e.g. 1E+4, +4e5, +9e+20, etc., but in a way that _doesn't_ trigger the resolver for things that YAML already handles just fine like "1.0e-4".



Discussion:
YAML handles the numbers correctly when the mantissa is a float, and the exponent has a sign. For example, +1.2E+3, -3.e-2 works fine, while 3e+2, 2.1e2 don't. 

Great. Let me know if you have any trouble with getting the implicit resolver set up. I think what you want is to just call float() on the entire match.

A good strategy would be to write tests for all the cases you know about, and ensure they fail in the way that you expect. Then implement the solution and ensure they pass.

If you want to send an early pull request with just the tests in order to get feedback sooner, go for it. Up to you.


Resolving patch:
diff --git a/pylearn2/config/yaml_parse.py b/pylearn2/config/yaml_parse.py
--- a/pylearn2/config/yaml_parse.py
+++ b/pylearn2/config/yaml_parse.py
@@ -8,9 +8,12 @@
 from collections import namedtuple
 import logging
 import warnings
+import re
 
 from theano.compat import six
 
+SCIENTIFIC_NOTATION_REGEXP = r'^[\-\+]?(\d+\.?\d*|\d*\.?\d+)?[eE][\-\+]?\d+$'
+
 is_initialized = False
 additional_environ = None
 logger = logging.getLogger(__name__)
@@ -462,6 +465,9 @@ def initialize():
     yaml.add_constructor('!import', constructor_import)
     yaml.add_constructor("!float", constructor_float)
 
+    pattern = re.compile(SCIENTIFIC_NOTATION_REGEXP)
+    yaml.add_implicit_resolver('!float', pattern)
+
     is_initialized = True
 
 


Test patch:
diff --git a/pylearn2/config/tests/test_yaml_parse.py b/pylearn2/config/tests/test_yaml_parse.py
--- a/pylearn2/config/tests/test_yaml_parse.py
+++ b/pylearn2/config/tests/test_yaml_parse.py
@@ -18,7 +18,9 @@
 from pylearn2.config.yaml_parse import load, load_path, initialize
 from pylearn2.utils import serial
 from pylearn2.utils.exc import reraise_as
+from pylearn2.config.yaml_parse import SCIENTIFIC_NOTATION_REGEXP
 import yaml
+import re
 
 
 def test_load_path():
@@ -43,6 +45,43 @@ def test_floats():
     assert_((loaded['a']['b'] - 1.23e-1) < 1e-3)
 
 
+def test_notation_regexp():
+    """
+    Tests for regular expression aiming to filter scientific notations which
+    are not correctly parsed as floats by YAML.
+
+    Notes
+    -----
+    This regexp overmatches. E.g. 1.2e+3 matches the regexp,
+    but is already correctly parsed by YAML.
+    """
+    pattern = re.compile(SCIENTIFIC_NOTATION_REGEXP)
+    matches = ['1e3', '1.E4', '-1e-3', '2.3e+4', '.2e4']
+    fails = ['string', '4', '2.1', '1.2e3.2', '.e4', 'a3e4']
+
+    for match in matches:
+        assert pattern.match(match)
+
+    for fail in fails:
+        assert not pattern.match(fail)
+
+
+def test_scientific_notation():
+    """
+    Test if yaml parses scientific notation as floats.
+    """
+    loaded = load('a: {a: 1e3, b: 1.E4, c: -1e-3, d: 2.3e+4, e: .2e4, '
+                  'f: 23.53e1, g: 32284.2e+9, h: 2.333993939e-3}')
+    assert isinstance(loaded['a']['a'], float)
+    assert isinstance(loaded['a']['b'], float)
+    assert isinstance(loaded['a']['c'], float)
+    assert isinstance(loaded['a']['d'], float)
+    assert isinstance(loaded['a']['e'], float)
+    assert isinstance(loaded['a']['f'], float)
+    assert isinstance(loaded['a']['g'], float)
+    assert isinstance(loaded['a']['h'], float)
+
+
 def test_import():
     loaded = load("a: !import 'decimal.Decimal'")
     assert_(loaded['a'] == Decimal)