Repository: python-babel/babel
Pull request #519 (2017-08-18 13:47:23)

Parse multi-line __future__ imports better
Augments #426
Augments #427
Augments #510
Closes #510

codecov-io: # [Codecov](https://codecov.io/gh/python-babel/babel/pull/519?src=pr&el=h1) Report
> Merging [#519](https://codecov.io/gh/python-babel/babel/pull/519?src=pr&el=desc) into [master](https://codecov.io/gh/python-babel/babel/commit/e6a926a78d7286b84ff482ed1f61131518532190?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/python-babel/babel/pull/519/graphs/tree.svg?height=150&width=650&token=lEYaWCjCnu&src=pr)](https://codecov.io/gh/python-babel/babel/pull/519?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #519      +/-   ##
==========================================
+ Coverage   90.18%   90.19%   +<.01%     
==========================================
  Files          24       24              
  Lines        4015     4018       +3     
==========================================
+ Hits         3621     3624       +3     
  Misses        394      394
```


| [Impacted Files](https://codecov.io/gh/python-babel/babel/pull/519?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [babel/util.py](https://codecov.io/gh/python-babel/babel/pull/519?src=pr&el=tree#diff-YmFiZWwvdXRpbC5weQ==) | `92% <100%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/python-babel/babel/pull/519?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/python-babel/babel/pull/519?src=pr&el=footer). Last update [e6a926a...7d2a89c](https://codecov.io/gh/python-babel/babel/pull/519?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

akx: @sublee, care to take a look? :)
sublee: @akx Looks better.

I suggest you to add more multi-line import styles for unit testing:

```python
from __future__ import (
    print_function,
    division)
```

```python
from __future__ import \
    print_function, \
    division
```
akx: @sublee Thanks! Excellent points -- neither of those cases actually worked. They do now! :)

Patch:
diff --git a/babel/util.py b/babel/util.py
index 996f902..af8c762 100644
--- a/babel/util.py
+++ b/babel/util.py
@@ -109,10 +109,23 @@ def parse_future_flags(fp, encoding='latin-1'):
     flags = 0
     try:
         body = fp.read().decode(encoding)
+
+        # Fix up the source to be (hopefully) parsable by regexpen.
+        # This will likely do untoward things if the source code itself is broken.
+
+        # (1) Fix `import (\n...` to be `import (...`.
+        body = re.sub(r'import\s*\([\r\n]+', 'import (', body)
+        # (2) Join line-ending commas with the next line.
+        body = re.sub(r',\s*[\r\n]+', ', ', body)
+        # (3) Remove backslash line continuations.
+        body = re.sub(r'\\\s*[\r\n]+', ' ', body)
+
         for m in PYTHON_FUTURE_IMPORT_re.finditer(body):
-            names = [x.strip() for x in m.group(1).split(',')]
+            names = [x.strip().strip('()') for x in m.group(1).split(',')]
             for name in names:
-                flags |= getattr(__future__, name).compiler_flag
+                feature = getattr(__future__, name, None)
+                if feature:
+                    flags |= feature.compiler_flag
     finally:
         fp.seek(pos)
     return flags


Test patch:
diff --git a/tests/test_util.py b/tests/test_util.py
index 068a102..4c769be 100644
--- a/tests/test_util.py
+++ b/tests/test_util.py
@@ -13,8 +13,11 @@
 
 import unittest
 
+import pytest
+
 from babel import util
 from babel._compat import BytesIO
+from babel.util import parse_future_flags
 
 
 def test_distinct():
@@ -69,3 +72,34 @@ def test_parse_encoding_undefined():
 
 def test_parse_encoding_non_ascii():
     assert parse_encoding(u'K\xf6ln') is None
+
+
+@pytest.mark.parametrize('source, result', [
+    ('''
+from __future__ import print_function,
+    division, with_statement,
+    unicode_literals
+''', 0x10000 | 0x2000 | 0x8000 | 0x20000),
+    ('''
+from __future__ import print_function, division
+print('hello')
+''', 0x10000 | 0x2000),
+    ('''
+from __future__ import print_function, division, unknown,,,,,
+print 'hello'
+''', 0x10000 | 0x2000),
+    ('''
+from __future__ import (
+    print_function,
+    division)
+''', 0x10000 | 0x2000),
+    ('''
+from __future__ import \\
+    print_function, \\
+    division
+''', 0x10000 | 0x2000),
+])
+def test_parse_future(source, result):
+    fp = BytesIO(source.encode('latin-1'))
+    flags = parse_future_flags(fp)
+    assert flags == result