Repository: python-babel/babel
Pull request #396 (2016-04-25 13:12:27)

jslexer.py: Change jsx_tag regex again
We've had some syntactic variations that my previous PR, #392, did not
account for. One case had a component as a prop that tripped up the
regex.

akx: @karloskar CI seems to be failing. Also, I think the test case is maybe a little too verbose? Would be better to test that extraction still works as it should, not that the lexer knows about every atom :)
karloskar: I'll have to admit that I'm maybe in to deep here :)

I'm not that familiar with neither the codebase nor Travis but from what I can tell the error stems from Codecov complaining on some missing file. Not really sure what I can do about that :/

And I agree on the verbosity of the test. I had a hard time figuring out something that took in account all the different syntactic variants that I've seen in other peoples issues and that we are using. Would separating them to different methods be a suitable way forward? All tips are appreciated!
akx: Hey @karloskar -- sorry for the delay in getting back to you!

Could you list some of the cases you'd need to handle in your codebase?
karloskar: Sure! Thanks for taking a look!

The ones in the test file covers all of our variations right now. And despite a rather high tempo and a growing code base the last couple of weeks we haven't found a need for anything else.

As children to a component
```jsx
<option value="{ true }">{ i18n._('String') }</option>
```
As an "interpolated" prop
```jsx
<component value={ i18n._('String') } />
```
And this variant where there are components as props that may or may not contain translatable strings themselves.
```jsx
<component prop={ <otherComponent /> } data={ {active: true} }>
  <btn text={ i18n._('String') } />
</component>
```

The first case added to the test file (multiple option tags) was taken from this issue python-babel/flask-babel#79.
codecov-io: ## [Current coverage](https://codecov.io/gh/python-babel/babel/pull/396?src=pr) is 89.47% (diff: 100%)


> No coverage report found for **master** at b5e0ad8.

> Powered by [Codecov](https://codecov.io?src=pr). Last update [b5e0ad8...33e2725](https://codecov.io/gh/python-babel/babel/compare/b5e0ad81340d339314ae57d365186f0d9d4ce961...33e2725ea3b12be355121c0de616d1cb0da49e07?src=pr)
akx: Sorry it took me so long to get back to this!

Can you rebase this on top of the current  `master`? We've disabled OSX CI in the meanwhile since it seems to be broken, but this branch doesn't currently have that CI config :)
karloskar: Can't say I really knew what I did. First time rebasing for a pull request update. Would the conflicts be resolvable by you or do you want me to change anything?
akx: Ahh, yeah, something went wrong there alright. Mind if I just grab your patches and create a new PR out of them, @karloskar? :) Will naturally keep your Author credit.
karloskar: I do not mind at all. Please do as you see fit!
akx: Oh, wait, @karloskar: I think you might be able to resolve this on your end, as the pt2 commit d2fc566f66272c026c50472e20d4e23748c59279 applies cleanly :)

```bash
# (if you don't yet have this repo as the upstream)
git remote add upstream https://github.com/python-babel/babel.git
# ensure your upstream remote is up to date
git fetch upstream
# checkout this branch
git checkout feature/evolve-jsx-extraction-pt2
# keep a stash branch around for possible disaster recovery purposes
git branch orig-jsx-extraction
# reset it to upstream/master
git reset --hard upstream/master
# apply the pt2 patch
git cherry-pick d2fc566f66272c026c50472e20d4e23748c59279
# force-push the branch (now with only one commit on top of upstream/master)
git push -f origin HEAD
```

Feel free to remove the disaster recovery branch at your leisure

Patch:
diff --git a/babel/messages/jslexer.py b/babel/messages/jslexer.py
index 665a86f..a9bab2f 100644
--- a/babel/messages/jslexer.py
+++ b/babel/messages/jslexer.py
@@ -45,7 +45,7 @@ _rules = [
         ([eE][-+]?\d+)? |
         (0x[a-fA-F0-9]+)
     )''')),
-    ('jsx_tag', re.compile(r'</?[^> ]+', re.I)),  # May be mangled in `get_rules`
+    ('jsx_tag', re.compile(r'(?:</?[^>\s]+|/>)', re.I)),  # May be mangled in `get_rules`
     ('operator', re.compile(r'(%s)' % '|'.join(map(re.escape, operators)))),
     ('template_string', re.compile(r'''`(?:[^`\\]*(?:\\.[^`\\]*)*)`''', re.UNICODE)),
     ('string', re.compile(r'''(?xs)(


Test patch:
diff --git a/tests/messages/test_jslexer.py b/tests/messages/test_jslexer.py
index e125fec..b70621f 100644
--- a/tests/messages/test_jslexer.py
+++ b/tests/messages/test_jslexer.py
@@ -40,55 +40,86 @@ def test_jsx():
          <option value="val2">{ i18n._('String 2') }</option>
          <option value="val3">{ i18n._('String 3') }</option>
          <component value={i18n._('String 4')} />
+         <comp2 prop={<comp3 />} data={{active: true}}>
+             <btn text={ i18n._('String 5') } />
+         </comp2>
     """, jsx=True)) == [
-         ('jsx_tag', '<option', 2),
-         ('name', 'value', 2),
-         ('operator', '=', 2),
-         ('string', '"val1"', 2),
-         ('operator', '>', 2),
-         ('operator', '{', 2),
-         ('name', 'i18n._', 2),
-         ('operator', '(', 2),
-         ('string', "'String1'", 2),
-         ('operator', ')', 2),
-         ('operator', '}', 2),
-         ('jsx_tag', '</option', 2),
-         ('operator', '>', 2),
-         ('jsx_tag', '<option', 3),
-         ('name', 'value', 3),
-         ('operator', '=', 3),
-         ('string', '"val2"', 3),
-         ('operator', '>', 3),
-         ('operator', '{', 3),
-         ('name', 'i18n._', 3),
-         ('operator', '(', 3),
-         ('string', "'String 2'", 3),
-         ('operator', ')', 3),
-         ('operator', '}', 3),
-         ('jsx_tag', '</option', 3),
-         ('operator', '>', 3),
-         ('jsx_tag', '<option', 4),
-         ('name', 'value', 4),
-         ('operator', '=', 4),
-         ('string', '"val3"', 4),
-         ('operator', '>', 4),
-         ('operator', '{', 4),
-         ('name', 'i18n._', 4),
-         ('operator', '(', 4),
-         ('string', "'String 3'", 4),
-         ('operator', ')', 4),
-         ('operator', '}', 4),
-         ('jsx_tag', '</option', 4),
-         ('operator', '>', 4),
-         ('jsx_tag', '<component', 5),
-         ('name', 'value', 5),
-         ('operator', '=', 5),
-         ('operator', '{', 5),
-         ('name', 'i18n._', 5),
-         ('operator', '(', 5),
-         ('string', "'String 4'", 5),
-         ('operator', ')', 5),
-         ('operator', '}', 5),
-         ('operator', '/', 5),
-         ('operator', '>', 5)
-     ]
+        ('jsx_tag', '<option', 2),
+        ('name', 'value', 2),
+        ('operator', '=', 2),
+        ('string', '"val1"', 2),
+        ('operator', '>', 2),
+        ('operator', '{', 2),
+        ('name', 'i18n._', 2),
+        ('operator', '(', 2),
+        ('string', "'String1'", 2),
+        ('operator', ')', 2),
+        ('operator', '}', 2),
+        ('jsx_tag', '</option', 2),
+        ('operator', '>', 2),
+        ('jsx_tag', '<option', 3),
+        ('name', 'value', 3),
+        ('operator', '=', 3),
+        ('string', '"val2"', 3),
+        ('operator', '>', 3),
+        ('operator', '{', 3),
+        ('name', 'i18n._', 3),
+        ('operator', '(', 3),
+        ('string', "'String 2'", 3),
+        ('operator', ')', 3),
+        ('operator', '}', 3),
+        ('jsx_tag', '</option', 3),
+        ('operator', '>', 3),
+        ('jsx_tag', '<option', 4),
+        ('name', 'value', 4),
+        ('operator', '=', 4),
+        ('string', '"val3"', 4),
+        ('operator', '>', 4),
+        ('operator', '{', 4),
+        ('name', 'i18n._', 4),
+        ('operator', '(', 4),
+        ('string', "'String 3'", 4),
+        ('operator', ')', 4),
+        ('operator', '}', 4),
+        ('jsx_tag', '</option', 4),
+        ('operator', '>', 4),
+        ('jsx_tag', '<component', 5),
+        ('name', 'value', 5),
+        ('operator', '=', 5),
+        ('operator', '{', 5),
+        ('name', 'i18n._', 5),
+        ('operator', '(', 5),
+        ('string', "'String 4'", 5),
+        ('operator', ')', 5),
+        ('operator', '}', 5),
+        ('jsx_tag', '/>', 5),
+        ('jsx_tag', '<comp2', 6),
+        ('name', 'prop', 6),
+        ('operator', '=', 6),
+        ('operator', '{', 6),
+        ('jsx_tag', '<comp3', 6),
+        ('jsx_tag', '/>', 6),
+        ('operator', '}', 6),
+        ('name', 'data', 6),
+        ('operator', '=', 6),
+        ('operator', '{', 6),
+        ('operator', '{', 6),
+        ('name', 'active', 6),
+        ('operator', ':', 6),
+        ('name', 'true', 6),
+        ('operator', '}', 6),
+        ('operator', '}', 6),
+        ('operator', '>', 6),
+        ('jsx_tag', '<btn', 7),
+        ('name', 'text', 7),
+        ('operator', '=', 7),
+        ('operator', '{', 7),
+        ('name', 'i18n._', 7),
+        ('operator', '(', 7),
+        ('string', "'String 5'", 7),
+        ('operator', ')', 7),
+        ('operator', '}', 7),
+        ('jsx_tag', '/>', 7),
+        ('jsx_tag', '</comp2', 8),
+        ('operator', '>', 8)
+    ]