Repository: python-babel/babel
Pull request #877 (2022-05-12 21:30:46)

Support for hex escapes in JavaScript strings
Currently Babel only supports `\u`-style Unicode escapes in JavaScript string literals.

Unfortunately, I have to deal witth JS files (WebPack-generated) that contain `\x`-style escapes. I believe Babel should support it, as it's perfectly legitimate JS syntax.

codecov[bot]: # [Codecov](https://codecov.io/gh/python-babel/babel/pull/877?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) Report
> Merging [#877](https://codecov.io/gh/python-babel/babel/pull/877?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) (cf2c37a) into [master](https://codecov.io/gh/python-babel/babel/commit/40059883cc260af890bcc81a1a21c0f84781857f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) (4005988) will **decrease** coverage by `0.57%`.
> The diff coverage is `88.88%`.

```diff
@@            Coverage Diff             @@
##           master     #877      +/-   ##
==========================================
- Coverage   91.30%   90.73%   -0.58%     
==========================================
  Files          23       21       -2     
  Lines        4187     4186       -1     
==========================================
- Hits         3823     3798      -25     
- Misses        364      388      +24     
```


| [Impacted Files](https://codecov.io/gh/python-babel/babel/pull/877?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | Coverage Δ | |
|---|---|---|
| [babel/messages/jslexer.py](https://codecov.io/gh/python-babel/babel/pull/877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel#diff-YmFiZWwvbWVzc2FnZXMvanNsZXhlci5weQ==) | `87.00% <88.88%> (+0.33%)` | :arrow_up: |
| [babel/localtime/\_win32.py](https://codecov.io/gh/python-babel/babel/pull/877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel#diff-YmFiZWwvbG9jYWx0aW1lL193aW4zMi5weQ==) | `21.56% <0.00%> (-41.40%)` | :arrow_down: |
| [babel/localtime/\_\_init\_\_.py](https://codecov.io/gh/python-babel/babel/pull/877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel#diff-YmFiZWwvbG9jYWx0aW1lL19faW5pdF9fLnB5) | `61.53% <0.00%> (-2.57%)` | :arrow_down: |
| [babel/messages/frontend.py](https://codecov.io/gh/python-babel/babel/pull/877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel#diff-YmFiZWwvbWVzc2FnZXMvZnJvbnRlbmQucHk=) | `87.17% <0.00%> (-0.75%)` | :arrow_down: |
| [babel/messages/pofile.py](https://codecov.io/gh/python-babel/babel/pull/877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel#diff-YmFiZWwvbWVzc2FnZXMvcG9maWxlLnB5) | `95.49% <0.00%> (-0.02%)` | :arrow_down: |
| [babel/localedata.py](https://codecov.io/gh/python-babel/babel/pull/877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel#diff-YmFiZWwvbG9jYWxlZGF0YS5weQ==) | `95.86% <0.00%> (ø)` | |
| [babel/messages/\_\_init\_\_.py](https://codecov.io/gh/python-babel/babel/pull/877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel#diff-YmFiZWwvbWVzc2FnZXMvX19pbml0X18ucHk=) | | |
| [babel/\_\_init\_\_.py](https://codecov.io/gh/python-babel/babel/pull/877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel#diff-YmFiZWwvX19pbml0X18ucHk=) | | |
| [babel/dates.py](https://codecov.io/gh/python-babel/babel/pull/877/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel#diff-YmFiZWwvZGF0ZXMucHk=) | `92.62% <0.00%> (+0.12%)` | :arrow_up: |
| ... and [1 more](https://codecov.io/gh/python-babel/babel/pull/877/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel) | |

:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=python-babel)


Patch:
diff --git a/babel/messages/jslexer.py b/babel/messages/jslexer.py
index c1c2557..1264b2d 100644
--- a/babel/messages/jslexer.py
+++ b/babel/messages/jslexer.py
@@ -1,3 +1,4 @@
+# -*- coding: utf-8 -*-
 """
     babel.messages.jslexer
     ~~~~~~~~~~~~~~~~~~~~~~
@@ -27,6 +28,7 @@ regex_re = re.compile(r'/(?:[^/\\]*(?:\\.[^/\\]*)*)/[a-zA-Z]*', re.DOTALL)
 line_re = re.compile(r'(\r\n|\n|\r)')
 line_join_re = re.compile(r'\\' + line_re.pattern)
 uni_escape_re = re.compile(r'[a-fA-F0-9]{1,4}')
+hex_escape_re = re.compile(r'[a-fA-F0-9]{1,2}')
 
 Token = namedtuple('Token', 'type value lineno')
 
@@ -127,6 +129,17 @@ def unquote_string(string):
             else:
                 add(next_char)
 
+        # hex escapes. conversion from 2-digits hex to char is infallible
+        elif next_char in 'xX':
+            escaped = hex_escape_re.match(string, escape_pos + 2)
+            if escaped is not None:
+                escaped_value = escaped.group()
+                add(chr(int(escaped_value, 16)))
+                pos = escape_pos + 2 + len(escaped_value)
+                continue
+            else:
+                add(next_char)
+
         # bogus escape.  Just remove the backslash.
         else:
             add(next_char)


Test patch:
diff --git a/tests/messages/test_jslexer.py b/tests/messages/test_jslexer.py
index 35204ee..bd6322e 100644
--- a/tests/messages/test_jslexer.py
+++ b/tests/messages/test_jslexer.py
@@ -4,6 +4,8 @@ from babel.messages import jslexer
 def test_unquote():
     assert jslexer.unquote_string('""') == ''
     assert jslexer.unquote_string(r'"h\u00ebllo"') == u"hëllo"
+    assert jslexer.unquote_string(r'"h\xebllo"') == u"hëllo"
+    assert jslexer.unquote_string(r'"\xebb"') == u"ëb"
 
 
 def test_dollar_in_identifier():