Repository: lingui/js-lingui
Pull request #1032 (2021-04-08 08:57:01)

fix: respect unicode chars in t macro
Will unblock the migration of #1022 

vercel[bot]: This pull request is being automatically deployed with Vercel ([learn more](https://vercel.link/github-learn-more)).  
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: [https://vercel.com/lingui-js/js-lingui/DeRDexAuswbirT85GZtm3kEVnemH](https://vercel.com/lingui-js/js-lingui/DeRDexAuswbirT85GZtm3kEVnemH)  
✅ Preview: *In Progress*


github-actions[bot]: ## size-limit report 📦 
| Path                                                               | Size         |
| ------------------------------------------------------------------ | ------------ |
| ./packages/core/build/cjs/core.production.min.js                   | 2.73 KB (0%) |
| ./packages/detect-locale/build/cjs/detect-locale.production.min.js | 798 B (0%)   |
| ./packages/react/build/cjs/react.production.min.js                 | 4.85 KB (0%) |
codecov[bot]: # [Codecov](https://codecov.io/gh/lingui/js-lingui/pull/1032?src=pr&el=h1) Report
> Merging [#1032](https://codecov.io/gh/lingui/js-lingui/pull/1032?src=pr&el=desc) (9ebae95) into [main](https://codecov.io/gh/lingui/js-lingui/commit/76553af0bea2e31048083454005ea9e6366ccec0?el=desc) (76553af) will **decrease** coverage by `0.14%`.
> The diff coverage is `50.00%`.

[![Impacted file tree graph](https://codecov.io/gh/lingui/js-lingui/pull/1032/graphs/tree.svg?width=650&height=150&src=pr&token=CuJ1VD5BY5)](https://codecov.io/gh/lingui/js-lingui/pull/1032?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##             main    #1032      +/-   ##
==========================================
- Coverage   83.35%   83.21%   -0.15%     
==========================================
  Files          53       53              
  Lines        1622     1626       +4     
  Branches      436      441       +5     
==========================================
+ Hits         1352     1353       +1     
- Misses        157      158       +1     
- Partials      113      115       +2     
```


| [Impacted Files](https://codecov.io/gh/lingui/js-lingui/pull/1032?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [packages/macro/src/macroJsx.ts](https://codecov.io/gh/lingui/js-lingui/pull/1032/diff?src=pr&el=tree#diff-cGFja2FnZXMvbWFjcm8vc3JjL21hY3JvSnN4LnRz) | `91.11% <0.00%> (-0.68%)` | :arrow_down: |
| [packages/macro/src/macroJs.ts](https://codecov.io/gh/lingui/js-lingui/pull/1032/diff?src=pr&el=tree#diff-cGFja2FnZXMvbWFjcm8vc3JjL21hY3JvSnMudHM=) | `91.11% <100.00%> (+0.06%)` | :arrow_up: |
| [packages/core/src/context.ts](https://codecov.io/gh/lingui/js-lingui/pull/1032/diff?src=pr&el=tree#diff-cGFja2FnZXMvY29yZS9zcmMvY29udGV4dC50cw==) | `82.35% <0.00%> (-3.65%)` | :arrow_down: |
| [packages/conf/src/index.ts](https://codecov.io/gh/lingui/js-lingui/pull/1032/diff?src=pr&el=tree#diff-cGFja2FnZXMvY29uZi9zcmMvaW5kZXgudHM=) | `80.76% <0.00%> (ø)` | |
| [...ackages/babel-plugin-extract-messages/src/index.ts](https://codecov.io/gh/lingui/js-lingui/pull/1032/diff?src=pr&el=tree#diff-cGFja2FnZXMvYmFiZWwtcGx1Z2luLWV4dHJhY3QtbWVzc2FnZXMvc3JjL2luZGV4LnRz) | `82.40% <0.00%> (+0.14%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/lingui/js-lingui/pull/1032?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/lingui/js-lingui/pull/1032?src=pr&el=footer). Last update [76553af...9ebae95](https://codecov.io/gh/lingui/js-lingui/pull/1032?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/packages/macro/src/macroJs.ts b/packages/macro/src/macroJs.ts
index 512b86a3..6b1e61e9 100644
--- a/packages/macro/src/macroJs.ts
+++ b/packages/macro/src/macroJs.ts
@@ -360,6 +360,9 @@ export default class MacroJs {
    * We clean '//\` ' to just '`'
    */
   clearBackslashes(value: string) {
+    // it's an unicode char so we should keep them
+    if (value.includes('\\u')) return value
+    // if not we replace the extra scaped literals
     return value.replace(removeExtraScapedLiterals, "`")
   }
 
diff --git a/packages/macro/src/macroJsx.ts b/packages/macro/src/macroJsx.ts
index 506d2e48..d7c93730 100644
--- a/packages/macro/src/macroJsx.ts
+++ b/packages/macro/src/macroJsx.ts
@@ -364,6 +364,9 @@ export default class MacroJSX {
    * We clean '//\` ' to just '`'
    * */
   clearBackslashes(value: string) {
+    // it's an unicode char so we should keep them
+    if (value.includes('\\u')) return value
+    // if not we replace the extra scaped literals
     return value.replace(removeExtraScapedLiterals, "`")
   }
 


Test patch:
diff --git a/packages/macro/src/macroJs.test.ts b/packages/macro/src/macroJs.test.ts
index 00957532..4e70dfc9 100644
--- a/packages/macro/src/macroJs.test.ts
+++ b/packages/macro/src/macroJs.test.ts
@@ -79,6 +79,55 @@ describe("js macro", () => {
         },
       ])
     })
+
+    it("message with unicode chars is respected", () => {
+      const macro = createMacro()
+      const exp = parseExpression('t`Message \\u0020`')
+      const tokens = macro.tokenizeTemplateLiteral(exp)
+      expect(tokens).toEqual([
+        {
+          type: "text",
+          value: 'Message \\u0020',
+        },
+      ])
+    })
+
+    it("message with double scaped literals it's stripped", () => {
+      const macro = createMacro()
+      const exp = parseExpression('t\`Passing \\`${argSet}\\` is not supported.\`')
+      const tokens = macro.tokenizeTemplateLiteral(exp)
+      expect(tokens).toEqual([
+        {
+          type: "text",
+          value: 'Passing `',
+        },
+        {
+          name: "argSet",
+          type: "arg",
+          value: {
+            end: 20,
+            loc:  {
+              end:  {
+                column: 20,
+                line: 1,
+              },
+              identifierName: "argSet",
+              start:  {
+                column: 14,
+                line: 1,
+              },
+            },
+            name: "argSet",
+            start: 14,
+            type: "Identifier",
+          },
+        },
+        {
+          type: "text",
+          value: "` is not supported.",
+        },
+      ])
+    })
   })
 
   describe("tokenizeChoiceMethod", () => {