Repository: Khan/KaTeX
Pull request #950 (2017-10-24 20:16:20)

Support Unicode Symbols
This is the third in a series of PRs to give KaTeX the ability to recognize Unicode character input. This one supports some but not all letter-like symbols and some other symbols. All the characters in this PR will produce `textord` atoms in math mode.

The list is not comprehensive. It’s a bit eclectic. Some of these characters are important. Others were just easy to include. A few important characters are omitted because it wasn’t clear to me which is the best way to implement them. I’ll address those characters in future PRs.

ronkok: Double-struck characters cannot be done with current fonts by just setting a flag in `symbols.js`. Take ℂ, for example. In the katex-ams font, the character that looks like ℂ is not at code point U+2102 but is instead at code point U+0043. So, the `symbols.js` flag technique doesn’t work. Rather than wait for font revisions, I wrote macros.
ronkok: As in PR #933, the following table shows the KaTeX function to which each character is mapped and confirmation whether [unicode math](http://ctan.sharelatex.com/tex-archive/macros/latex/contrib/unicode-math/unicode-math-table.tex) and/or John Cook’s [conversion page](https://www.johndcook.com/unidata.js) map it the same way.

| **Character** | **Code point** | **Maps to**   | **unicode math** | **Cook** | **Other**       |
|:-------------:|:--------------:|:--------------|:----------------:|:--------:|:----------------|
|£  |  U+00A3  |  \pounds  |  \mathsterling  |  \sterling  |  \textsterling
|¥  |  U+00A5  |  \yen  |  \mathyen  |  ✓  |  
|ð  |  U+00F0  |  \eth  |  \matheth  |  \dh  |  
|ℂ  |  U+2102  |  \mathbb{C}  |  \BbbC  |  \BbbC  |  
|ℍ  |  U+210D  |  \mathbb{H}  |  \BbbH  |  \BbbH  |  
|ℑ  |  U+2111  |  \Im  |  ✓  |  ✓  |  \frakI
|ℓ  |  U+2113  |  \ell  |  ✓  |  ✓  |  
|ℕ  |  U+2115  |  \mathbb{N}  |  \BbbN  |  \BbbN  |  
|℘  |  U+2118  |  \wp  |  ✓  |  ✓  |  
|ℙ  |  U+2119  |  \mathbb{P}  |  \BbbP  |  \BbbP  |  
|ℚ  |  U+211A  |  \mathbb{Q}  |  \BbbQ  |  \BbbQ  |  
|ℜ  |  U+211C  |  \Re  |  ✓  |  ✓  |  \frakR
|ℝ  |  U+211D  |  \mathbb{R}  |  \BbbR  |  \BbbR  |  
|ℤ  |  U+2124  |  \mathbb{Z}  |  \BbbZ  |  \BbbZ  |  
|Ⅎ  |  U+2132  |  \Finv  |  ✓  |  ✓  |  
|ℵ  |  U+2135  |  \aleph  |  ✓  |  ✓  |  
|ℶ  |  U+2136  |  \beth  |  ✓  |  ✓  |  
|ℷ  |  U+2137  |  \gimel  |  ✓  |  ✓  |  
|ℸ  |  U+2138  |  \daleth  |  ✓  |  ✓  |  
|⅁  |  U+2141  |  \Game  |  ✓  |  ✓  |  
|∀  |  U+2200  |  \forall  |  ✓  |  ✓  |  
|∁  |  U+2201  |  \complement  |  ✓  |  ✓  |  
|∂  |  U+2202  |  \partial  |  ✓  |  ✓  |  
|∃  |  U+2203  |  \exists  |  ✓  |  ✓  |  
|∇  |  U+2207  |  \nabla  |  ✓  |  ✓  |  \del
|∞  |  U+221E  |  \infty  |  ✓  |  ✓  |  
|∠  |  U+2220  |  \angle  |  ✓  |  ✓  |  
|∡  |  U+2221  |  \measuredangle  |  ✓  |  ✓  |  
|∢  |  U+2222  |  \sphericalangle  |  ✓  |  \sphericalangle \ltrpar  |  
|♠  |  U+2660  |  \spadesuit  |  ✓  |  ✓  |  
|♡  |  U+2661  |  \heartsuit  |  ✓  |  ✓  |  
|♢  |  U+2662  |  \diamondsuit  |  ✓  |  ✓  |  
|♣  |  U+2663  |  \clubsuit  |  ✓  |  ✓  |  
|♭  |  U+266D  |  \flat  |  ✓  |  ✓  |  
|♮  |  U+266E  |  \natural  |  ✓  |  ✓  |  
|♯  |  U+266F  |  \sharp  |  ✓  |  ✓  |  
|✓  |  U+2713  |  \checkmark  |  ✓  |  ✓  |  

kevinbarabash: > Rather than wait for font revisions, I wrote macros.

I think that's sensible.  I think revising the fonts would introduce other complications as it relates to handling font commands such as `\mathbb`, etc.
ronkok: Let's hold off on this until after PR #973 lands. I'll revise this PR accordingly.
ronkok: The most recent commit reverts a RegEx in `MacroExpander.js`. So the code in this PR is now dependent on PR #973.  
ronkok: Now that PR #973 has landed, this should be good to go.

Patch:
diff --git a/src/macros.js b/src/macros.js
index f08c95fe..6944e87f 100644
--- a/src/macros.js
+++ b/src/macros.js
@@ -48,6 +48,15 @@ defineMacro("\\egroup", "}");
 defineMacro("\\begingroup", "{");
 defineMacro("\\endgroup", "}");
 
+// Unicode double-struck letters
+defineMacro("\u2102", "\\mathbb{C}");
+defineMacro("\u210D", "\\mathbb{H}");
+defineMacro("\u2115", "\\mathbb{N}");
+defineMacro("\u2119", "\\mathbb{P}");
+defineMacro("\u211A", "\\mathbb{Q}");
+defineMacro("\u211D", "\\mathbb{R}");
+defineMacro("\u2124", "\\mathbb{Z}");
+
 // We don't distinguish between math and nonmath kerns.
 // (In TeX, the mu unit works only with \mkern.)
 defineMacro("\\mkern", "\\kern");
diff --git a/src/symbols.js b/src/symbols.js
index b867ffe3..a8313774 100644
--- a/src/symbols.js
+++ b/src/symbols.js
@@ -108,22 +108,22 @@ defineSymbol(math, main, textord, "\u0023", "\\#");
 defineSymbol(text, main, textord, "\u0023", "\\#");
 defineSymbol(math, main, textord, "\u0026", "\\&");
 defineSymbol(text, main, textord, "\u0026", "\\&");
-defineSymbol(math, main, textord, "\u2135", "\\aleph");
-defineSymbol(math, main, textord, "\u2200", "\\forall");
+defineSymbol(math, main, textord, "\u2135", "\\aleph", true);
+defineSymbol(math, main, textord, "\u2200", "\\forall", true);
 defineSymbol(math, main, textord, "\u210f", "\\hbar");
-defineSymbol(math, main, textord, "\u2203", "\\exists");
-defineSymbol(math, main, textord, "\u2207", "\\nabla");
-defineSymbol(math, main, textord, "\u266d", "\\flat");
-defineSymbol(math, main, textord, "\u2113", "\\ell");
-defineSymbol(math, main, textord, "\u266e", "\\natural");
-defineSymbol(math, main, textord, "\u2663", "\\clubsuit");
-defineSymbol(math, main, textord, "\u2118", "\\wp");
-defineSymbol(math, main, textord, "\u266f", "\\sharp");
-defineSymbol(math, main, textord, "\u2662", "\\diamondsuit");
-defineSymbol(math, main, textord, "\u211c", "\\Re");
-defineSymbol(math, main, textord, "\u2661", "\\heartsuit");
-defineSymbol(math, main, textord, "\u2111", "\\Im");
-defineSymbol(math, main, textord, "\u2660", "\\spadesuit");
+defineSymbol(math, main, textord, "\u2203", "\\exists", true);
+defineSymbol(math, main, textord, "\u2207", "\\nabla", true);
+defineSymbol(math, main, textord, "\u266d", "\\flat", true);
+defineSymbol(math, main, textord, "\u2113", "\\ell", true);
+defineSymbol(math, main, textord, "\u266e", "\\natural", true);
+defineSymbol(math, main, textord, "\u2663", "\\clubsuit", true);
+defineSymbol(math, main, textord, "\u2118", "\\wp", true);
+defineSymbol(math, main, textord, "\u266f", "\\sharp", true);
+defineSymbol(math, main, textord, "\u2662", "\\diamondsuit", true);
+defineSymbol(math, main, textord, "\u211c", "\\Re", true);
+defineSymbol(math, main, textord, "\u2661", "\\heartsuit", true);
+defineSymbol(math, main, textord, "\u2111", "\\Im", true);
+defineSymbol(math, main, textord, "\u2660", "\\spadesuit", true);
 
 // Math and Text
 defineSymbol(math, main, textord, "\u2020", "\\dag");
@@ -251,11 +251,11 @@ defineSymbol(math, ams, textord, "\u25ca", "\\lozenge");
 defineSymbol(math, ams, textord, "\u24c8", "\\circledS");
 defineSymbol(math, ams, textord, "\u00ae", "\\circledR");
 defineSymbol(text, ams, textord, "\u00ae", "\\circledR");
-defineSymbol(math, ams, textord, "\u2221", "\\measuredangle");
+defineSymbol(math, ams, textord, "\u2221", "\\measuredangle", true);
 defineSymbol(math, ams, textord, "\u2204", "\\nexists");
 defineSymbol(math, ams, textord, "\u2127", "\\mho");
-defineSymbol(math, ams, textord, "\u2132", "\\Finv");
-defineSymbol(math, ams, textord, "\u2141", "\\Game");
+defineSymbol(math, ams, textord, "\u2132", "\\Finv", true);
+defineSymbol(math, ams, textord, "\u2141", "\\Game", true);
 defineSymbol(math, ams, textord, "\u006b", "\\Bbbk");
 defineSymbol(math, ams, textord, "\u2035", "\\backprime");
 defineSymbol(math, ams, textord, "\u25b2", "\\blacktriangle");
@@ -263,22 +263,24 @@ defineSymbol(math, ams, textord, "\u25bc", "\\blacktriangledown");
 defineSymbol(math, ams, textord, "\u25a0", "\\blacksquare");
 defineSymbol(math, ams, textord, "\u29eb", "\\blacklozenge");
 defineSymbol(math, ams, textord, "\u2605", "\\bigstar");
-defineSymbol(math, ams, textord, "\u2222", "\\sphericalangle");
-defineSymbol(math, ams, textord, "\u2201", "\\complement");
-defineSymbol(math, ams, textord, "\u00f0", "\\eth");
+defineSymbol(math, ams, textord, "\u2222", "\\sphericalangle", true);
+defineSymbol(math, ams, textord, "\u2201", "\\complement", true);
+// unicode-math maps U+F0 to \matheth. We map to AMS function \eth
+defineSymbol(math, ams, textord, "\u00f0", "\\eth", true);
 defineSymbol(math, ams, textord, "\u2571", "\\diagup");
 defineSymbol(math, ams, textord, "\u2572", "\\diagdown");
 defineSymbol(math, ams, textord, "\u25a1", "\\square");
 defineSymbol(math, ams, textord, "\u25a1", "\\Box");
 defineSymbol(math, ams, textord, "\u25ca", "\\Diamond");
-defineSymbol(math, ams, textord, "\u00a5", "\\yen");
-defineSymbol(math, ams, textord, "\u2713", "\\checkmark");
+// unicode-math maps U+A5 to \mathyen. We map to AMS function \yen
+defineSymbol(math, ams, textord, "\u00a5", "\\yen", true);
+defineSymbol(math, ams, textord, "\u2713", "\\checkmark", true);
 defineSymbol(text, ams, textord, "\u2713", "\\checkmark");
 
 // AMS Hebrew
-defineSymbol(math, ams, textord, "\u2136", "\\beth");
-defineSymbol(math, ams, textord, "\u2138", "\\daleth");
-defineSymbol(math, ams, textord, "\u2137", "\\gimel");
+defineSymbol(math, ams, textord, "\u2136", "\\beth", true);
+defineSymbol(math, ams, textord, "\u2138", "\\daleth", true);
+defineSymbol(math, ams, textord, "\u2137", "\\gimel", true);
 
 // AMS Greek
 defineSymbol(math, ams, textord, "\u03dd", "\\digamma");
@@ -436,8 +438,8 @@ defineSymbol(text, main, textord, "%", "\\%");
 defineSymbol(math, main, textord, "_", "\\_");
 defineSymbol(text, main, textord, "_", "\\_");
 defineSymbol(text, main, textord, "_", "\\textunderscore");
-defineSymbol(math, main, textord, "\u2220", "\\angle");
-defineSymbol(math, main, textord, "\u221e", "\\infty");
+defineSymbol(math, main, textord, "\u2220", "\\angle", true);
+defineSymbol(math, main, textord, "\u221e", "\\infty", true);
 defineSymbol(math, main, textord, "\u2032", "\\prime");
 defineSymbol(math, main, textord, "\u25b3", "\\triangle");
 defineSymbol(math, main, textord, "\u0393", "\\Gamma", true);
@@ -563,7 +565,7 @@ defineSymbol(math, ams, bin, "\u22bb", "\\veebar", true);
 defineSymbol(math, main, bin, "\u2299", "\\odot", true);
 defineSymbol(math, main, bin, "\u2295", "\\oplus", true);
 defineSymbol(math, main, bin, "\u2297", "\\otimes", true);
-defineSymbol(math, main, textord, "\u2202", "\\partial");
+defineSymbol(math, main, textord, "\u2202", "\\partial", true);
 defineSymbol(math, main, bin, "\u2298", "\\oslash", true);
 defineSymbol(math, ams, bin, "\u229a", "\\circledcirc", true);
 defineSymbol(math, ams, bin, "\u22a1", "\\boxdot", true);
@@ -669,7 +671,7 @@ defineSymbol(text, main, textord, "\u00b0", "\\degree");
 // TODO: In LaTeX, \pounds can generate a different character in text and math
 // mode, but among our fonts, only Main-Italic defines this character "163".
 defineSymbol(math, main, mathord, "\u00a3", "\\pounds");
-defineSymbol(math, main, mathord, "\u00a3", "\\mathsterling");
+defineSymbol(math, main, mathord, "\u00a3", "\\mathsterling", true);
 defineSymbol(text, main, mathord, "\u00a3", "\\pounds");
 defineSymbol(text, main, mathord, "\u00a3", "\\textsterling");
 defineSymbol(math, ams, textord, "\u2720", "\\maltese");


Test patch:
diff --git a/test/katex-spec.js b/test/katex-spec.js
index 52b26229..d30798b9 100644
--- a/test/katex-spec.js
+++ b/test/katex-spec.js
@@ -2678,6 +2678,10 @@ describe("Unicode", function() {
         expect("⊂⊃⊆⊇⊏⊐⊑⊒⊢⊣⊩⊪⊸⋈⋍⋐⋑⋔⋙⋛⋞⋟⌢⌣⩾⪆⪌⪕⪖⪯⪰⪷⪸⫅⫆").toParse();
     });
 
+    it("should parse symbols", function() {
+        expect("£¥ðℂℍℑℓℕ℘ℙℚℜℝℤℲℵℶℷℸ⅁∀∁∂∃∇∞∠∡∢♠♡♢♣♭♮♯✓").toParse();
+    });
+
     it("should parse arrows", function() {
         expect("←↑→↓↔↕↖↗↘↙↚↛↞↠↢↣↦↩↪↫↬↭↮↰↱↶↷↼↽↾↾↿⇀⇁⇂⇃⇄⇆⇇⇈⇉").toParse();
     });