Repository: zulip/zulip-flutter (language: dart)

Issue:
TeX in the message list
Zulip's Markdown flavor supports TeX for mathematical notation, using [KaTeX](https://katex.org/). This will be one of the more complex message-content features for us to support.

The way this works for web and for the existing RN-based mobile client is:
* KaTeX consists of some JavaScript code to convert TeX into HTML, and then some CSS to style that HTML. The KaTeX-generated HTML also has some per-element style attributes.
* The server takes care of the conversion to HTML, by running the KaTeX JS code on Node. That HTML goes into the message's HTML content.
* Then the web and RN-mobile clients ship with KaTeX's CSS, and include that in their styles.
* Those clients also ship with the KaTeX JS code. They use that for local echo.

What I think we'll want to do in Flutter is:
* Skip TeX for local echo, and just show the source TeX. That seems fine.
  * If we implement a "preview" feature for before sending a message, then it'll be important to get the actual rendering there… but even the web app relies on the server for its preview feature. So we'll do the same.
* Then we'll consume KaTeX HTML and convert it into Flutter widgets, just like we consume other kinds of Zulip content HTML and convert it into Flutter widgets, in `lib/model/content.dart` and `lib/widgets/content.dart`.

Converting the HTML to Flutter widgets will involve having code in `lib/widgets/content.dart` that is written to correspond to the KaTeX CSS, just like much of the existing code there is written to correspond to the Zulip markdown-content CSS.

That CSS is [a few hundred lines of code](https://github.com/KaTeX/KaTeX/blob/c5ac4c769588761fb68d89890998234e83fc1c1a/src/katex.less). It's written in Less; but looking at the compiled output, it's not much longer than that source, and most of the difference is highly repetitive. So the real amount of information there is about what's in that one source file. That file is also highly stable over time — just as one might hope, given that TeX itself is famously extremely stable — with the last non-NFC change 20 months ago, in 2021-07.

So while this won't be a small subproject, it's one where I'm pretty confident we can do it well.



Discussion hints:
We got a request for this as beta feedback in the Google Play Console:

> Looks like there's no Latex support yet. This makes it a non-starter for our purposes unfortunately. Are there plans to bring that in soon?
And again today:

> Math doesn't work, glitches with links. I'll try again another time (love Zulip generally).
In the Lean community we [asked](https://leanprover.zulipchat.com/#narrow/channel/236604-Zulip-meta/topic/LaTeX.20display.20in.20mobile.20app/near/489857078) how disappointed people would be if the app were rolled out to all users before full support for TeX.

One user answered "heavily disappointed"; others reacted to that message suggesting they disagree.

A different user "wouldn't mind" the current state, and another added:
> I mean, I'm old enough to remember when people wrote latex in `sci.math.research` in their posts and it was just displayed like that anyway.

See discussion with an [idea](https://chat.zulip.org/#narrow/channel/48-mobile/topic/TeX.20formatting.20.23F46/near/2048201) from Tim:
> So I think it would be possible to write the Flutter app logic to convert MathML into display rather than working with the HTML.
It's looking like we won't have all the features of KaTeX covered before we launch the new app in a few weeks ([#mobile-team > Flutter beta timeline @ 💬](https://chat.zulip.org/#narrow/channel/243-mobile-team/topic/Flutter.20beta.20timeline/near/2174273)). I think we can still cover most usage of it, though, with a respectable fallback behavior. Then we can pursue the remaining KaTeX features as follow-up issues.

Details in chat: [#mobile > TeX formatting #F46 @ 💬](https://chat.zulip.org/#narrow/channel/48-mobile/topic/TeX.20formatting.20.23F46/near/2182738)


Pull request: Render KaTeX by default
Fixes #46. (By enabling the TeX support whose substance was added over a number of preceding PRs.)

Here's a version of the commit I've included in each release recently, atop (previous versions of) #1559, to enable our KaTeX support. Now that #1559 and its preceding PRs (#1698, and others before that) are merged to main, this is ready to go to main too.

This was a minimal version for the release, just flipping the flag's default to true. But in general we should remove experimental flags from the codebase as soon as they're no longer relevant, to keep complexity from building up. (See the doc on [GlobalSettingType.experimentalFeatureFlag].) @rajveermalviya would you write a follow-up commit that removes this flag entirely, and cleans up the logic elsewhere which that makes possible to clean up?

#### 5287d39b9 content: Render KaTeX by default, the way we have in recent releases

This is cherry-picked from commits we included in each of the last
several releases, starting with 8f3723768 in v0.0.31.


Test command: flutter test test/model/content_test.dart test/widgets/content_test.dart --reporter expanded 2>&1

Fix patch:
diff --git a/lib/model/settings.dart b/lib/model/settings.dart
index e8a309ac29..4e71224444 100644
--- a/lib/model/settings.dart
+++ b/lib/model/settings.dart
@@ -178,7 +178,7 @@ enum BoolGlobalSetting {
   upgradeWelcomeDialogShown(GlobalSettingType.internal, false),
 
   /// An experimental flag to toggle rendering KaTeX content in messages.
-  renderKatex(GlobalSettingType.experimentalFeatureFlag, false),
+  renderKatex(GlobalSettingType.experimentalFeatureFlag, true),
 
   /// An experimental flag to enable rendering KaTeX even when some
   /// errors are encountered.


Test patch:
diff --git a/test/model/content_test.dart b/test/model/content_test.dart
index e4dd622b19..88bd11c66c 100644
--- a/test/model/content_test.dart
+++ b/test/model/content_test.dart
@@ -512,7 +512,7 @@ class ContentExample {
   static final mathInline = ContentExample.inline(
     'inline math',
     r"$$ \lambda $$",
-    expectedText: r'\lambda',
+    expectedText: r'λ',
     '<p><span class="katex">'
       '<span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>λ</mi></mrow>'
         '<annotation encoding="application/x-tex"> \\lambda </annotation></semantics></math></span>'
@@ -532,7 +532,7 @@ class ContentExample {
   static const mathBlock = ContentExample(
     'math block',
     "```math\n\\lambda\n```",
-    expectedText: r'\lambda',
+    expectedText: r'λ',
     '<p><span class="katex-display"><span class="katex">'
       '<span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>λ</mi></mrow>'
         '<annotation encoding="application/x-tex">\\lambda</annotation></semantics></math></span>'
diff --git a/test/widgets/content_test.dart b/test/widgets/content_test.dart
index 964d7e1208..46ed10079e 100644
--- a/test/widgets/content_test.dart
+++ b/test/widgets/content_test.dart
@@ -558,7 +558,11 @@ void main() {
   group('MathBlock', () {
     testContentSmoke(ContentExample.mathBlock);
 
-    testWidgets('displays KaTeX source; experimental flag default', (tester) async {
+    testWidgets('displays KaTeX source; experimental flag disabled', (tester) async {
+      addTearDown(testBinding.reset);
+      final globalSettings = testBinding.globalStore.settings;
+      await globalSettings.setBool(BoolGlobalSetting.renderKatex, false);
+
       await prepareContent(tester, plainContent(ContentExample.mathBlock.html));
       tester.widget(find.text(r'\lambda', findRichText: true));
     });
@@ -1102,6 +1106,23 @@ void main() {
     });
 
     testWidgets('maintains font-size ratio with surrounding text, when showing TeX source', (tester) async {
+      const html = '<span class="katex">'
+        '<span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>λ</mi></mrow>'
+          '<annotation encoding="application/x-tex"> \\lambda </annotation></semantics></math></span>'
+        '<span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal">λ</span></span></span></span>';
+      await checkFontSizeRatio(tester,
+        targetHtml: html,
+        targetFontSizeFinder: mkTargetFontSizeFinderFromPattern(r'λ'));
+    }, skip: true // TODO(#46): adapt this test
+                  //   (it needs a more complex targetFontSizeFinder;
+                  //    see other uses in this file for examples.)
+    );
+
+    testWidgets('maintains font-size ratio with surrounding text, when showing TeX source', (tester) async {
+      addTearDown(testBinding.reset);
+      final globalSettings = testBinding.globalStore.settings;
+      await globalSettings.setBool(BoolGlobalSetting.renderKatex, false);
+
       const html = '<span class="katex">'
         '<span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>λ</mi></mrow>'
           '<annotation encoding="application/x-tex"> \\lambda </annotation></semantics></math></span>'
@@ -1111,7 +1132,11 @@ void main() {
         targetFontSizeFinder: mkTargetFontSizeFinderFromPattern(r'\lambda'));
     });
 
-    testWidgets('displays KaTeX source; experimental flag default', (tester) async {
+    testWidgets('displays KaTeX source; experimental flag disabled', (tester) async {
+      addTearDown(testBinding.reset);
+      final globalSettings = testBinding.globalStore.settings;
+      await globalSettings.setBool(BoolGlobalSetting.renderKatex, false);
+
       await prepareContent(tester, plainContent(ContentExample.mathInline.html));
       tester.widget(find.text(r'\lambda', findRichText: true));
     });