Repository: gohugoio/hugo (language: go)

Issue:
markup/goldmark: Enable pass-through of raw content blocks
**Continuing the discussion from https://github.com/gohugoio/hugo/pull/10888 and [the forum](https://discourse.gohugo.io/t/katex-in-hugo/43274).** 


Hey everyone,

Following up on the PR, I wanted to open an issue to discuss an improvement to LaTeX support in Hugo. Standard LaTeX in Markdown support has been requested by the community for a while, especially among the scientific blogging community.  However, Hugo only supports a special syntax that requires modifying existing Markdown files to work with Hugo. Platforms like GitHub already support the standard dollar-sign syntax, e.g. that's an inline expression: $x^2$.

Here's the proposal:
- add a Goldmark extension to Hugo that finds all inline and block LaTeX in Markdown file and emit them as plain text. This can be enabled using a config flag and is disabled by default.
- With this extension, users can then include the Katex client side scripts and stylesheets, enabling them to render LaTeX without modifying their existing Markdown files.
- This proposal doesn't require any additional dependencies and doesn't involve server side rendering of the LaTeX expressions (which needs a JavaScript engine and is potentially slow)

This is already implemented here: https://github.com/janhuenermann/hugo-katex/commit/ad382464769740c73235cbc3b20c4a5143408aa2

Looking forward to hearing your thoughts on this!


Discussion / hints:
>LaTeX in Markdown file 

What is "LaTeX in Markdown"? If this is about adding yet another delimiter for this particular syntax, why not use a shortcode (for inline latex) and code fences for block latex?


> > LaTeX in Markdown file
> 
> What is "LaTeX in Markdown"? If this is about adding yet another delimiter for this particular syntax, why not use a shortcode (for inline latex) and code fences for block latex?

Hey @bep,

Thanks for your reply! When I mentioned "LaTeX in Markdown," I was referring to the ability to include standard LaTeX expressions directly within Markdown files using the familiar dollar-sign syntax (single $ for inline and double $$ for block expressions). This is the standard syntax in [Jekyll](https://www.fabriziomusacchio.com/blog/2021-08-10-How_to_use_LaTeX_in_Markdown/), [Jupyter notebooks](https://stackoverflow.com/questions/13208286/how-to-write-latex-in-ipython-notebook),  [VSCode Markdown extension](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one#math), [on platforms like GitHub](https://github.blog/2022-05-19-math-support-in-markdown/), and more, making it a widely recognized and expected feature for many users. Shortcodes however require to change Markdown files for Hugo, which is not only inconvenient but prevents users from reusing the same Markdown files across projects.

There's an in-depth discussion about this topic in the forum as well: https://discourse.gohugo.io/t/katex-in-hugo/43274


Miscellaneous thoughts...

#### Display mode (blocks)

This syntax is supported by GitHub:

````
```math
\sqrt{3}  # with or without delimiters
```
````

This syntax is supported by GitLab:

````
```math
\sqrt{3}  # must not include delimeters
```
````

The fenced code block syntax is consistent with Mermaid diagram support by both services:

````
```mermaid
xxx
```
````

#### Alternate delimiters

KaTeX and MathJax support alternate delimiters:

```text
\\[ \\] # display mode
\\( \\) # inline
```

These are not supported by either GitHub or GitLab. Markdown in the wild using this syntax is not portable to either service.

#### Inline mode

With Hugo you must use a shortcode. The resulting markdown is not portable.

#### Minor advantage of fenced code blocks and shortcodes

JS and CSS can be loaded as needed (per page) without requiring a front matter flag (i.e., `.HasShortcode`, `.Page.Store`).

#### Alternative (opinion)

For diagrams, I prefer to use the free [Kroki](https://kroki.io/) service because it embeds an SVG---no client side rendering.

For math, I prefer to use the free [Math API](https://math.vercel.app/) service because it embeds an SVG---no client side rendering. The fenced code block syntax is portable; the shortcode syntax is not portable.





I am sorry, if I do not understand the proposal. Does Hugo already support for `$` for inline math and `$$` for block math using KateX? I believe I have used it for a while and it is already very convenient. The missing is only refering the equation feature, but it is unsupported by KaTeX and not Hugo.
> I am sorry, if I do not understand the proposal. Does Hugo already support for `$` for inline math and `$$` for block math using KateX?

Hugo itself does not recognize math stuff delimited by `$` and `$$`. More specifically, it does not register `$` `$$` AST nodes with the underlying markdown parser (e.g. goldmark), like what [goldmark-mathjax](https://github.com/litao91/goldmark-mathjax) and #7435 are doing. 

> I believe I have used it for a while and it is already very convenient. The missing is only refering the equation feature, but it is unsupported by KaTeX and not Hugo.

Themes shipped with MathJax support (KaTeX or something else) will *probably* display `$` `$$` math correctly.

But as a result of what I said before, taking `$a^*=x-b^*$` for example:
* should be rendered as $a^\star=x-b^\star$ if `$a^*=x-b^*$` appears in raw HTML and is captured by JavaScript math engines
* but Hugo produces `$a^<em>=x-b^</em>$` and math engines just won't capture, you should've guessed how it comes

Also brings similar problems mentioned in #6694 #6864 #7249. These issues will be solved cleanly once `$` `$$` blocks are recognized by Hugo.
Here is a brief survey on the status quo of different math block support among popular Markdown editors:

| Editor / Platform    | ` ```math ` display | `$...$` inline             | `$$...$$` display | `\(` inline | `\[` display |
|--------------------  |---------------------|----------------------------|-------------------|-------------|--------------|
| GitHub               | Yes                 | Yes (bugged<sup>1</sup>)   | Yes (bugged)      | No          | No           |
| GitLab               | Yes                 | No                         | No                | No          | No           |
| VSCode (w/o plugins) | No                  | Yes                        | Yes               | No          | No           |
| Typora               | No                  | Yes (disabled by default)  | Yes               | No          | No           |
| StackEdit            | No                  | Yes                        | Yes               | No          | No           |
| MarkText             | No                  | Yes                        | Yes               | No          | No           |

Note 1: GitHub markdown preview is also bugged, which does not treat contents inside `$` `$$` as a block. Stuff like `$\{a\}$` also won't work.

`$` `$$` and other stuff mess Markdown a lot at first, but by inspecting the column, dollars are de facto delimiters for math blocks now in popular Markdown editors. If Hugo supports dollar signs (or even make the delimiter configurable), it will increase much in interopability.
I think it'll be better if there's an way to configure the delimiters since `\(...\)` and `\[...\]` seems better than `$...$` and `$$...$$`.

Reference
- [Are \\( and \\) preferable to dollar signs for math mode?](https://tex.stackexchange.com/questions/510/are-and-preferable-to-dollar-signs-for-math-mode)
- [Why is \\[ ... \\] preferable to $$ ... $$?](https://tex.stackexchange.com/questions/503/why-is-preferable-to)
I don't understand this issue. I have LaTeX working properly in Hugo by loading Matjax JS. 

In a `script.html` partial, going at the end of the page, I have : 

```
{{ if .Params.latex}}
  <script>
    MathJax = {
        packages: {'[+]': ['autoload', 'require']},
        tex: {
          tags: 'all',
          inlineMath: [ ['$','$'] ],
          displayMath: [ ['$$','$$'] ],
          processEscapes: true,
          processEnvironments: true,
          processRefs: true,
        },
        svg: {
          mtextInheritFont: true,
          merrorInheritFont: true,
          mathmlSpacing: false,
          skipAttributes: {},
          exFactor: .5,
          displayAlign: 'center',
          displayIndent: '0',
          fontCache: 'global',
          localID: null,
          internalSpeechTitles: true,
          titleID: 0
        },
        options: {
          ignoreHtmlClass: 'no_math',//  class that marks tags not to search
          processHtmlClass: 'math',  //  class that marks tags that should be searched
        }
    };
  </script>
  <script type="text/javascript" id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-svg.js"></script>
{{ end }}
```

Note that the config script lets you decide what delimiters you want. Then, on the `.md` files, I write typical LaTeX equations, using the dollar syntax. In the frontmatter of pages using maths, I set `latex: true` to avoid loading the heavy MathJax lib everywhere if it's not needed. After that, equations are rendered client-side with no issue.
@aurelienpierre 

This will fail:

```text
$$
\begin{array} {lcl}
  L(p,w_i) &=& \dfrac{1}{N}\Sigma_{i=1}^N(\underbrace{f_r(x_2
  \rightarrow x_1
  \rightarrow x_0)G(x_1
  \longleftrightarrow x_2)f_r(x_3
  \rightarrow x_2
  \rightarrow x_1)}_{sample\, radiance\, evaluation\, in\, stage2}
  \\\\\\ &=&
  \prod_{i=3}^{k-1}(\underbrace{\dfrac{f_r(x_{i+1}
  \rightarrow x_i
  \rightarrow x_{i-1})G(x_i
  \longleftrightarrow x_{i-1})}{p_a(x_{i-1})}}_{stored\,in\,vertex\, during\,light\, path\, tracing\, in\, stage1})\dfrac{G(x_k
  \longleftrightarrow x_{k-1})L_e(x_k
  \rightarrow x_{k-1})}{p_a(x_{k-1})p_a(x_k)})
\end{array}
$$
```

Ampersands are converted `&amp;` and paired underscores create `em` elements. And that is the correct rendering behavior when converting markdown to HTML.
Right, now that you mention it, I remember that I had to extend PHPMarkdown to discard LaTeX in order to handle those corner cases for [WordPress](https://github.com/aurelienpierre/wp-scholar/blob/main/classes/MarkdownExtraCustom.php). 
@bep @jmooring 
Author of the goldmark here. I've created a new (currently experimental) extension : [goldmark-dynamic](https://github.com/yuin/goldmark-dynamic).
This extension allows you to write a goldmark extension in Lua and load it at runtime without re-compilation.
This extension may be worth to evaluate for this kind of feature requests(want to add a xxx goldmark extension).

A big +1 here: I had tried to migrate my enormous Wordpress math blog to Hugo five years ago (see https://github.com/gohugoio/hugo/issues/5431), and LaTeX had a similar problem with the old markdown parser (blackfriday). This was the main blocker that stopped my migration.

The original comment in this issue https://github.com/gohugoio/hugo/issues/10894#issue-1665121600 has an implementation. Is there any opposition to refreshing it for the current build and opening a PR? If @janhuenermann isn't able to do it I can give it a swing.

Fix patch:
diff --git a/go.mod b/go.mod
index 5a7af842bf8..dd544ba4096 100644
--- a/go.mod
+++ b/go.mod
@@ -35,6 +35,7 @@ require (
 	github.com/gobuffalo/flect v1.0.2
 	github.com/gobwas/glob v0.2.3
 	github.com/gohugoio/go-i18n/v2 v2.1.3-0.20230805085216-e63c13218d0e
+	github.com/gohugoio/hugo-goldmark-extensions/passthrough v0.1.0
 	github.com/gohugoio/locales v0.14.0
 	github.com/gohugoio/localescompressed v1.0.1
 	github.com/gohugoio/testmodBuilder/mods v0.0.0-20190520184928-c56af20f2e95
diff --git a/go.sum b/go.sum
index d461f12c35d..305d2aa9dbc 100644
--- a/go.sum
+++ b/go.sum
@@ -166,6 +166,8 @@ github.com/gobwas/glob v0.2.3 h1:A4xDbljILXROh+kObIiy5kIaPYD8e96x1tgBhUI5J+Y=
 github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8=
 github.com/gohugoio/go-i18n/v2 v2.1.3-0.20230805085216-e63c13218d0e h1:QArsSubW7eDh8APMXkByjQWvuljwPGAGQpJEFn0F0wY=
 github.com/gohugoio/go-i18n/v2 v2.1.3-0.20230805085216-e63c13218d0e/go.mod h1:3Ltoo9Banwq0gOtcOwxuHG6omk+AwsQPADyw2vQYOJQ=
+github.com/gohugoio/hugo-goldmark-extensions/passthrough v0.1.0 h1:oFQ3f1M3Ook6amHmbqVu/uBRrQ6yjMDFkIv4HQr0f1Y=
+github.com/gohugoio/hugo-goldmark-extensions/passthrough v0.1.0/go.mod h1:g9CCh+Ci2IMbPUrVJuXbBTrA+rIIx5+hDQ4EXYaQDoM=
 github.com/gohugoio/locales v0.14.0 h1:Q0gpsZwfv7ATHMbcTNepFd59H7GoykzWJIxi113XGDc=
 github.com/gohugoio/locales v0.14.0/go.mod h1:ip8cCAv/cnmVLzzXtiTpPwgJ4xhKZranqNqtoIu0b/4=
 github.com/gohugoio/localescompressed v1.0.1 h1:KTYMi8fCWYLswFyJAeOtuk/EkXR/KPTHHNN9OS+RTxo=
diff --git a/markup/goldmark/convert.go b/markup/goldmark/convert.go
index 6ebcd8a7779..56cc56fcdd4 100644
--- a/markup/goldmark/convert.go
+++ b/markup/goldmark/convert.go
@@ -19,6 +19,7 @@ import (
 
 	"github.com/gohugoio/hugo/identity"
 
+	"github.com/gohugoio/hugo-goldmark-extensions/passthrough"
 	"github.com/gohugoio/hugo/markup/goldmark/codeblocks"
 	"github.com/gohugoio/hugo/markup/goldmark/goldmark_config"
 	"github.com/gohugoio/hugo/markup/goldmark/images"
@@ -154,6 +155,33 @@ func newMarkdown(pcfg converter.ProviderConfig) goldmark.Markdown {
 		extensions = append(extensions, c)
 	}
 
+	if cfg.Extensions.Passthrough.Enable {
+		configuredInlines := cfg.Extensions.Passthrough.Delimiters.Inline
+		configuredBlocks := cfg.Extensions.Passthrough.Delimiters.Block
+
+		inlineDelimiters := make([]passthrough.Delimiters, len(configuredInlines))
+		blockDelimiters := make([]passthrough.Delimiters, len(configuredBlocks))
+
+		for i, d := range configuredInlines {
+			inlineDelimiters[i] = passthrough.Delimiters{
+				Open:  d[0],
+				Close: d[1],
+			}
+		}
+
+		for i, d := range configuredBlocks {
+			blockDelimiters[i] = passthrough.Delimiters{
+				Open:  d[0],
+				Close: d[1],
+			}
+		}
+
+		extensions = append(extensions, passthrough.NewPassthroughWithDelimiters(
+			inlineDelimiters,
+			blockDelimiters,
+		))
+	}
+
 	if pcfg.Conf.EnableEmoji() {
 		extensions = append(extensions, emoji.Emoji)
 	}
diff --git a/markup/goldmark/goldmark_config/config.go b/markup/goldmark/goldmark_config/config.go
index cdfb4e7cc00..ba1874a184b 100644
--- a/markup/goldmark/goldmark_config/config.go
+++ b/markup/goldmark/goldmark_config/config.go
@@ -49,6 +49,13 @@ var Default = Config{
 			EastAsianLineBreaksStyle: "simple",
 			EscapedSpace:             false,
 		},
+		Passthrough: Passthrough{
+			Enable: false,
+			Delimiters: DelimitersConfig{
+				Inline: [][]string{},
+				Block:  [][]string{},
+			},
+		},
 	},
 	Renderer: Renderer{
 		Unsafe: false,
@@ -75,6 +82,7 @@ type Extensions struct {
 	Typographer    Typographer
 	Footnote       bool
 	DefinitionList bool
+	Passthrough    Passthrough
 
 	// GitHub flavored markdown
 	Table           bool
@@ -112,6 +120,28 @@ type Typographer struct {
 	Apostrophe string
 }
 
+// Passthrough hold passthrough configuration.
+// github.com/hugoio/hugo-goldmark-extensions/passthrough
+type Passthrough struct {
+	// Whether to enable the extension
+	Enable bool
+
+	// The delimiters to use for inline and block passthroughs.
+	Delimiters DelimitersConfig
+}
+
+type DelimitersConfig struct {
+	// The delimiters to use for inline passthroughs. Each entry in the list
+	// is a size-2 list of strings, where the first string is the opening delimiter
+	// and the second string is the closing delimiter, e.g.,
+	//
+	// [["$", "$"], ["\\(", "\\)"]]
+	Inline [][]string
+
+	// The delimiters to use for block passthroughs. Same format as Inline.
+	Block [][]string
+}
+
 type CJK struct {
 	// Whether to enable CJK support.
 	Enable bool


Test patch:
diff --git a/markup/goldmark/integration_test.go b/markup/goldmark/integration_test.go
index a2bf1637489..c9c6ef33864 100644
--- a/markup/goldmark/integration_test.go
+++ b/markup/goldmark/integration_test.go
@@ -711,3 +711,103 @@ echo "hello";
 	b := hugolib.Test(t, files)
 	b.AssertFileContent("public/index.html", "<div class=foo><?php\necho \"hello\";\n?>\n</div>")
 }
+
+// Issue #10894
+func TestPassthroughInlineFences(t *testing.T) {
+	t.Parallel()
+
+	files := `
+-- config.toml --
+[markup.goldmark.extensions.passthrough]
+enable = true
+[markup.goldmark.extensions.passthrough.delimiters]
+inline = [['$', '$'], ['\(', '\)']]
+-- content/p1.md --
+---
+title: "p1"
+---
+## LaTeX test
+
+Inline equation that would be mangled by default parser: $a^*=x-b^*$
+
+-- layouts/_default/single.html --
+{{ .Content }}
+`
+
+	b := hugolib.Test(t, files)
+	b.AssertFileContent("public/p1/index.html", `
+		$a^*=x-b^*$
+	`)
+}
+
+func TestPassthroughBlockFences(t *testing.T) {
+	t.Parallel()
+
+	files := `
+-- config.toml --
+[markup.goldmark.extensions.passthrough]
+enable = true
+[markup.goldmark.extensions.passthrough.delimiters]
+block = [['$$', '$$']]
+-- content/p1.md --
+---
+title: "p1"
+---
+## LaTeX test
+
+Block equation that would be mangled by default parser:
+
+$$a^*=x-b^*$$
+
+-- layouts/_default/single.html --
+{{ .Content }}
+`
+
+	b := hugolib.Test(t, files)
+	b.AssertFileContent("public/p1/index.html", `
+		$$a^*=x-b^*$$
+	`)
+}
+
+func TestPassthroughWithAlternativeFences(t *testing.T) {
+	t.Parallel()
+
+	files := `
+-- config.toml --
+[markup.goldmark.extensions.passthrough]
+enable = true
+[markup.goldmark.extensions.passthrough.delimiters]
+inline = [['(((', ')))']]
+block = [['%!%', '%!%']]
+-- content/p1.md --
+---
+title: "p1"
+---
+## LaTeX test
+
+Inline equation that would be mangled by default parser: (((a^*=x-b^*)))
+Inline equation that should be mangled by default parser: $a^*=x-b^*$
+
+Block equation that would be mangled by default parser:
+
+%!%
+a^*=x-b^*
+%!%
+
+-- layouts/_default/single.html --
+{{ .Content }}
+`
+
+	b := hugolib.Test(t, files)
+	b.AssertFileContent("public/p1/index.html", `
+		(((a^*=x-b^*)))
+	`)
+	b.AssertFileContent("public/p1/index.html", `
+		$a^<em>=x-b^</em>$
+	`)
+	b.AssertFileContent("public/p1/index.html", `
+%!%
+a^*=x-b^*
+%!%
+	`)
+}