npm package: @khanacademy/mathjax-renderer
Description: Turns TeX into HTML+CSS with MathJax 3.2.2
Latest version: 3.0.2

# MathJax Renderer

This is the math renderer used in webapp as of November 2023. It relies on MathJax 3.2.2 to do the heavy lifting of converting TeX to HTML, accessible MathML, and CSS. Most of the code in this repo is "adapter" code that fixes up TeX in our content (originally written for KaTeX) so it works with MathJax. For additional documentation, please check out the Confluence page [MathJax at Khan Academy](https://khanacademy.atlassian.net/wiki/spaces/LC/pages/2525823154/MathJax+at+Khan+Academy).

## Installation

```bash
pnpm add @khanacademy/mathjax-renderer
```

## Usage

```js
import {MathJaxRenderer} from "@khanacademy/mathjax-renderer";
import "@khanacademy/mathjax-renderer/src/css/mathjax.css";

// if you want \text{} to look good on Safari:
import "@khanacademy/mathjax-renderer/src/css/safari-hacks.css";

// if you want copy-paste support:
import "@khanacademy/mathjax-renderer/src/css/selectable.css";

const renderer = new MathJaxRenderer({
    // shouldFixUnicodeLayout ensures that non-ASCII text is correctly
    // measured and positioned in e.g. \overbrace and \underbrace expressions.
    // Set shouldFixUnicodeLayout to false if you're rendering in an
    // environment without a layout engine, e.g. jsdom.
    shouldFixUnicodeLayout: true,
    fontURL: "https://cdn.kastatic.org/fonts/mathjax",
    // The locale being used to render the output
    locale: "en",
});

const {
    domElement, // the rendered result
    normalizedTex, // TeX with various syntax fixes
    error, // non-null if there was a problem rendering
} = renderer.render("c = \\sqrt{a^2 + b^2}");

// If you're rendering in a browser, be sure to call renderer.updateStyles()
// after each render() to update MathJax's CSS.
renderer.updateStyles();

// If you're rendering in not-a-browser (e.g. during SSR or static site
// generation), call renderer.getStyleSheet() after all expressions on the page
// have been rendered to generate the CSS.
const styleElement = `<style>${renderer.getStyleSheet().textContent}</style>`;
```

## Known Usages

-   [Webapp](https://github.com/Khan/webapp/blob/d952600066df0b769a87efd9eb588b23e5115b5a/services/static/package.json#L71)
-   [The Khan Academy mobile app webview](https://github.com/Khan/mobile/blob/0ee9012a607e2b892f9df3ae425c3c27378ff531/webview/javascript/package.json#L69)
-   [graphie-to-png](https://github.com/Khan/graphie-to-png) (via a Git submodule)
-   [Perseus in Webapp](https://github.com/Khan/webapp/blob/4030eeedeca3b8011f28e6281d277bc798557f7e/services/static/javascript/perseus/webapp-dependencies.ts#L162)

## Development

### One-time setup

<details>
<summary>show/hide</summary>

It's recommended to clone this repo with `ka-clone`, which installs Khan Academy's standard git hooks.
if you used plain `git clone` instead, don't worry. You can run `ka-clone --repair` at any time to
install the hooks.

You will need `node` and `pnpm` for development. If you're a KA engineer, you probably installed these when
setting up your computer, but if you don't have them, see [this guide](https://gist.github.com/benchristel/ede6615bf1c6cf91fd5cda70f02f57d7).

Versions used for initial development of this repo:

| Program | Recommended Version |
| ------- | ------------------- |
| `node`  | 20.19.0             |
| `pnpm`  | 10.9.0              |

Later versions will _probably_ work fine—but if you run into issues, just know that a version mismatch here _might_ be the cause.

Install dependencies and additional git hooks by running:

```
pnpm repo-setup
```

</details>

### Everyday Tools

This repo is configured for ergonomic development with **Visual Studio Code**.

```bash
pnpm dev          # run the dev UI
pnpm test:watch   # run the unit tests on each code change
pnpm test:all     # run unit and regression tests (slow!)
pnpm tsc          # run the typechecker
pnpm tsc -w       # run the typechecker on each code change
pnpm lint         # run eslint
pnpm verify       # run all checks
pnpm build        # update the dist/ folder
```

### Adding a Preprocessing Step

<details>
<summary>show/hide</summary>

In the preprocessing folder, create a file (ex. to fix a bug) titled something like `fix-bug.ts` and a second file titled `fix-bug.test.ts`

Create a function to replace specific TeX (found using regex) with a specific expression (ex below)

```
export function fixBug(tex: string): string {
    return tex.replace(/<regex>/g, "<new-tex>");
```

Create unit tests in the test file to confirm the processing step is working. Consider any potential edge cases that your fix might be tested against

```
import {describe, it, expect} from "@jest/globals";
import {fixBug} from "./fix-bug";

describe("fixBug", () => {
    it("fixes the bug", () => {
        const input = "<old-tex>";
        const fixed = "<new-tex>";
        expect(fixBug(input)).toBe(fixed);
    });
});
```

Go to the `index.ts` file and add the fix to the pipeline. It should not be necessary to add to `fixNonstandardSyntax` as all new math should be MathJax-compatible. As a result, future changes here should be added to `workAroundMathjaxDeficiences`. (Don’t forget to import the function at the top!)

You should almost always add a regression test (there is a how-to guide for this below too). The only time not to do a regression test is if the feature/fix cannot be covered by these kinds of regression tests (ex. purely visual changes via CSS). To add a regression test, go to the `regression-tests` folder and create a file titled something like `bug-fix.test.ts`

Here you will add some standardized code that is used for all the regression tests, but with a couple parts changed (indicated with angle brackets)

```
import {describe} from "@jest/globals";
import {testEachTexExpressionInFileIsRenderable} from "./test-generator";

describe("the <tex> command in MathJax", () => {
    testEachTexExpressionInFileIsRenderable({
        filename: "<bug-fix>_data.txt",
        separator: "\n\n\n",
    });
});
```

As you can see, data is needed. Create a file in that same folder titled `bug-fix_data.txt`. Conduct a regex search of all the content for the specific phrase in question. Add some of that to the file, depending on how many examples there are. The README has more info on this.

Once you add a new regression test, you will want to run all the tests and see if any of the other tests notice a regression. Check to make sure the errors are expected, then update the snapshots by passing `-u` to the `pnpm test` command. You may be able to update the tests in the IDE interface too.

Summary:

-   Create fix file and unit test file in preprocessing
-   Add fix to the pipeline in the index file
-   Move over to regression-tests and add a test file and data text file to watch for regressions
-   Run regression tests and update snapshots (`pnpm test -u <test-filename>`)
Here is an [example PR for adding a new preprocessing step](https://github.com/Khan/mathjax-renderer/pull/12)
</details>

### How to Add a New TeX Macro

<details>
<summary>show/hide</summary>

Head over to the `tex-input-config.ts` file in `src/ts`

Follow the format of the other macros there. `<Macro-name>: "<new-macro>",`

Head over to `renderer.test.ts` and add in the new macro for testing, following the syntax there

```
[`\\<Macro-name>`, "<description of how it looks>"]
```

Run the tests, check for anything unexpected in the results, and then update the snapshots

Here is an [example PR for adding a new TeX macro](https://github.com/Khan/mathjax-renderer/pull/9).

</details>

### Downloading Test Data

<details>
<summary>show/hide</summary>

It's useful to be able to test MathJax on actual math from our content.

Download a snapshot of English and Hindi content by running:

```bash
pnpm download-content-snapshot
```

The download will take about 15 minutes. Once it's done, run

```bash
pnpm extract-tex
```

This will scan the downloaded content for TeX expressions and write each expression to a file in `data/tex`. It takes about 30 minutes.

The download and extract steps are split up because sometimes, the download hangs at 99% (bug in `gsutil`, maybe?) and has to be manually killed.
If you encounter this bug, it's fine to kill the download, then go ahead and run `pnpm extract-tex` without re-running the download.

</details>

### Adding Regression Tests

<details>
<summary>show/hide</summary>

If you add features or fix bugs in the renderer, you'll probably want to add regression tests for them.
See `regression-tests/README.md` for details about what the regression tests cover and how they work.

#### Getting Data For Regression Tests

Assuming you have done all the "Downloading Test Data" steps above, you can search for TeX expressions matching a regex by running:

```bash
# NOTE: use single quotes for the regex! Double-quotes in bash use different escaping rules.
pnpm search-tex '\\begin\{align(ed)?\}' | pbcopy
```

The pbcopy command (on macOS only) puts the search results on your clipboard.

Each TeX expression in the results is annotated with the path of the Perseus JSON file it originally came from.

You can then paste the search results into a regression test data file. See the `regression-tests` folder for examples.

</details>

### Releasing

<details>
<summary>show/hide</summary>

The release process uses [@changesets/cli](https://npmjs.com/package/@changesets/cli).

The workflow is:

-   Each feature PR must include a changeset file, generated by running `pnpm changeset`, describing what is
    in that change and whether a major, minor, or patch version bump is required.
-   A GitHub workflow watches the `main` branch and updates a "Version Packages" PR whenever a feature PR is landed.
-   When the "Version Packages" PR is landed, a GitHub workflow updates the package version and changelog based on outstanding changesets, builds a new version of the package, and publishes it to NPM.

Landing the "Version Packages" PR is a manual action (you can use `git land` for this) but everything else should be automatic.

After releasing, please consider updating the [known usages](#known-usages) of this repo to the new release.

</details>

### Upgrading Usages

<details>
<summary>show/hide</summary>

#### Webapp

Once a new version is released, the version in webapp will need to be updated. This involves manually updating the version number of MathJax in Webapp's `package.json` file (services/static/package.json) and then running `pnpm install` in the `services/static` directory to update the `pnpm-lock.yaml` file.

Here is an example PR of bumping the version of MathJax in Webapp: [Update mathjax to newest version in webapp](https://github.com/Khan/webapp/pull/18417) (Does not have to be done as an audit as it is here)

It is possible there will be merge conflicts with the `package.json`. Make sure to confirm the version update is still present after resolving all conflicts!

#### Graphie Editor

The graphie editor consumes MathJax as a Git submodule. It is connected to a specific commit and does not update automatically, though updating it is similar to updating a regular git repo. Here is an approximate example of how to update it:

```
cd ~/Khan/graphie-to-png

# go into the submodule directory
cd mathjax-renderer

# download the latest commits from the mathjax-renderer repo
git fetch origin

# update to the desired version
git co vX.Y.Z

# re-pin the submodule to the new version
cd ..
git add mathjax-renderer
git commit
```

Example PR - [Switch the math renderer from KaTeX to MathJax](https://github.com/Khan/graphie-to-png/pull/21)

PR from creating a bundle for the editor. This is regularly done as part of the regular release, so nothing needs to be done regarding the bundle when bumping MathJax in the graphie repo - [Build to a single-file bundle, for graphie-to-png](https://github.com/Khan/mathjax-renderer/pull/11)

#### Mobile

This must be done as part of the mobile release cycle. The version is specified in `webview/javascript/package.json`. Update the version number there, install dependencies, and commit the changes.

Example PR - [Bumping the version of MathJax-Render in mobile](https://github.com/Khan/mobile/pull/3545/files)

</details>