ChatCraft Adventures #8
Tags: beginners, opensource, openai, webdev
Published: 2024-03-09 02:59:59

This week in ChatCraft

## Release 1.4
After a short hiatus, ChatCraft release 1.4 has been released and is available [here](https://github.com/tarasglek/chatcraft.org/releases/tag/v1.4.0):
This week has been slow for me. I've been stuck on my Pull Requests described in [my previous blog post](https://dev.to/rjwignar/chatcraft-adventures-7-492i), and I haven't been able to contribute PR reviews or find new issues. I'm hoping to have these resolved quickly before they go stale, so I can devote more time to larger issues and features.

## Pull Requests

###  [Prompt Updating](https://github.com/tarasglek/chatcraft.org/pull/482)

In this Pull Request, I've updated the ChatCraft system prompt to return inline math in [KaTeX](https://katex.org/), like it currently does with math markdown in its own block:
```ts
- When responding with math, inline or otherwise, I ALWAYS use KaTeX syntax in math Markdown by enclosing EVERY mathematical expression, equation, variable, and formula with double-dollar signs \`($$)\`, for example: $$O(n\\log n)$$, $$1024 * 1024 = 1048576$$, $$1024^2$$, $$X$$
```

This updated is meant to solve the issue of ChatCraft returning inline math expressions in [LaTeX Math mode syntax](https://www.overleaf.com/learn/latex/Mathematical_expressions):
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wqna6rkeczx0uxkb92ee.png) (like `\(2^{20}\)` instead of `$$2^{20}$$`).

With the new prompt, the GPT-3.5-turbo LLM seems to prefer returning inline math markdown as KaTeX, as seen in this [conversation](https://issue-460.console-overthinker-dev.pages.dev/api/share/chatcraft_dev/Q5gq6_wbAfMeZeWvJ6Ab-).

However, before this can be merged, we're worried that my updated prompt might instruct the LLM to use KaTeX syntax when returning code, which we don't want. We want the prompt to instruct the LLM to use KaTeX only when returning math markdown, and not to mix KaTeX syntax when returning code. I've tried to mitigate this by including the clause "When responding with math..." to the rule, but whether an LLM will properly follow these instructions can be hard to test. I'm hoping to receive feedback from my instructor or even the open source community. 


Until working on this issue I never really considered using ChatCraft or ChatGPT for math, but I see the utility now.
I found that the updated prompt can address in-line Math Markdown, but it can fail when asking ChatCraft to describe scientific formulas, like the [gas laws](https://issue-460.console-overthinker-dev.pages.dev/api/share/chatcraft_dev/kHyRkBnSlCfxC3tyUjGTG) or the [Michaelis-Menten formula](https://issue-460.console-overthinker-dev.pages.dev/api/share/chatcraft_dev/XX-K1VWmOtdQCW75S0RRy). 

In these conversations, ChatCraft fails to return individual variables in KaTeX syntax:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6fm65389jbfx0a7kqlxr.png)


I originally had a second rule that fixes this issue:
```ts
- I NEVER wrap mathematical expressions, equations, variables, or formulas in round brackets, \`(\`, \`)\`. For example, instead of responding with (V_{max}), I would respond with $$V_{max}$$
```
...but I removed it because I was worried about introducing new ways for the LLM to give improper responses.
I'll need to clarify with the community whether this rule should be added back.

### [Dynamic Imports](https://github.com/tarasglek/chatcraft.org/pull/488)
This Pull Request involves dynamically importing a few large dependencies to reduce bundle size:
- elkjs (for Mermaid)
- cytoscape (for Mermaid)
- katex (for Math rendering)

In an older PR, I tried dynamically importing the modules that use these dependencies:
- mermaid
- rehype-katex
- remark-math

... but I found that dynamically importing rehype-katex and remark-math [crashes my preview app](https://rjwignar-issue-365.console-overthinker-dev.pages.dev/) due to a workbox error:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s5qhvs1vmu3lhxfeddsk.png)

In the new PR, I was able to dynamically import mermaid, but the problem is mermaid is almost always needed, so dynamically importing it doesn't help much. elkjs and cytoscape are never directly called in the source code (at least according to `git grep`), and are only invoked through mermaid (as dependencies). I'm not sure if there's another way to dynamically import only these dependencies of mermaid.

I'll need some help dynamically importing rehype-katex and remark-math.


## Plans for Release 1.5
In release 1.5, I hope to have these PRs revised and merged.
I'm actually thinking about postponing these PRs further so I can focus on a new feature I'd like to add, adding files to a ChatCraft conversation.