My contributions to ChatCraft summary (v1.1.0 - v2.0.0)
Published: 2024-04-20 04:18:56

Recently, I contributed to the open source project [Chatcraft](https://github.com/tarasglek/chatcraft.org), and participated in the development of 10 [release](https://github.com/tarasglek/chatcraft.org/releases) versions from `v1.1.0` to `v2.0.0`. I would like to write a blog post to summarize my [contributions](https://github.com/tarasglek/chatcraft.org/commits/main/?author=mingming-ma).

## New Feature contributions

### 1. Enables users to upload images to ChatCraft and work with vision-related models (related release: `v1.2.0`, `v1.3.0`, `v1.4.0`, `v1.6.0`, `v1.9.0`, `v2.0.0`)

#### Release note:
`v1.2.0`: first release
`v1.3.0`: supports mobile
`v1.4.0`: adds image compression
`v1.6.0`: supports multiple image compression options
`v1.9.0`: removes workaround hardcode
`v2.0.0`: supports GPT-4 Turbo

#### Demonstration 

![first version](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hsh616s6wm6k2esosyea.gif)
(Please click on the GIF file to trigger the animation if it doesn't start automatically.)

Note, although the message time displayed is current, here is the `v1.2.0` commit branch being used to show the initial version functionality. 

The latest release `v2.0.0` is as follows, since the new model of `gpt-4-turbo` as of 04-09 also supports "Vision", I have adapted it accordingly.

![now version](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/39gyburn3bep9zn56e3c.gif)(Please click on the GIF file to trigger the animation if it doesn't start automatically.)

Users can also add images by using drag and drop and copy-paste methods.

| drag and drop | copy-paste |
| ------------- | ------------- |
|![drag](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ntzhcqwao15ic1k8rx7c.gif) | ![copypaste](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/54kk8naa8hdo2daeg2fc.gif) |
(Please click on the GIF file to trigger the animation if it doesn't start automatically.)

To implement this feature, I studied the newly emerged vision-related API from the [openai-node](https://github.com/openai/openai-node) project and integrated it into the codebase, added new UI components to display user images, and made modifications to the model, database, and optimization operations such as image compression. For example, the integration of new API can be found in the changes from original type `CreateChatCompletionRequestMessage` to new type `ChatCompletionMessageParam` as below:

```diff
@@ -90,16 +96,28 @@ export class ChatCraftMessage {
       date: this.date.toISOString(),
       type: this.type,
       text: this.text,
+      imageUrls: this.imageUrls,
     };
   }
 
-  toOpenAiMessage(): OpenAI.Chat.Completions.CreateChatCompletionRequestMessage {
+  toOpenAiMessage(model: ChatCraftModel): OpenAI.Chat.Completions.ChatCompletionMessageParam {
     const text = this.text;
+
+    const content: OpenAI.Chat.Completions.ChatCompletionContentPart[] = [{ type: "text", text }];
+    if (model.supportsImages && this.imageUrls.length > 0) {
+      this.imageUrls.forEach((imageUrl) => {
+        content.push({
+          type: "image_url",
+          image_url: { url: imageUrl },
+        });
+      });
+    }
+
     switch (this.type) {
       case "ai":
         return { role: "assistant", content: text };
       case "human":
-        return { role: "user", content: text };
+        return { role: "user", content };
       case "system":
         return { role: "system", content: text };
       case "function":
@@ -119,6 +137,7 @@ export class ChatCraftMessage {
       chatId,
       type: this.type,
       text: this.text,
+      imageUrls: this.imageUrls,
     };
   }
```

This new type suggests a more flexible structure that can accommodate multiple types of content within a single message. This is crucial for integrating capabilities for the image processing and handling, which are essential for models that support vision features.

The image compression function is completed client-side, which can reduce the cost of the user's API token. Users can make related settings in the settings modal.

![compression](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tdln2oi8wvt4jieh4v58.png)

As a side effect, I use it to do quick image compress.

This feature has been the biggest challenge I've faced in the open-source community so far. In my previous post, '[A 55-commit PR](https://dev.to/mingming-ma/a-55-commit-pr-5d7o),' I also wrote about my feelings after releasing. Thanks again to the ChatCraft community for their help and support.

### 2. Enables users to generate images with the DALL·E 3 model (related release:`v1.5.0`, `v1.7.0`, `v1.8.0`, `v2.0.0`)

Since the image generation model kind of independent than the language models, my approach is to use ChatCraft's command interaction, and I added an `/image` command, usage:
```text
/image [layout-option]<prompt> 
 ```
`layout=[l\|landscape\|p\|portrait]`

#### Release note:
`v1.5.0`: first release
`v1.7.0`: add loading info
`v1.8.0`: fix for issue: generated image not available after some time
`v2.0.0`: fix for issue: the loading spinner does not disappear when image generation fails


#### Demonstration 

![image generate](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/94ldk6p2lp9v8mqdqr5x.gif)
(Please click on the GIF file to trigger the animation if it doesn't start automatically.)

### 3. Supports rendering of mathematical expressions (LaTeX) (`v1.1.0`)

#### Demonstration 

![latex](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xmg7n6itp9s7w4tssaf7.png)

(a night mode example)

## Improved UI contribution

### Allows for the hiding and showing of long chat. (`v1.1.0`, `v1.2.0`)

It works great when the message is super long. During the development, I encountered a Minified React error issue. If you are interested, you can check my post '[Minified React error #426](https://dev.to/mingming-ma/minified-react-error-426-317k)' to see a possible solution. An interesting update in `v1.2.0` is we don't fold super long final message to give the user better experience.

#### Demonstration 
![showingmore](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wgaz4hlljyzfkr0fbgqc.gif)
(Please click on the GIF file to trigger the animation if it doesn't start automatically.)

## Bug fixes contributions

### 1.dollar symbol render conflict vs Latex rendering issue (`v1.1.0`)

| Bug  | Fix |
| ------------- | ------------- |
| ![bug](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vukfudare3rq37z7cexe.png)| ![fix](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ia0aw7xt10gy8ks3s4gm.png) |

### 2.no label code block rendering issue (`v1.1.0`)
| Bug  | Fix |
| ------------- | ------------- |
| ![bug](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f59tu3vwpjxd0gpxs3uf.png)| ![fix](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jw0h85hs7eb10p29ii67.png) |

### 3.Html no syntax highlight issue (`v1.2.0`)

| Bug  | Fix |
| ------------- | ------------- |
| ![bug](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jx0jbolv96g4ytzam4cc.png) | ![fix](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u5j82ykqddjkgjcf1eha.png)|

### 4.encode & in Markdown URLs issue (`v1.2.0`)

| Bug  | Fix |
| ------------- | ------------- |
| command failed, converting that to `&amp;`| ![fix](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ng5dnaqtkwtrz3nxxmvj.png)|

### 5.non-OpenAI model uses OpenAI logo (`v1.8.0`)

| Bug  | Fix |
| ------------- | ------------- |
| ![bug](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3w5y3f46u85uuyate87c.png)|![fix](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9yr7wntgk5ncujv7wrwj.png)|

### 6. Disable message editing on shared chats (`v1.9.0`)

| Bug  | Fix |
| ------------- | ------------- |
| ![bug](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9s6466vge9k6r712xrvx.png) | ![fix](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pgdaa8unpldo6mo1klgx.png) |


## Reflections

My contributions to ChatCraft from versions `v1.1.0` to `v2.0.0` have been both challenging and rewarding. Throughout this journey, I have been deeply involved in enhancing the functionality and user experience of ChatCraft. These contributions have not only improved the platform but have also provided me with invaluable learning experiences.

My work on ChatCraft has been a profound learning curve and a testament to the power of community and collaboration in the open-source space. I am grateful for the support from the ChatCraft community, which has been instrumental in overcoming challenges and achieving milestones. I look forward to engaging more with the community and continuing to contribute to the evolution of ChatCraft.