Locally hosted AI writing assistant
Tags: ollama, webdev, javascript, ai
Published: 2024-02-10 14:10:53

Most artificial intelligence (AI) tools are either closed-source or require a server to function. However, with the help of Ollama, it is possible to create simple yet powerful local AI tools without any of these constraints. In this blog post, I will demonstrate how I built my own local AI tool using Ollama's user-friendly interface and flexible architecture. By doing so, I aim to showcase the ease with which one can leverage AI capabilities without relying on proprietary software or remote servers.

![An AI image of a bird robot writing](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9wn0ik9q7gw17nook4t1.jpg)
> An AI image of a bird robot writing

I recently developed an AI-powered writing assistant using Ollama. To build this innovative tool, I leveraged Vue.js and the Ollama JavaScript package, both of which proved to be indispensable in the development process. By harnessing the power of these advanced technologies, I was able to create a user-friendly interface that streamlines the writing process and yields high-quality content with ease.

> ###The solution only works on Linux, Windows Subsystem Linux (WSL), and maybe Mac

![A screenshot of AI writer showing how the first draft of this blog was made](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9lr4przwntkgm4xzj8gi.png)
> A screenshot of AI writer showing how the first draft of this blog was made.

## Integrate into JavaScript
Ollama will send the message to Llama2 model. The response can be returned with streaming, allowing for a seamless and interactive experience. By displaying the model's response in real-time, users will no longer have to look at loading bars, further enhancing their overall experience. This feature not only simplifies the programming process but also provides a more dynamic and engaging environment for all involved.
It will use a model named blog that I create in the next chapter.

```js
const output = ref(“”)
async function sendToModel(message: String) {
// Empty the output
output.value = ""


// Send the message to Ollama
// It will use the model named ‘blog’
 const response = await ollama.chat({
   model: 'blog',
   messages: [{ role: 'user', content:message }],
   stream: true
 })


// At every part of the stream back to the response in real time
 for await (const part of response) {
   output.value += part.message.content
 }
}
```

## Fine-tuning the model
Ollama's default model is Llama2, which is suitable for casual conversations. However, I wanted to modify it to generate rewritten versions of my initial ideas. To achieve this, I created a custom model file and made two key adjustments. Firstly, I lowered the temperature to 0.5, which results in less creative output that still conveys my intended message. Secondly, I modified the system message to avoid generating answers and instead focus on rewriting the input text. Below is the updated model file for your reference.

```
FROM llama2

# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 0.5

# set the system message
SYSTEM """
Rewrite the following into a paragraph of a technical blog with simple and formal English.
"""
```

To utilize the newly introduced model file, I executed the following command. This resulted in the generation of a freshly minted model named "blog." With this novel model at my disposal, I am now able to incorporate it into my JavaScript applications for seamless usage.
`ollama create blog -f ./Modelfile`

## Conclusion 
In conclusion, Ollama provides an accessible and user-friendly means of utilizing Large Language Models (LLMs) in local projects. The platform allows for seamless customization to meet individual needs and preferences, making it a valuable tool for those looking to leverage the power of LLMs in their projects. By providing an easy-to-use interface and a wide range of customization options, Ollama simplifies the process of incorporating LLMs into local projects, enabling users to unlock their full potential.