Introducing Jan
Tags: llm, janai, opensource, genai
Published: 2024-05-05 17:57:36

I really like LM Studio and its capability to simplify the utilization of local models on my personal computer. The project is ever-evolving, supporting an expanding range of models and features. However, as [I noted several months ago](https://dev.to/worldlinetech/introducing-lm-studio-54f1), it is not open source.

As we continue this blog series, let's explore a **fully open-source alternative to LM Studio** - [Jan](https://jan.ai), a project from Southeast Asia. 

![Jan Logo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5l5btjaaeix67yix6575.png)

## Jan: A Highly Versatile, Open Platform

Jan's slogan is *Turn your computer into an AI computer*. This solution, designed to optimize any device’s performance - be it PCs or multi-GPU clusters, is available as a desktop version for MacOS, Windows, and Linux. Furthermore, it can also be  deployed as a server on public Cloud or on premises. 

This platform allows you to run LLMs locally and offline, or connect to remote GenAI APIs. Jan's philosophical principles are rooted in [*local-first*](https://www.inkandswitch.com/local-first/) and [*file over app*](https://stephango.com/file-over-app) approaches. 

![Nitro Logo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3fewlysptd2i7nscegk7.png)

Jan incorporates a lightweight, built-in inference server called [Nitro](https://nitro.jan.ai/). Nitro supports both llama.cpp and NVIDIA's TensorRT-LLM engines. This means many open LLMs in the **GGUF format** are supported. Jan's *Model Hub* is designed for easy installation of pre-configured models but it also allows you to install virtually any model from Hugging Face or even your own.

Jan goes a step further by integrating with other local engines like LM Studio and [ollama](https://ollama.com).

Jan’s interoperability even extends to exposing these LLMs through a [server API](https://jan.ai/api-reference). This API is compatible with the widely-recognized **[OpenAI API](https://platform.openai.com/docs/api-reference)**, simplifying its integration with many components of the GenAI ecosystem!

Importantly, Jan is free software (think 'freedom') licensed under [**Affero GPL**](https://github.com/janhq/jan/blob/dev/LICENSE). This grants anyone the liberty to run it offline, self-host, or modify it. Note that this license applies copyleft aspect also to network distribution, ensuring Cloud/SaaS providers contribute back to the core project and community.

Jan’s open source philosophy can also be seen in its design for [extensions](https://jan.ai/docs/extensions) and TypeScript API, making it user-friendly for [customization and extension](https://github.com/janhq/jan/tree/dev/extensions/model-extension) by anyone.

Now that we've introduced Jan, let's demonstrate how easy it is to use.

## Installation and Use of Local Models

You can easily browse and install pre-configured models from the *Model Hub*.

![Model Hub](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8dqaoy6z4uq4cm9keh2n.png)

I prefer importing and reusing the models I've previously downloaded for LM Studio to achieve storage optimization.

![Import Model](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/veakwcjoinuugow9cp4z.png)
![Reuse Model](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yvk7opqlari8gm8umkn8.png)

From the *Thread* panel, I can simply select the model (Llama 3, in this case), define the assistant and model parameters (I just tweaked the Max Token value to 200), and initiate a new Thread.

![Select Model](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lmke61uu7go583a4drj7.png)
![Thread](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zwrprgt3pidbxzu8qmqb.png)

## Using Jan as a Server

Configuring **Jan Desktop as a Local API Server** and monitoring its logs is straightforward.

![Local API Server](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/87t4ccmu35iidxvaibmj.png)

Next, you can make OpenAI-like API calls to the server, as shown here from a Jupyter JavaScript Notebook.

![API call from Jupyter](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tit8apq9lbrxgqr5ei06.png)

*(For more information on using a JavaScript Notebook with GenAI APIs, refer to this [previous article](https://dev.to/worldlinetech/in-bed-with-gpt-and-nodejs-4kh2) and my [GenAI's Lamp](https://www.youtube.com/playlist?list=PLpEPN3tmJMNaQg4spmz1y2kGcqFaE2_8m) YouTube Series)*.

**Jan Server** can also be independently deployed (no GUI) in [Docker or Kubernetes](https://jan.ai/docs/server-installation/onprem) for on-premise or Cloud running. Detailed instructions for AWS, GCP, and Azure are available in the project's [documentation](https://jan.ai/docs/)).

## Conclusion

The Jan project is ever-expanding, promising many more enhancements and features – all visible thanks to its  *build in the open philosophy* - on the [roadmap](https://github.com/orgs/janhq/projects/5/views/31). 

AFter activating the *Experimental Mode*, you can preview upcoming features, such as the promising [Retrieval](https://jan.ai/docs/tools/retrieval) tool, designed to implement the RAG pattern and chat with PDF documents. 

So definitely stay tuned for updates!

![Jan Mark](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b8wvz4cg6df396oumwbv.png)