21 Open Source LLM Projects to Become 10x AI Developer
Tags: programming, opensource, ai, webdev
Published: 2024-07-17 09:00:56

The time of AI is high especially because of powerful LLMs like GPT-4o and Claude.

Today, I'm covering 21 open source LLM projects that can help you to build something exciting and integrate AI into your project.

As a developer, I can confidently say that AI is not as scary as others make it sound and those who don't learn will get left behind.

Let's cover it all.

![gif](https://media4.giphy.com/media/v1.Y2lkPTc5MGI3NjExZ21oZnA1ZDlzaHVsMHdrOHFvYzJvdHlucnhiZjA2dmU0d21idmFrbiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/dJHaTbNQOjYMWdHSTG/giphy.webp)

---

By the way, I'm part of Latitude and we're building an open source LLM development platform. You can join the waitlist at [ai.latitude.so](https://ai.latitude.so/).

You would be able to do a lot of cool stuff like:

⚡ Deploy prompts as api endpoints.

⚡ Automated evaluations using LLMs.

⚡ Collaborate on prompt engineering.

I'm very confident that you will love it after its release!

{% cta https://ai.latitude.so/ %} Join Waitlist ⭐️ {% endcta %}

![llm platform by latitude](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d1xuu1yhksyd2yy4dnsx.png)

---

## 1. [Vanna](https://github.com/vanna-ai/vanna) - Chat with your SQL database.

![vanna](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uxryt1lsw41hwt31bqti.png)

&nbsp;

Vanna is an MIT-licensed open-source Python RAG (Retrieval-Augmented Generation) framework for SQL generation.

Basically, it's a Python package that uses retrieval augmentation to help you generate accurate SQL queries for your database using LLMs.

It's perfect for developers like me who are not very fond of SQL queries!

![low level diagram on how vanna works](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sinq74k898iyfvsnjimx.png)

Vanna works in two easy steps - train a `RAG model` on your data, and then ask questions that will return SQL queries that can be set up to run on your database automatically.

![how vanna works](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5fjc5966fphrge71s8sg.png)

You don't need to know how this whole stuff works to use it. 
You just have to `train` a model, which stores some metadata, and then use it to `ask` questions.

Get started with the following command.
```npm
pip install vanna
```

To make things a little easier, they have built user interfaces that you can use as a starting point for your own custom interface. Find all the [interfaces](https://github.com/vanna-ai/vanna?tab=readme-ov-file#user-interfaces) including Jupyter Notebook and Flask.

![variations](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/20nymucrkb9wlsvi4kpd.png)

You can read the [docs](https://vanna.ai/docs/) and you can try this [Colab notebook](https://vanna.ai/docs/app/) in case you want to see how it works after training.

![flask UI gif](https://vanna.ai/blog/img/vanna-flask.gif)

Watch this [demo](https://github.com/vanna-ai/vanna?tab=readme-ov-file#vanna) for a complete walkthrough!

![vanna demo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/40gg9oeb4piys15p2re2.gif)

They have 9.5k+ stars on GitHub and are built using Python.

{% cta https://github.com/vanna-ai/vanna %} Star Vanna ⭐️ {% endcta %}

---

## 2. [Khoj](https://github.com/khoj-ai/khoj) - Your AI second brain.

![khoj](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/31d9m3po4oppf4ignvbh.png)

&nbsp;

Khoj is the open source, AI copilot for search. Easily get answers without having to sift through online results or your own notes.

For me, the concept seems exciting and It can help me in researching a lot of projects.

Khoj can understand your Word, PDF, org-mode, markdown, plaintext files, GitHub projects, and even Notion pages.

![type of documents](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5ory8aio202w6ce5x6ul.png)

It's available as a Desktop app, Emacs package, Obsidian plugin, Web app, and Whatsapp AI. Obsidian with Khoj might be the most powerful combo!

You can get started with Khoj locally in a few minutes with the following commands.

```
$ pip install khoj-assistant
$ khoj
```

Watch it in action!

![khoj walkthrough gif](https://media0.giphy.com/media/v1.Y2lkPTc5MGI3NjExN3RzNHA0MWE2NnB5aTMxOXlnY3puMDhkaHpwa2ZrYWlzMWJ0ZXRnNyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/0VIZl3xIVyzzfS9hiJ/giphy.gif)

Some of the exciting features:

✅ You can share your notes and documents to extend your digital brain.

✅ Your AI agents have access to the internet, allowing you to incorporate real-time information.

✅ You'll get a fast, accurate semantic search on top of your docs.

✅ Your agents can create deeply personal images and understand your speech.

For instance, saying: "Create a picture of my dream house, based on my interests". It will draw this!

![image generation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s1f6gtwssn08mhk6s6ud.png)

Read all the [features](https://docs.khoj.dev/category/features) including shareable chat, online chat, file summarization, and complete details in various categories.

You can read the [docs](https://docs.khoj.dev/#/) and you can try [Khoj Cloud](https://app.khoj.dev/) to try it quickly.

Watch the complete walkthrough on YouTube!

{% embed https://www.youtube.com/watch?v=Lnx2K4TOnC4&pp=ygUYa2hvaiBvcGVuIHNvdXJjZSBwcm9qZWN0 %}

It has 12k stars on GitHub and is backed by YCombinator.

{% cta https://github.com/khoj-ai/khoj %} Star Khoj ⭐️ {% endcta %}

---

## 3.  [Flowise](https://github.com/FlowiseAI/Flowise) - Drag & drop UI to build your customized LLM flow.

![flowiseai](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r5bp43nil764fhe4a05z.png)

&nbsp;

Flowise is an open source UI visual tool to build your customized LLM orchestration flow & AI agents. 

We shouldn't compare any projects but I can confidently say this might be the most useful one among the projects listed here!

![flowise gif](https://github.com/FlowiseAI/Flowise/raw/main/images/flowise.gif)

Get started with the following npm command.
```npm
npm install -g flowise
npx flowise start
OR
npx flowise start --FLOWISE_USERNAME=user --FLOWISE_PASSWORD=1234
```

This is how you integrate the API.

```javascript
import requests

url = "/api/v1/prediction/:id"

def query(payload):
  response = requests.post(
    url,
    json = payload
  )
  return response.json()

output = query({
  question: "hello!"
)}
```

![integrations](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ahk2ovjrpq1qk3r5pfot.png)

You can read the [docs](https://docs.flowiseai.com/).

Cloud host is not available so you would have to self-host using these [instructions](https://github.com/FlowiseAI/Flowise?tab=readme-ov-file#-self-host).

Let's explore some of the use cases:

⚡ Let's say you have a website (could be a store, an e-commerce site, or a blog), and you want to scrap all the relative links of that website and have LLM answer any question on your website. You can follow this [step-by-step tutorial](https://docs.flowiseai.com/use-cases/web-scrape-qna) on how to achieve the same. 

![scraper](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e91sz2mga5wvc0x2hp2g.png)

⚡ You can also create a custom tool that will be able to call a webhook endpoint and pass in the necessary parameters into the webhook body. Follow this [guide](https://docs.flowiseai.com/use-cases/webhook-tool) which will be using Make.com to create the webhook workflow.

![webhook](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ckyivo9dvue461jc9pv4.png)

There are a lot of other use cases such as building a SQL QnA or interacting with API. Explore and build cool stuff!

FlowiseAI has 27.5k stars on GitHub and has more than 14k forks so it has a good overall ratio.

{% cta https://github.com/FlowiseAI/Flowise %} Star Flowise ⭐️ {% endcta %}

---

## 4. [LLAMA GPT](https://github.com/getumbrel/llama-gpt) - a self-hosted, offline, ChatGPT like chatbot (Powered by Llama 2).

![llama gpt](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m5qoizptgncd0u1wlvqc.png)

&nbsp;

LlamaGPT is a self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. It's 100% private and your data doesn't leave your device.

In the recent versions, they have also provided support for Code Llama models and Nvidia GPUs.

You can install it from the [Umbrel App Store](https://apps.umbrel.com/app/llama-gpt) or you can also [install it with Kubernetes](https://github.com/getumbrel/llama-gpt?tab=readme-ov-file#install-llamagpt-with-kubernetes).

You can read about the [supported models](https://github.com/getumbrel/llama-gpt?tab=readme-ov-file#supported-models) on the docs.

It's very simple to use!

![llama gpt video](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8e0x0vkbt3jifov3vb3j.gif)

I know, at this point it feels confusing that there are so many ways to run LLM locally. As a developer, I think it's important to evaluate which method works for our situation!

They have 10k+ stars on GitHub and offer 2 packages.

{% cta https://github.com/getumbrel/llama-gpt %} Star LLAMA GPT ⭐️ {% endcta %}

---

## 5. [LocalAI](https://github.com/mudler/LocalAI) - free OpenAI alternative.

![localAI](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ln76pv7ffcsq11c7wx45.png)

&nbsp;

LocalAI is free, open source, and considered as an alternative to OpenAI. 

LocalAI acts as a drop-in replacement REST API that’s compatible with OpenAI (Elevenlabs, Anthropic... ) API specifications for local AI inferencing. 

It allows you to run LLMs, generate images, and audio (and not only) locally or on-prem with consumer-grade hardware, supporting multiple model families. The best part is that it does not require GPU. 

I never thought that there would be such an option so it's a goldmine for devs who don't want to pay much. Plus, it allows to generate of Text, Audio, Video, and Images and also has voice cloning capabilities. What more do you need?

You can watch the complete walkthrough by Semaphore CI!

{% embed https://www.youtube.com/watch?v=Xh57mMlfuMk %}

There are a lot of [integration options](https://localai.io/docs/integrations/) and developers have built awesome stuff such as [Extension for attaching LocalAI instance to VSCode](https://github.com/badgooooor/localai-vscode-plugin).

You can read the [quickstart guide](https://localai.io/basics/getting_started/) and how to [run it with kubernetes](https://localai.io/basics/kubernetes/). Find all the [resources](https://github.com/mudler/LocalAI?tab=readme-ov-file#book--media-blogs-social) including how to run it on AWS, k8sgpt, and more.

They have 21k+ stars on GitHub and are on the `v2.18` release.

{% cta https://github.com/mudler/LocalAI %} Star LocalAI ⭐️ {% endcta %}

---

## 6.  [Continue](https://github.com/continuedev/continue) - enable you to create an AI software development system.

![continue](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7ro5ctus5tdfvqdnysby.png)

&nbsp;

Continue is one of the best AI code assistants I've seen in my developer journey.
 
You can connect any models and any context to build custom autocomplete and chat experiences inside [VS Code](https://marketplace.visualstudio.com/items?itemName=Continue.continue) and [JetBrains](https://plugins.jetbrains.com/plugin/22707-continue-extension).

You can easily set it up. Below are some of the snapshots while I was installing it.

![step 1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rq5ld2djixiv5pmz8zzx.png)
<figcaption>Step 1</figcaption>

&nbsp;

![step 2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j9u0lan7koa5sjtysykw.png)
<figcaption>run it through terminal</figcaption>

&nbsp;

![step2 complete](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/duhhgmg2eqr1tk1r2ecy.png)
<figcaption>step2 complete</figcaption>

&nbsp;

![step3](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wzufhvtre8o97wq3zqlr.png)
<figcaption>run it through terminal</figcaption>

&nbsp;

![step3 complete](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nd4lu8r0shiah1v7odq0.png)
<figcaption>step3 complete</figcaption>

&nbsp;

After you've configured it, you're all set to use all the amazing concepts it provides.

They have a lot of awesome features such as:

> Tab to autocomplete code suggestions.

![autocomplete](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/09xt6urla4jic5x3m5rr.gif)

> Ask questions about your codebase.

![questions](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qd95frn0j9cd417yighz.gif)

> Understand terminal errors immediately.

![errors](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kaaq6x5978tm1u61moxb.gif)

> Kick off actions with slash commands.

![commands](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j4vlzc2vuiuoivgqy5e7.png)

> Refactor functions where you are coding.

![refactor](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7wz1tzon8afivi79ulvn.png)

Read about all the [features](https://docs.continue.dev/how-to-use-continue).

You will have to install the [VSCode extension](https://marketplace.visualstudio.com/items?itemName=Continue.continue) from the marketplace and then read the [quickstart guide](https://docs.continue.dev/quickstart).

You can read the [docs](https://docs.continue.dev/intro).

You can also watch this basic demo on YouTube!

{% embed https://www.youtube.com/watch?v=V3Yq6w9QaxI&t=29s&pp=ygUaY29udGludWUgYWkgY29kZSBhc3Npc3RhbnQ%3D %}

They have 13k+ stars on GitHub and are built using TypeScript.

{% cta https://github.com/continuedev/continue %} Star Continue ⭐️ {% endcta %}

---

## 7. [Chat2DB](https://github.com/chat2db/Chat2DB) - AI-driven data management platform.

![chat2db](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0fu2hozmkm3ps8l99rj9.png)

&nbsp;

Chat2DB is an AI-first data management, development, and analysis tool.

Its core is AIGC (Artificial Intelligence Generation Code), which can convert natural language into SQL, SQL into natural language, and automatically generate reports, taking efficiency to another level. 

Even operations that do not understand SQL can use it to quickly query business data and generate reports.

When you do any operation, it will give you some suggestions. 
For instance, when you are doing database development, it will help you generate SQL directly in natural language, give you SQL optimization suggestions, help you analyze SQL performance, analyze SQL execution plan, and can also help you quickly generate SQL test data, system code, etc. It's actually very powerful :)

They have excellent support for multi-data sources and can easily integrate up to 17 different database types including PostgreSQL, MySQL, MongoDB, and Redis.

![databases](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c5k3ryojynrovdlx508i.png)

You can download or [try it in the browser](https://app.chat2db.ai/).

Let's see some of the exciting features:

✅ Intelligent reports.

![reports](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g4580tabu3t3vq48e0k5.png)

✅ Data Exploration.

![data exploration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mqq58sgw239ugep7ocez.png)

✅ SQL Development.

![SQL Development](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7fpvwv5lpiyqil2t6aig.png)

&nbsp;

![sql development](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j6zszr0onru157uuvhpw.png)

You can read the [quickstart guide](https://docs.chat2db.ai/docs/start-guide/getting-started) on the official docs.

They have 14k+ stars on GitHub and are on the release `v3.2`.

{% cta https://github.com/chat2db/Chat2DB %} Star Chat2DB ⭐️ {% endcta %}

---

## 8. [LibreChat](https://github.com/danny-avila/LibreChat?tab=readme-ov-file) - Enhanced ChatGPT Clone.

![libre chat](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/khdouc1ssvs8g3wwfets.png)

&nbsp;

LibreChat is a free, open source AI chat platform. This Web UI offers vast customization, supporting numerous AI providers, services, and integrations. 

It serves all AI conversations in one place with a familiar interface, and innovative additions, for as many users as you need.

Some of the features are:

✅ Upload and analyze images seamlessly with advanced models like Claude 3, GPT-4, Gemini Vision, Llava, and Assistants.

✅ Chat with files using various powerful endpoints using OpenAI, Azure, Anthropic, and Google.

✅ Multilingual UI with support for 20+ languages.

✅ [Diverse Model options](https://www.librechat.ai/docs/features#diverse-model-options) including OpenAI, BingAI, Anthropic (Claude), Azure OpenAI, and Google’s premier machine learning offerings.

![ai model image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3lwh3xae7ax2yejsqwxp.png)

You can read the [quickstart guide](https://www.librechat.ai/docs/quick_start) to get started.

Watch this video for the complete walkthrough!

{% embed https://www.youtube.com/watch?v=bSVHEbVPNl4 %}

They have 15k+ stars on GitHub and offer 4 packages.

{% cta https://github.com/danny-avila/LibreChat?tab=readme-ov-file %} Star LibreChat ⭐️ {% endcta %}

---

## 9. [Lobe Chat](https://github.com/lobehub/lobe-chat) - modern-design LLMs/AI chat framework.

![lobe chat](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ddxibf7xxx931tdoj1mn.png)

&nbsp;

An open-source, modern-design ChatGPT/LLMs UI/Framework.
Supports speech-synthesis, multi-modal, and extensible (function call) plugin systems. You can deploy your private OpenAI with one click. 

![journey](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/39se198xal53r854sdps.png)

Let's see some of the exciting features of LobeChat:

✅ Multi-Model Service Provider Support.

![multi service](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nodazgxel962wrp2hnvo.png)

They have expanded our support to multiple model service providers, rather than being limited to a single one.

Find the complete list of [10+ model service providers](https://lobehub.com/docs/usage/features/multi-ai-providers) that they support.

✅ Assistant Market.

![Assistant Market](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/35z3kz2jr4mnxid9dwsg.png)

In LobeChat's [Assistant Market](https://lobehub.com/assistants), creators can discover an innovative community that brings together numerous carefully designed assistants. 

![market](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ex23f2epblfp2cxtxbnl.png)

There are so many awesome applications there. WOW!

✅ Model Vision Recognition.

![Model Vision Recognition](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fuxz350091223cj36dq7.png)

LobeChat now supports large language models with visual recognition capabilities such as OpenAI's gpt-4-vision, Google Gemini Pro vision, and Zhipu GLM-4 Vision, enabling LobeChat to have multimodal interaction capabilities. 

Users can easily upload or drag and drop images into the chat box, and the assistant will be able to recognize the content of the images and engage in intelligent conversations.

✅ Text to Image Generation.

![Text to Image Generation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z2q6qzcy8anjgsg2381o.png)

You can directly utilize the Text-to-image tool during conversations with the assistant. 

By using the power of AI tools such as DALL-E 3, MidJourney, and Pollinations, assistants can now implement it properly.

✅ Local Large Language Model (LLM) Support.

![Local Large Language Model (LLM) Support.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ucn4rpa4p2vb11hhvkn1.png)

With the powerful infrastructure of Ollama AI and the community's collaborative efforts, you can now engage in conversations with a local LLM (Large Language Model) in LobeChat! 

By running the following Docker command, you can experience conversations with a local LLM in LobeChat.

```bash
docker run -d -p 3210:3210 -e OLLAMA_PROXY_URL=http://host.docker.internal:11434/v1 lobehub/lobe-chat
```

✅ Progressive Web App (PWA).

![Progressive Web App (PWA)](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sccmha74iz01rr12gphr.png)

They have adopted Progressive Web App PWA technology, which is a modern web technology that elevates web applications to a near-native app experience. 

✅ Custom Themes.

![custom themes](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7cl73pplbor4z1381kdm.png)

&nbsp;

Read about all of the [features and use cases](https://lobehub.com/docs/usage/start).

You can self-host or deploy it using docker. The [ecosystem](https://github.com/lobehub/lobe-chat/tree/main?tab=readme-ov-file#-ecosystem) of lobe chat provides 4 packages: `lobehub/ui`, `lobehub/icons`, `lobehub/tts`, and `lobehub/lint`.

They also provide [plugins market](https://lobehub.com/plugins) where you can find lots of useful plugins that can be used to introduce new function calls and even new ways to render message results. If you want to develop your own plugin, refer to [📘 Plugin Development Guide](https://lobehub.com/docs/usage/plugins/development) in the wiki.

![plugins market](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uqtxt31vc42uwnw2ukgr.png)

You can read the [docs](https://lobehub.com/docs/usage/start).

You can check the [live demo](https://chat-preview.lobehub.com/chat). It's pretty cool!

![demo snapshot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xe3ngshtwpps2kmpu98f.png)

They have 35k+ stars on GitHub with more than 500 releases.

{% cta https://github.com/lobehub/lobe-chat %} Star Lobe Chat ⭐️ {% endcta %}

---

## 10. [MindsDB](https://github.com/mindsdb/mindsdb) - The platform for customizing AI from enterprise data.

![MindsDB](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i9q3jdswxdx6wqfk0vqw.png)

&nbsp;

MindsDB is the platform for customizing AI from enterprise data.

With MindsDB, you can deploy, serve, and fine-tune models in real-time, utilizing data from databases, vector stores, or applications, to build AI-powered apps - using universal tools developers already know.

![mindsdb](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d7ggww5ner38r4r9qc06.png)

With MindsDB and its nearly [200 integrations](https://docs.mindsdb.com/integrations/data-overview) to data sources and AI/ML frameworks, any developer can use their enterprise data to customize AI for their purpose, faster and more securely.

![how MindsDB works](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4q1gfmhq43gopdix03gr.png)

You can read the [docs](https://docs.mindsdb.com/) and [quickstart guide](https://docs.mindsdb.com/quickstart-tutorial) to get started.

They currently support a total of [3 SDKs](https://docs.mindsdb.com/sdks/overview) that is using using Mongo-QL, Python, and JavaScript.

There are several applications of MindsDB such as integrating with numerous data sources and AI frameworks so you can easily bring data and AI together to create & automate custom workflows.

The other common use cases include fine-tuning models, chatbots, alert systems, content generation, natural language processing, classification, regressions, and forecasting. Read more about the [use cases](https://docs.mindsdb.com/use-cases/) and each of them has an architecture diagram with a little info.

![use cases](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wuhxzbioqh9a5s9f0w7s.png)

For instance, the chatbot architecture diagram with MindsDB. You can read about all the [solutions](https://github.com/mindsdb/mindsdb?tab=readme-ov-file#-get-started) provided along with their SQL Query examples.

```
// SQL Query Example for Chatbot
CREATE CHATBOT slack_bot USING database='slack',agent='customer_support'; 
```

![chatbot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/otoqsro02ghqb709yglk.png)

Just to tell you about the overall possibilities, you can check out [How to Forecast Air Temperatures with AI + IoT Sensor Data](https://mindsdb.com/blog/how-to-forecast-air-temperatures-with-ai-iot-sensor-data). Exciting right :)

![mindsdb](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/82wrjyrkch44taeurv1r.png)

They have 25.4k+ stars on GitHub and are on the `v24.7.2.0` with more than 200 releases. By the way, this is the first time I've seen 4 parts in any release as I always followed the semantic release.

{% cta https://github.com/mindsdb/mindsdb %} Star MindsDB ⭐️ {% endcta %}

---

## 11. [AutoGPT](https://github.com/Significant-Gravitas/AutoGPT) - more exciting than ChatGPT.

![auto gpt](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3hjamyxzkhy7luwsi9vp.png)

&nbsp;

At the core of AutoGPT lies its primary project, a semi-autonomous agent driven by large language models (LLMs), designed to perform any tasks for you.

The AutoGPT project consists of [four main components](https://docs.agpt.co/#agent):

- The Agent – also known as just "AutoGPT"
- The Benchmark – AKA agbenchmark
- The Forge
- The Frontend

Read on how you can [set up AutoGPT](https://docs.agpt.co/autogpt/setup/) using the OpenAI key.

You can see this [YouTube video by Fireship](https://www.youtube.com/watch?v=_rGXIXyNqpk) on what is AutoGPT.

{% embed https://www.youtube.com/watch?v=_rGXIXyNqpk %}

You can also watch this [AutoGPT tutorial](https://www.youtube.com/watch?v=FeIIaJUN-4A) by Sentral Media.

You can read the [docs](https://docs.agpt.co/) and check out the [project board](https://github.com/orgs/Significant-Gravitas/projects/1) on what things are under development right now.

Even if you don't know much about AI, you can try AutoGPT to understand how you can save time and build cool stuff.

They have 164k+ stars on GitHub due to such an excellent use case and automation capabilities. 

{% cta https://github.com/Significant-Gravitas/AutoGPT %} Star AutoGPT ⭐️ {% endcta %}

---

## 12. [reor](https://github.com/reorproject/reor) - self organizing AI note-taking app.

![reor](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c0x2q2a67bg7gzdekizw.png)

&nbsp;

One of the most exciting projects that I've seen so far, especially because it runs models locally.

Reor is an AI-powered desktop note-taking app. It automatically links related notes, answers questions on your notes, and provides semantic search. 

Everything is stored locally and you can edit your notes with an Obsidian-like markdown editor. The project hypothesizes that AI tools for thought should run models locally by default. 

Reor stands on the shoulders of the giants `Ollama`, `Transformers.js` & `LanceDB` to enable both LLMs and embedding models to run locally. Connecting to OpenAI or OpenAI-compatible APIs like Oobabooga is also supported.

> I know you're wondering How can it possibly be `self-organizing`?

a. Every note you write is chunked and embedded into an internal vector database.
b. Related notes are connected automatically via vector similarity.
c. LLM-powered Q&A does RAG on the corpus of notes.
d. Everything can be searched semantically.

You can watch the demo here!

![demo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k1whpg9m7ubt5xluyf7f.gif)

One way to think about Reor is as a RAG app with two generators: the LLM and the human. In Q&A mode, the LLM is fed retrieved context from the corpus to help answer a query. 

Similarly, in editor mode, we can toggle the sidebar to reveal related notes `retrieved` from the corpus. This is quite a powerful way of `augmenting` your thoughts by cross-referencing ideas in a current note against related ideas from your digital collection.

You can read the [docs](https://www.reorproject.org/docs) and [download](https://www.reorproject.org/) from the website. Mac, Linux & Windows are all supported.

They have also provided starter guides to help you get started.

![get started guides](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bx3w7nalcwc9egumu0hm.png)

You can also watch this walkthrough!

{% embed https://www.youtube.com/watch?v=2XFYyrQz9xQ&pp=ygUYcmVvciBvcGVuIHNvdXJjZSBwcm9qZWN0 %}

They have 6.5k stars on GitHub and are built using TypeScript.

{% cta https://github.com/reorproject/reor %} Star reor ⭐️ {% endcta %}

---

## 13. [Leon](https://github.com/leon-ai/leon) - your open source personal assistant.

![leon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mnv85osce6ps9xodf07t.png)

&nbsp;

Leon is an open source personal assistant who can live on your server. You're definitely curious, right? 😅

He does stuff when you ask him to. You can even talk to him and he will revert by talking to you. Similarly, you can also text him!

If you are a developer (or not), you may want to build many things that could help in your daily life. Instead of building a dedicated project for each of those ideas, Leon can help you with his `Skills` structure.

![leon gif](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ygd52jmxnyz5ctjfoyto.gif)

If you want to, Leon can communicate with you by being offline to protect your privacy. This is a list of [skills](https://github.com/leon-ai/leon/tree/develop/skills) that Leon can do for now. 

You should read the [story behind Leon](https://blog.getleon.ai/the-story-behind-leon/). You can also watch this demo to learn more about Leon.

{% embed https://www.youtube.com/watch?v=p7GRGiicO1c %}

![features](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/70mddmgadcbfwzugd1bl.png)

This is the High-level architecture schema of Leon.

![architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a6b9vgj3fagera0bsyur.png)

You can install Leon by using this command.

```
# install leon global cli
npm install --global @leon-ai/cli

# install leon
leon create birth
```

You can read the [docs](https://docs.getleon.ai/).

Appwrite is one of the sponsors which says a lot about its overall impact.

It has 15k stars on GitHub and has released some drastic changes recently so make sure to read the docs with extra caution.

{% cta https://github.com/leon-ai/leon %} Star Leon ⭐️ {% endcta %}

---

## 14.  [Instrukt](https://github.com/blob42/Instrukt) - Integrated AI in the terminal.

![instrukt](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wsk64pf5yuosui91tmz9.png)

&nbsp;

Instrukt is a terminal-based AI-integrated environment. It offers a platform where users can:

- Create and instruct modular AI agents.
- Generate document indexes for question-answering.
- Create and attach tools to any agent.

Instruct them in natural language and, for safety, run them inside secure containers (currently implemented with Docker) to perform tasks in their dedicated, sandboxed space.

It's built using `Langchain`, `Textual`, and `Chroma`.

Get started with the following command.
```
pip install instrukt[all]
```

![instrukt](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r3aza7hnlji7hbi2o0js.gif)

There are a lot of exciting features such as:

✅ A terminal-based interface for power keyboard users to instruct AI agents without ever leaving the keyboard.

✅ Index your data and let agents retrieve it for question-answering. You can create and organize your indexes with an easy UI.

✅ Index creation will auto-detect programming languages and optimize the splitting/chunking strategy accordingly.

✅ Run agents inside secure docker containers for safety and privacy.

✅ Integrated REPL-Prompt for quick interaction with agents, and a fast feedback loop for development and testing.

✅ You can automate repetitive tasks with custom commands. It also has a built-in prompt/chat history.

You can read about all the [features](https://github.com/blob42/Instrukt?tab=readme-ov-file#features).

You can read the [installation guide](https://blob42.github.io/Instrukt/install.html).

You can also debug and introspect agents using an in-built IPython console which is a neat little feature.

![console debugging](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qaan8np68e3fk1yueexm.png)

Instrukt is licensed with an AGPL license meaning that it can be used by anyone for whatever purpose.

It is safe to say that Instrukt is an AI commander for your terminal.

It is a new project so they have around 240 stars on GitHub but the use case is damn good.

{% cta https://github.com/blob42/Instrukt %} Star Instrukt ⭐️ {% endcta %}

---

## 15. [Quivr](https://github.com/QuivrHQ/quivr) - RAG Framework for building GenAI Second Brains.

![quivr](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oiowtt3ys9shf4iivglq.png)

&nbsp;

Quivr, your second brain, utilizes the power of GenerativeAI to be your personal assistant! You can think of it as Obsidian but turbocharged with AI powers.

Quivr is a platform that enables the creation of AI assistants, referred to as `Brain`. These assistants are designed with specialized capabilities like some can connect to specific data sources, allowing users to interact directly with the data. 

While others serve as specialized tools for particular use cases, powered by Rag technology. These tools process specific inputs to generate practical outputs, such as summaries, translations, and more.

Watch a quick demo of Quivr!

![quivr gif](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k7xx3290daebg2pw11i2.gif)

Some of the amazing features are:

✅ You can choose the type of Brain you want to use, based on the data source you wish to interact with.

✅ They also provide a powerful feature to share your brain with others. This can be done by sharing with individuals via their emails and assigning them specific rights. 

![sharing brain](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e3mn636j0sl6si5lzpzf.png)

✅ Quivr works offline, so you can access your data anytime, anywhere.

✅ You can access and continue your past conversations with your brains.

✅ But the best one that I loved is that you can literally install a Slack bot. Refer to this demo to see what you can do. Very cool!

{% embed https://youtu.be/1yMe21vIl9E %}

Anyway, read about all the [awesome stuff](https://docs.quivr.app/features/brain-creation) that you can do with Quivr.

You can read the [installation guide](https://github.com/QuivrHQ/quivr?tab=readme-ov-file#getting-started-) and [60 seconds installation video](https://www.youtube.com/watch?v=cXBa6dZJN48). I really loved this idea!

You can read the [docs](https://docs.quivr.app/home/intro).

![stats](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5a27c2ubbmri0b2xlh1l.png)

They have also provided [guides](https://docs.quivr.app/deployment/porter) on how to deploy Quivr with Vercel, Porter, AWS, and Digital Ocean.

They could provide a better free tier plan but it's more than enough to test things on your end.

It has 30k+ Stars on GitHub with 220+ releases which means they're constantly improving.

{% cta https://github.com/QuivrHQ/quivr %} Star Quivr ⭐️ {% endcta %}

---

## 16. [Open Interpreter](https://github.com/OpenInterpreter/open-interpreter) - natural language interface for terminal.

![Open Interpreter](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/av7udc5fibj1wz88w0u8.png)

&nbsp;

Open Interpreter lets LLMs run code (Python, Javascript, Shell, and more) locally. You can chat with Open Interpreter through a ChatGPT-like interface in your terminal by running `$ interpreter` after installing.

This provides a natural-language interface to your computer's general-purpose capabilities:

✅ Create and edit photos, videos, PDFs, etc.
✅ Control a Chrome browser to perform research Plot, clean, and analyze large datasets.

I don't know about you, but their [website](https://www.openinterpreter.com/) made me say WOW!

Quickstart using this command.
```python
pip install open-interpreter

// After installation, simply run:
interpreter
```

You can read the [quickstart guide](https://docs.openinterpreter.com/getting-started/introduction).

You should read about the [comparison to ChatGPT's Code Interpreter](https://github.com/OpenInterpreter/open-interpreter?tab=readme-ov-file#comparison-to-chatgpts-code-interpreter) and the [commands](https://github.com/OpenInterpreter/open-interpreter?tab=readme-ov-file#commands) that you can use.

You can read the [docs](https://docs.openinterpreter.com/getting-started/introduction).

Open Interpreter works with both hosted and local language models. Hosted models are faster and more capable, but require payment while local models are private and free but are often less capable. Choose based on your use case!

They have 48k+ stars on GitHub and are used by 300+ developers.

{% cta https://github.com/OpenInterpreter/open-interpreter %} Star Open Interpreter ⭐️ {% endcta %}

---

## 17. [CopilotKit](https://github.com/CopilotKit/CopilotKit) - 10x easier to build AI Copilots.

![copilotKit](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iq3tyn2vi21qmnso1lg7.png)

&nbsp;

You will agree that it's tough to add AI features in React, that's where Copilot helps you as a framework for building custom AI Copilots.

You can build in-app AI chatbots, and in-app AI Agents with simple components provided by Copilotkit which is at least 10x easier compared to building it from scratch. 

You shouldn't reinvent the wheel if there is already a very simple and fast solution!

They also provide built-in (fully-customizable) Copilot-native UX components like `<CopilotKit />`, `<CopilotPopup />`, `<CopilotSidebar />`, `<CopilotTextarea />`. 

![components](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/abbaw6u488uu6xw8y88x.png)

Get started with the following npm command.
```npm
npm i @copilotkit/react-core @copilotkit/react-ui
```

This is how you can integrate a Chatbot.

A `CopilotKit` must wrap all components which interact with CopilotKit. It’s recommended you also get started with `CopilotSidebar` (you can swap to a different UI provider later).

```javascript
"use client";
import { CopilotKit } from "@copilotkit/react-core";
import { CopilotSidebar } from "@copilotkit/react-ui";
import "@copilotkit/react-ui/styles.css"; 

export default function RootLayout({children}) {
  return (
    <CopilotKit url="/path_to_copilotkit_endpoint/see_below">
      <CopilotSidebar>
        {children}
      </CopilotSidebar>
    </CopilotKit>
  );
}
```

You can read the [docs](https://docs.copilotkit.ai/getting-started/quickstart-textarea) and check the [demo video](https://github.com/CopilotKit/CopilotKit?tab=readme-ov-file#demo).

You can integrate Vercel AI SDK, OpenAI APIs, Langchain, and other LLM providers with ease. You can follow this [guide](https://docs.copilotkit.ai/getting-started/quickstart-chatbot) to integrate a chatbot into your application.

The basic idea is to build AI Chatbots very fast without a lot of struggle, especially with LLM-based apps. 

You can watch the complete walkthrough!

{% embed https://youtu.be/VFXdSQxTTww %}

CopilotKit has recently crossed 7k+ stars on GitHub with 300+ releases.

{% cta https://github.com/CopilotKit/CopilotKit %} Star CopilotKit ⭐️ {% endcta %}

---

## 18. [GPT Engineer](https://github.com/gpt-engineer-org/gpt-engineer) - AI builds what you ask.

![gpt engineer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8zdpd1ndd7afnt8m4cvz.png)

&nbsp;

GPT-engineer lets you specify software in natural language, sit back, and watch as an AI writes and executes the code, and you can ask the AI to implement improvements.

It's safe to say that it's an engineer who doesn't need a degree 😅

It's a commercial project for the automatic generation of web apps. It features a UI for non-technical users connected to a git-controlled codebase.

I know this feels confusing, so watch the below demo to understand how you can use GPT Engineer.

![demo gif](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4lpk4ctcuypzr7ms2pii.gif)

You can get started by installing the stable release using this command.

```
python -m pip install gpt-engineer
```

By default, gpt-engineer expects text input via a prompt file. It can also accept image inputs for vision-capable models. This can be useful for adding UX or architecture diagrams as additional context for GPT engineer. Read about all the [awesome features](https://github.com/gpt-engineer-org/gpt-engineer?tab=readme-ov-file#features).

If you want a complete walkthrough, watch this awesome demo by David!

{% embed https://www.youtube.com/watch?v=gWy-pJ2ofEM %}

I recommend checking out the [roadmap](https://github.com/gpt-engineer-org/gpt-engineer/blob/main/ROADMAP.md) to understand the overall vision.

![roadmap](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v6vhrqgn8brw8woklvm0.png)

They have 51k+ stars on GitHub and are on the `v0.3` release.

{% cta https://github.com/gpt-engineer-org/gpt-engineer %} Star GPT Engineer ⭐️ {% endcta %}

---

## 19. [Dalai](https://github.com/cocktailpeanut/dalai) - the simplest way to run LLaMA and Alpaca locally.

![dalai](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hthbzdoar9s2li6kepgp.png)

&nbsp;

Dalai lets you run LLaMA and Alpaca on your computerP and is powered by `llama.cpp`, `llama-dl CDN`, and `alpaca.cpp`.

Dalai runs on operating systems such as Linux, Mac, and Windows so that's a plus point!

![gif demo](https://github.com/cocktailpeanut/dalai/raw/main/docs/alpaca.gif)

Dalai is also an NPM package:

- programmatically install
- locally make requests to the model
- run a dalai server (powered by socket.io)
- programmatically make requests to a remote dalai server (via socket.io)

You can install the package using the below npm command.

```
npm install dalai
```

![latest package](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/27mx3wt2l025oxtzzz6p.png)

You can read the [memory requirements](https://github.com/cocktailpeanut/dalai?tab=readme-ov-file#2-memory-requirements) and [how to archive elsewhere](https://github.com/cocktailpeanut/dalai?tab=readme-ov-file#using-a-different-home-folder) rather than home directory.

They have 13k stars on GitHub and are still in the very early stages.

{% cta https://github.com/cocktailpeanut/dalai %} Star Dalai ⭐️ {% endcta %}

---

## 20. [OpenLLM](https://github.com/bentoml/OpenLLM) - run LLMs as OpenAI compatible API endpoint in the cloud

![open llm](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jf3fjhw0bqxyletw847q.png)

&nbsp;

OpenLLM lets developers run any open-source LLMs as OpenAI-compatible API endpoints with a single command.

⚡ Build for fast and production usage.
⚡ Support llama3, qwen2, gemma, etc, and many quantized versions full list.
⚡ OpenAI-compatible API & includes ChatGPT like UI.
⚡ Accelerated LLM decoding with state-of-the-art inference backends.
⚡ Ready for enterprise-grade cloud deployment (Kubernetes, Docker, and BentoCloud).

Get started with the following command.
```npm
pip install openllm  # or pip3 install openllm
openllm hello
```

OpenLLM provides a chat user interface (UI) at the /chat endpoint for an LLM server. You can visit the chat UI at `http://localhost:3000/chat` and start different conversations with the model.

![open llm](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wgbyum3mgy51q46njf0q.png)

OpenLLM supports LLM cloud deployment via BentoML, the unified model serving framework, and BentoCloud, an AI inference platform for enterprise AI teams.  

![models](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mdlikmcik2dq746qha3b.png)

If you don't know, BentoCloud provides a fully managed infrastructure optimized for LLM inference with autoscaling, model orchestration, observability, and many more, allowing you to run any AI model in the cloud.

![bento cloud console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rmhtm95h90h6z2ei9e8b.png)
<figcaption>Once the deployment is complete, you can run model inference on the BentoCloud console:</figcaption>

You can read about the [supported models](https://github.com/bentoml/OpenLLM?tab=readme-ov-file#supported-models) and how to [start the LLM server](https://github.com/bentoml/OpenLLM?tab=readme-ov-file#start-an-llm-server).

Explore docs as you can also chat with a model in the CLI using `openllm run` and specifying model version - `openllm run llama3:8b`.

For people like me who love exploring walkthroughs, watch this demo by Matthew!

{% embed https://www.youtube.com/watch?v=8nZZ2oQhx4E&pp=ygUIb3BlbiBsbG0%3D %}

They have 9k+ stars on GitHub and have 100+ releases so it's growing at a rapid pace.

{% cta https://github.com/bentoml/OpenLLM %} Star OpenLLM ⭐️ {% endcta %}

---

## 21. [Unsloth](https://github.com/unslothai/unsloth) - Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory.

![unsloth](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3eoe4v0miesybe1p9n0e.png)

&nbsp;

Unsloth makes finetuning large language models like Llama-3, Mistral, Phi-3, and Gemma 2x faster, use 70% less memory, and with no degradation in accuracy!

> ✅ What is finetuning?

If we want a language model to learn a new skill, a new language, some new programming language, or simply want the language model to learn how to follow and answer instructions like how ChatGPT functions, we do finetuning!

Finetuning is the process of updating the actual `brains` of the language model through some process called back-propagation. But, finetuning can get very slow and very resource intensive.

&nbsp;

Unsloth can be installed locally or through another GPU service like Google Colab. Most use Unsloth through the interface Google Colab which provides a [free GPU](https://github.com/unslothai/unsloth?tab=readme-ov-file#-finetune-for-free) to train with.

![Finetune for Free](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6mnntbvbcho8hmihd59j.png)

Some of the things that stand out:

✅ Open source trains 5x faster, and the pro version claims to be 30x faster.

✅ No approximation methods are used resulting in a 0% loss in accuracy.

✅ No change of hardware, Works on Linux and Windows via WSL.

![unsloth model stats](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0147naadascce5oztcwf.png)

You can read the [installation instructions](https://github.com/unslothai/unsloth?tab=readme-ov-file#-installation-instructions) and [performance benchmarking tables](https://unsloth.ai/blog/mistral-benchmark#Benchmark%20tables) on the website.

![unsloth](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qm9hb9k6rqt8i1xr3re1.png)

You can read the [docs](https://docs.unsloth.ai/) and all the [uploaded models on Hugging Face](https://huggingface.co/unsloth) directly.

They have also provided a detailed guide on [How to Finetune Llama-3 and Export to Ollama](https://docs.unsloth.ai/tutorials/how-to-finetune-llama-3-and-export-to-ollama).

They have 12.5k+ stars on GitHub and it's an efficient solution.

{% cta https://github.com/unslothai/unsloth %} Star Unsloth ⭐️ {% endcta %}

---

I hope you learned something new!

I believe that learning to use these powerful LLM is a choice and it's on you (as a developer) to find better productivity solutions for your use case.

Have a great day! Till next time.

|  If you like this kind of stuff, <br /> please follow me for more :)  |  <a href="https://twitter.com/Anmol_Codes"><img src="https://img.shields.io/badge/Twitter-d5d5d5?style=for-the-badge&logo=x&logoColor=0A0209" alt="profile of Twitter with username Anmol_Codes" ></a> <a href="https://github.com/Anmol-Baranwal"><img src="https://img.shields.io/badge/github-181717?style=for-the-badge&logo=github&logoColor=white" alt="profile of GitHub with username Anmol-Baranwal" ></a> <a href="https://www.linkedin.com/in/Anmol-Baranwal/"><img src="https://img.shields.io/badge/LinkedIn-0A66C2?style=for-the-badge&logo=linkedin&logoColor=white" alt="profile of LinkedIn with username Anmol-Baranwal" /></a> |
|------------|----------|

Follow Latitude for more content like this.

{% embed https://dev.to/latitude %}