All the tools I need to build a perfect AI app.
Tags: ai, programming, opensource, python
Published: 2024-03-14 12:09:13

The world of AI has grown much in the past decade.

AI is everywhere, from voice assistants to software development, and it can be really helpful if we use it right.

Making AI apps is profitable in a world like this so here I am covering 25 open source projects that you can use to make your AI app and take it to the next level. 

There are exciting concepts like interactive communication with 3D characters using voice synthesis. Stick to the end. 

There will be plenty of resources, articles, project ideas, guides, and more to follow along.

Let's cover it all!

---

## 1. [Taipy](https://github.com/Avaiga/taipy) - Data and AI algorithms into production-ready web applications.

![taipy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/deak7rre409rzv5j5viv.png)

&nbsp;

Taipy is an open source Python library for easy, end-to-end application development, featuring what-if analyses, smart pipeline execution, built-in scheduling, and deployment tools.

I'm sure most of you don't understand so Taipy is used for creating a GUI interface for Python-based applications and improving data flow management.

So, you can plot a graph of the data set and use a GUI-like slider to give the option to play with the data with other useful features.

While Streamlit is a popular tool, its performance can decline significantly when handling large datasets, making it impractical for production-level use. 
Taipy, on the other hand, offers simplicity and ease of use without sacrificing performance. By trying Taipy, you'll experience firsthand its user-friendly interface and efficient handling of data. 

Under the hood, Taipy utilizes various libraries to streamline development and enhance functionality.

![libraries](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n9xts3nof4uapr7dakrl.png)

Get started with the following command.
```pip
pip install taipy
```

Let's talk about the latest [Taipy v3.1 release](https://docs.taipy.io/en/latest/relnotes/).

The latest release makes it possible to visualize any HTML or Python objects within Taipy's versatile part object.
This means libraries like [Folium](https://python-visualization.github.io/folium/latest/), [Bokeh](https://bokeh.org/), [Vega-Altair](https://altair-viz.github.io/), and [Matplotlib](https://matplotlib.org/) are now available for visualizations. 

This also brings native support for [Plotly python](https://plotly.com/python/), making it easier to plot charts.

![plotly python](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xdewvex88md09hvu3s80.png)

They have also improved performance using distributed computing but the best part is Taipy and all its dependencies are now fully compatible with Python 3.12 so you can work with the most up-to-date tools and libraries while using Taipy for your projects.

You can read the [docs](https://docs.taipy.io/en/latest/).

For instance, you can see the [chat demo](https://docs.taipy.io/en/release-3.1/gallery/llm/5_chatbot/) which uses OpenAI's GPT-4 API to generate responses to your messages. You can easily change the code to use any other API or model.

![chat demo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kug1mclhmzyad0hjchif.png)

Another useful thing is that the Taipy team has provided a VSCode extension called [Taipy Studio](https://docs.taipy.io/en/latest/manuals/studio/) to accelerate the building of Taipy applications.

![taipy studio](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kc1umm5hcxes0ydbuspb.png)

You can also deploy your applications with the Taipy cloud.

If you want to read a blog to see codebase structure, you can read [Create a Web Interface for your LLM in Python using Taipy](https://huggingface.co/blog/Alex1337/create-a-web-interface-for-your-llm-in-python) by HuggingFace.

It is generally tough to try out new technologies, but Taipy has provided [10+ demo tutorials](https://docs.taipy.io/en/release-3.1/gallery/) with code & proper docs for you to follow along.

For instance, some live demo examples:

- [Covid Dashboard](https://covid-dashboard.taipy.cloud/Country)
- [Tweet Generation](https://tweet-generation.taipy.cloud/)
- [Data Visualization](https://production-planning.taipy.cloud/Data-Visualization)
- [Real time Face Recognition](https://face-recognition.taipy.cloud/)

Taipy has 7k+ Stars on GitHub and is on the `v3` release so they are constantly improving.


![Lisan Al Gaib](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m8etards1b7qfpbk2scr.png)

{% cta https://github.com/Avaiga/taipy %} Star Taipy ⭐️ {% endcta %}

---

## 2. [Supabase](https://github.com/supabase/supabase) - open source Firebase alternative.

![supabase](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/an2b9aqiij0j2tml1c6b.png)

&nbsp;

To build an AI app, you need a backend, and Supabase serves as an excellent backend service provider to meet this need.

Get started with the following npm command (Next.js).
```npm
npx create-next-app -e with-supabase
```

This is how you can use CRUD operations.
```javascript
import { createClient } from '@supabase/supabase-js'
    
// Initialize 
const supabaseUrl = 'https://chat-room.supabase.co'
const supabaseKey = 'public-anon-key'
const supabase = createClient(supabaseUrl, supabaseKey)
  
// Create a new chat room
const newRoom = await supabase
  .from('rooms')
  .insert({ name: 'Supabase Fan Club', public: true })
    
// Get public rooms and their messages
const publicRooms = await supabase
  .from('rooms')
  .select(`
    name,
    messages ( text )
  `)
  .eq('public', true)
  
// Update multiple users
const updatedUsers = await supabase
  .from('users')
  .eq('account_type', 'paid')
  .update({ highlight_color: 'gold' })
```

You can read the [docs](https://supabase.com/docs).

You can build a crazy fast application with Auth, realtime, Edge functions, storage, and many more. Supabase covers it all!

Supabase also provides several starting kits like [Nextjs with LangChain](https://github.com/langchain-ai/langchain-nextjs-template), [Stripe with Nextjs](https://github.com/vercel/nextjs-subscription-payments) or [AI Chatbot](https://github.com/supabase-community/vercel-ai-chatbot).

Supabase has 63k+ Stars on GitHub and plenty of contributors with 27k+ commits.

{% cta https://github.com/supabase/supabase %} Star Supabase ⭐️ {% endcta %}

---

## 3. [Chatwoot](https://github.com/chatwoot/chatwoot) - live chat, email support, omni-channel desk & own your data.

![chatwoot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bpgjh0hdr5u5cpf2kdn7.png)

&nbsp;

Chatwoot connects with popular customer communication channels like Email, Website live-chat, Facebook, Twitter, WhatsApp, Instagram, Line, etc. This helps you deliver a consistent customer experience across channels – from a single dashboard.

This could be important in various cases, such as when you're building a community around your AI app.

![chatwoot features](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l0u3z2cdqvzhqb94h5zm.png)

You can read the [docs](https://www.chatwoot.com/docs/product) to discover a wide range of integration options so it's easier to manage the whole ecosystem.

They have very detailed documentation along with snapshot examples in each integration such as [WhatsApp channel with WhatsApp Cloud API](https://www.chatwoot.com/docs/product/channels/whatsapp/whatsapp-cloud). You can deploy to Heroku with 1-click or self-host as per needs.

They have 18k+ Stars on GitHub and are on the `v3.6` release.

{% cta https://github.com/chatwoot/chatwoot %} Star Chatwoot ⭐️ {% endcta %}

---

## 4. [CopilotKit](https://github.com/CopilotKit/CopilotKit) - AI Copilots for your product in hours.

![copilotKit](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nzuxjfog2ldam3csrl62.png)

&nbsp;

You can integrate key AI features into React apps using two React components. They also provide built-in (fully-customizable) Copilot-native UX components like `<CopilotKit />`, `<CopilotPopup />`, `<CopilotSidebar />`, `<CopilotTextarea />`. 

Get started with the following npm command.
```npm
npm i @copilotkit/react-core @copilotkit/react-ui @copilotkit/react-textarea
```

This is how you can integrate a CopilotTextArea.

```javascript
import { CopilotTextarea } from "@copilotkit/react-textarea";
import { useState } from "react";

export function SomeReactComponent() {
  const [text, setText] = useState("");

  return (
    <>
      <CopilotTextarea
        className="px-4 py-4"
        value={text}
        onValueChange={(value: string) => setText(value)}
        placeholder="What are your plans for your vacation?"
        autosuggestionsConfig={{
          textareaPurpose: "Travel notes from the user's previous vacations. Likely written in a colloquial style, but adjust as needed.",
          chatApiConfigs: {
            suggestionsApiConfig: {
              forwardedParams: {
                max_tokens: 20,
                stop: [".", "?", "!"],
              },
            },
          },
        }}
      />
    </>
  );
}
```

You can read the [docs](https://docs.copilotkit.ai/getting-started/quickstart-textarea).

The basic idea is to build AI Chatbots in minutes that can be useful for LLM-based full-stack applications.

{% cta https://github.com/CopilotKit/CopilotKit %} Star CopilotKit ⭐️ {% endcta %}

---

## 5. [DALL·E Mini](https://github.com/borisdayma/dalle-mini) - Generate images from a text prompt.

![generate images from text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mco3wf4nzc5j245aizpu.png)

&nbsp;

OpenAI had the first impressive model for generating images with DALL·E.

Craiyon/DALL·E mini is an attempt to reproduce those outcomes using an open source model.

If you're wondering about the name, DALL-E mini was renamed Craiyon at the request of the parent company and utilizes similar techniques in a more easily accessible web app format.

You can use the model on [Craiyon](https://www.craiyon.com/).

![craiyon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ntjhsr9f7t1y0idlysjw.png)

Get started with the following command (for development).
```
pip install dalle-mini
```

You can read the [docs](https://github.com/borisdayma/dalle-mini?tab=readme-ov-file#development).

You can read the [DALL-E Mini explained](https://wandb.ai/dalle-mini/dalle-mini/reports/DALL-E-Mini-Explained-with-Demo--Vmlldzo4NjIxODA) to know more about dataset, architecture, and algorithms involved.

You can read [Ultimate Guide to the Best Photorealistic AI Images & Prompts](https://www.craiyon.com/blog/ultimate-guide-best-ai-art-photorealistic-images-and-prompts) for better understanding with good resources.

DALL·E Mini has 14k+ Stars on GitHub, and it's currently at the `v0.1` release. 

{% cta https://github.com/borisdayma/dalle-mini %} Star DALL·E Mini ⭐️ {% endcta %}

---

## 6. [Deepgram](https://github.com/deepgram) - Build Voice AI into your apps.

![deepgram](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/32enxrtcwqk6g81eazay.png)

&nbsp;

From startups to NASA, Deepgram APIs are used to transcribe and understand millions of audio minutes every day. Fast, accurate, scalable, and cost-effective.

It provides speech-to-text and audio intelligence models for developers.

![deepgram options](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rdc3tqg7fvt3sw6ktle7.png)

Even though they have a freemium model, the limits on the free tier are sufficient to get you started.

The visualization is next-level. You can check live streaming response, or audio files and compare the intelligence levels of audio.

![streaming](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4wcvzzrqzn94gxe594hf.png)

![sentiment analysis](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uw6wkhzg7g6vgq7lphri.png)

You can read the [docs](https://developers.deepgram.com/docs/introduction).

You can also read a sample blog by Deepgram on [How to Add Speech Recognition to Your React and Node.js Project](https://deepgram.com/learn/how-to-add-speech-recognition-to-your-react-project). 

If you want to try the APIs to see for yourself with flexibility in models, do check out their [API Playground](https://playground.deepgram.com/?smart_format=true&language=en&model=nova-2).

{% cta https://github.com/deepgram %} Star Deepgram ⭐️ {% endcta %}

---

## 7. [InvokeAI](https://github.com/invoke-ai/InvokeAI) - leading creative engine for Stable Diffusion models.

![invoke AI](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a1uira3ta4ufauefp0ff.png)

&nbsp;

About

InvokeAI is an implementation of Stable Diffusion, the open source text-to-image and image-to-image generator. 

It runs on Windows, Mac, and Linux machines, and runs on GPU cards with as little as 4 GB of RAM.

The solution offers an industry-leading WebUI, supports terminal use through a CLI, and serves as the foundation for multiple commercial products.

![invoke ai](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g5802r0wtxlbkqdtclce.png)

You can read about [installation and hardware requirements](https://invoke-ai.github.io/InvokeAI/installation/INSTALLATION/), [How to install different Models](https://invoke-ai.github.io/InvokeAI/installation/050_INSTALLING_MODELS/), and most important [Automatic Installation](https://invoke-ai.github.io/InvokeAI/installation/010_INSTALL_AUTOMATED/). 

The exciting feature is the ability to generate an image using another image, as described in the [docs](https://invoke-ai.github.io/InvokeAI/features/IMG2IMG/).

InvokeAI has almost 21k stars on GitHub, 

{% cta https://github.com/invoke-ai/InvokeAI %} Star InvokeAI ⭐️ {% endcta %}

---

## 8. [OpenAI](https://github.com/openai) - Everything you will ever need.

![openAI](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k02duibi54zzzddck91z.png)

&nbsp;

Gemini by Google and OpenAI are very popular, but we're focusing on OpenAI for this list.

If you want to know more, you can read [Google AI Gemini API in web using React 🤖](https://generativeai.pub/google-gemini-api-in-web-using-react-7e5bf0bf0abc) on Medium. It's simple and to the point.

With OpenAI, you can use DALL·E (create original, realistic images and art from a text description), Whisper (speech recognition model), and GPT-4. Tell us about Sora in the comments!

You can start building with a simple API.

```javascript
completion = openai.ChatCompletion.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What are some famous astronomical observatories?"}
  ]
)
```

You can read the [docs](https://platform.openai.com/docs/introduction). It provides such an insane amount of options to build something very cool!

![docs overview](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o9yi0tar96jxi4pkni81.png)

Even Stripe uses GPT-4 to improve user experience.

For instance, you can create [Assistant applications](https://platform.openai.com/docs/assistants/overview) and see the [API playground](https://platform.openai.com/playground/p/default-chat?model=text-davinci-003) to understand it better.

![gpt-3](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t89658h4drhy4a8zf3xs.png)

In case you want a guide, you can read [Integrating ChatGPT With ReactJS](https://dzone.com/articles/integrating-chatgpt-with-reactjs-a-comprehensive-g) by Dzone.

Between, OpenAI acquired Sora to get the monopoly. What do you think?

{% cta https://github.com/openai %} Star OpenAI ⭐️ {% endcta %}

---

## 9. [DeepFaceLab](https://github.com/iperov/DeepFaceLab) - leading software for creating deepfakes.

![deepfacelab](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g32stb7uo201msv3jn8f.png)

&nbsp;

DeepFaceLab is the top open source tool for making deepfakes.

Deepfakes are altered images and videos made with deep learning. They're often used to swap faces in pictures or clips, sometimes as a joke, but also for harmful reasons. 

DeepFaceLab, built with Python is a strong deepfake tool. It can change faces in media and even erase wrinkles and signs of aging.

These are some of the things that you can do with DeepFaceLab.

- Replace the face.

![replace the face](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/86jnuec9l6eaalwf9w51.png)

- [De-age the face - YouTube](https://www.youtube.com/watch?v=Ddx5B-84ebo).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/axh2e6117felh4zhoh3p.png)

- Replace the head.

![replace the head](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nyvbncox7k1u28nait50.png)

- Manipulate the lips.

You can use this basic tutorial on [how to use DeepFaceLab](https://www.youtube.com/watch?v=kOIMXt8KK8M) in an effective way to accomplish these things.

You can see the videos on [YouTube](https://www.youtube.com/channel/UCGf4OlX_aTt8DlrgiH3jN3g/videos) that use this DeepLab algorithm.

Unfortunately, there is no "make everything ok" button in DeepFaceLab, but it's worthwhile to learn its workflow for your specific needs. 

Despite being archived on November 9, 2023, with nearly 44k+ stars on GitHub, it remains a solid choice for your AI app due to its numerous tutorials and reliable algorithms.

{% cta https://github.com/iperov/DeepFaceLab %} Star DeepFaceLab ⭐️ {% endcta %}

---

## 10. [Detectron2](https://github.com/facebookresearch/detectron2) - a PyTorch-based modular object detection library.

![detectron2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jxe7wuf8v8y7e039ziel.png)

&nbsp;

Detectron2 is Facebook AI Research's next-generation library that provides state-of-the-art detection and segmentation algorithms. It is the successor of Detectron and maskrcnn-benchmark. 
It supports several computer vision research projects and production applications on Facebook.

Use this [YouTube tutorial](https://www.youtube.com/watch?v=eUSgtfK4ivk) to use Detectron2 with Machine Learning from the Developer Advocate at Facebook.

Detectron2 is designed to support a diverse array of state-of-the-art object detection and segmentation models, while also adapting to the continuously evolving field of cutting-edge research.

You can read on [How to Get Started](https://detectron2.readthedocs.io/en/latest/tutorials/getting_started.html), and the [Meta Blog](https://ai.meta.com/blog/-detectron2-a-pytorch-based-modular-object-detection-library-/) which covers the objectives of Detectron in deep. 

The old version of Detectron used Caffe, making it hard to use with later changes in code that combined Caffe2 and PyTorch. Responding to community feedback, Facebook AI released Detectron2 as an updated, easier-to-use version.

Detectron2 comes with advanced algorithms for object detection like DensePose and panoptic feature pyramid networks. 

Plus, Detectron2 can do semantic segmentation and panoptic segmentation, which help detect and segment objects more accurately in images and videos.

Detectron2 not only supports object detection using bounding boxes and instance segmentation masks but also can predict human poses, similar to Detectron.

They have 28k+ Stars on GitHub repo and are used by 1.6k+ developers on GitHub.

{% cta https://github.com/facebookresearch/detectron2 %} Star Detectron2 ⭐️ {% endcta %}

---

## 11. [FastAI](https://github.com/fastai/fastai) - deep learning library.

![fast ai](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6qvxqd22q3qamqtvwe6p.png)

&nbsp;

Fastai is a versatile deep-learning library designed to cater to both practitioners and researchers. It offers high-level components for practitioners to achieve top-notch results quickly in common deep-learning tasks. 

At the same time, it provides low-level components for researchers to experiment and develop new approaches. 

Detectron2 achieves a balance between ease of use and flexibility through its layered architecture. 
This architecture breaks down complex deep-learning techniques into manageable abstractions, concisely using Python's dynamic nature and PyTorch's flexibility.

It is built on top of a hierarchy of lower-level APIs which provide composable building blocks. This way, a user wanting to rewrite part of the high-level API or add particular behavior to suit their needs does not have to learn how to use the lowest level.

![architecture api](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kfooe2mxrh3xplcxeg75.png)

Get started with the following command once you have [installed pyTorch](https://pytorch.org/get-started/locally/).
```
conda install -c fastai fastai
```

You can read the [docs](https://docs.fast.ai/).

They have different starting points as [tutorials](https://docs.fast.ai/tutorial.html) for beginners, intermediate, and experts.

If you're looking to contribute to FastAI, you should read their [code style guide](https://docs.fast.ai/dev/style.html).

If you're more into videos, you can watch [Lesson "0": Practical Deep Learning for Coders (fastai)](https://www.youtube.com/watch?v=gGxe2mN3kAg) on YouTube by Jeremy Howard.

They have 25k+ Stars on GitHub and are already used by 16k+ developers on GitHub.

{% cta https://github.com/fastai/fastai %} Star FastAI ⭐️ {% endcta %}

---

## 12. [Stable Diffusion](https://github.com/CompVis/stable-diffusion) - A latent text-to-image diffusion model.

![stable diffusion](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/63worvztgs1cmy2owtkf.png)

&nbsp;

> What is stable diffusion?

Stable diffusion refers to a technique used in generative models, particularly in the context of text-to-image synthesis, where the process of transferring information from text descriptions to images is done gradually and smoothly.

In a latent text-to-image diffusion model, stable diffusion ensures that the information from the text descriptions is diffused or spread consistently throughout the latent space of the model. This diffusion process helps in generating high-quality and realistic images that align with the given textual input.

Stable diffusion mechanisms ensure that the model doesn't experience sudden jumps or instabilities during the generation process. I hope this clears things up!

A simple way to download and sample Stable Diffusion is by using the [diffusers library](https://github.com/huggingface/diffusers/tree/main#new--stable-diffusion-is-now-fully-compatible-with-diffusers).

```python
# make sure you're logged in with `huggingface-cli login`
from torch import autocast
from diffusers import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_pretrained(
  "CompVis/stable-diffusion-v1-4", 
  use_auth_token=True
).to("cuda")

prompt = "a photo of an astronaut riding a horse on mars"
with autocast("cuda"):
    image = pipe(prompt)["sample"][0]  
    
image.save("astronaut_rides_horse.png")
```

You can read the [research paper](https://ommer-lab.com/research/latent-diffusion-models/) and more about [Image Modification with Stable Diffusion](https://github.com/CompVis/stable-diffusion?tab=readme-ov-file#image-modification-with-stable-diffusion).

For instance, this is the input.

![input](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zpvxxhrrvthd8w1a0rrl.png)

and this is the output after upscaling a bit.

![output](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gzqvd06kse8ifhzry0la.png)

Stable Diffusion v1 is a specific model configuration that employs an 860M UNet and a CLIP ViT-L/14 text encoder for the diffusion model, with a downsampling-factor 8 autoencoder. The model was pre-trained on 256x256 images and subsequently fine-tuned on 512x512 images.

They have around 64k+ Stars on the GitHub repository.

{% cta https://github.com/CompVis/stable-diffusion %} Star Stable Diffusion ⭐️ {% endcta %}

---

## 13. [Mocap Drones](https://github.com/jyjblrd/Mocap-Drones) - Low cost motion capture system for room scale tracking.

![mocap drones](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3hq4hnzbx2wtxboehosi.png)

&nbsp;

This project requires the SFM (structure from motion) OpenCV module, which requires you to compile OpenCV from the source. 

From the `computer_code` directory, run this command to install node dependencies.
```
yarn install

yarn run dev // to start the web server.
```

You will get a URL view of the frontend interface.

Open a separate terminal window and run the command `python3 api/index.py` to initiate the backend server. This server is responsible for receiving camera streams and performing motion capture computations.

The architecture is as follows.

![architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jt6e3f32scak65wfdp8s.png)

You can see this [YouTube video](https://www.youtube.com/watch?v=0ql20JKrscQ) to understand how Mocap drones work or the [blog](https://joshuabird.com/blog/post/mocap-drones) by the owner of the project.

{% embed https://www.youtube.com/watch?v=0ql20JKrscQ %}

You can read the [docs](https://github.com/jyjblrd/Mocap-Drones?tab=readme-ov-file#runing-the-code).

It is a recent open source project with 900+ stars on GitHub Repository.

{% cta https://github.com/jyjblrd/Mocap-Drones %} Star Mocap Drones ⭐️ {% endcta %}

---

## 14. [Whisper Speech](https://github.com/collabora/WhisperSpeech) - text-to-speech system built by inverting Whisper.

![whisper speech](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hpawahh7aqsh1pnsnu76.png)

&nbsp;


This model, similar to Stable Diffusion but for speech, is both powerful and highly customizable. 

The team ensures the use of properly licensed speech recordings, and all code is open source, making the model safe for commercial applications. 

Currently, the models are trained on the English LibreLight dataset.

You can study more about the  [architecture](https://github.com/collabora/WhisperSpeech?tab=readme-ov-file#architecture).

![architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hnfqick2y1yoxgkmwlk6.png)

You can hear the [sample voice](https://github.com/collabora/WhisperSpeech/assets/107984/aa5a1e7e-dc94-481f-8863-b022c7fd7434) and use [colab](https://colab.research.google.com/drive/1xxGlTbwBmaY6GKA24strRixTXGBOlyiw) to try out yourself.

They are fairly new and have around 3k+ stars on GitHub.

{% cta https://github.com/collabora/WhisperSpeech %} Star Whisper Speech ⭐️ {% endcta %}

---

## 15. [eSpeak NG](https://github.com/espeak-ng/espeak-ng) - speech synthesizer that supports more than a hundred languages and accents.

![eSpeak NG](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a28zdxcr1jthb5bht2fi.png)

&nbsp;

The eSpeak NG is a compact open source software text-to-speech synthesizer for Linux, Windows, Android, and other operating systems. It supports more than 100 languages and accents. It is based on the eSpeak engine created by Jonathan Duddington.

You can read the [installation guide](https://github.com/espeak-ng/espeak-ng/blob/master/docs/guide.md) on various systems.

For Debian-like distributions (e.g. Ubuntu, Mint, etc.). You can use this command.
```
sudo apt-get install espeak-ng
```

You can see the list of [supported languages](https://github.com/espeak-ng/espeak-ng/blob/master/docs/languages.md), read the [docs](https://github.com/espeak-ng/espeak-ng/tree/master?tab=readme-ov-file#documentation) and see [features](https://github.com/espeak-ng/espeak-ng/tree/master?tab=readme-ov-file#features).

This model translates text into phoneme codes, suggesting its potential power as a front end for another speech synthesis engine.

They have 2.7k+ Stars on GitHub and 

{% cta https://github.com/espeak-ng/espeak-ng %} Star eSpeak NG ⭐️ {% endcta %}

---

## 16. [Chatbot UI](https://github.com/mckaywrigley/chatbot-ui) - AI chat for every model.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k8smowkv6scq9lujjeab.png)

&nbsp;

We all have used ChatGPT, and this project can help us set up the user interface for any AI Chatbot. One less hassle!

You can read the [installation guide](https://github.com/mckaywrigley/chatbot-ui?tab=readme-ov-file#1-install-docker) to install docker, supabase CLI, and other stuff.

You can read the [docs]() and see the [demo](https://twitter.com/mckaywrigley/status/1738273242283151777?s=20).

This uses Supabase (Postgres) under the hood so that is why we discussed it before.

I didn't discuss the Vercel AI chatbot because it's a fairly new comparison to this one.

Chatbot UI has around 25k+ Stars on GitHub so it is still the first choice for developers to build a UI interface for any chatbot.

{% cta https://github.com/mckaywrigley/chatbot-ui %} Star Chatbot UI ⭐️ {% endcta %}

---

## 17. [GPT-4 & LangChain](https://github.com/mayooear/gpt4-pdf-chatbot-langchain) - GPT4 & LangChain Chatbot for large PDF docs.

![chat architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0pe0xehimhyw2mfubzu9.png)

&nbsp;

This can be used for the new GPT-4 API to build a chatGPT chatbot for multiple Large PDF files.

The system is built using LangChain, Pinecone, Typescript, OpenAI, and Next.js. LangChain is a framework that simplifies the development of scalable AI/LLM applications and chatbots. Pinecone serves as a vector store for storing embeddings and your PDFs in text format, enabling the retrieval of similar documents later on.

You can read the [development guide](https://github.com/mayooear/gpt4-pdf-chatbot-langchain?tab=readme-ov-file#development) that involved cloning, installing dependencies, and setting up environments API keys.

You can see the [YouTube video](https://www.youtube.com/watch?v=ih9PBGVVOO4) on how to follow along and use this.

They have 14k+ Stars on GitHub with just 34 commits. Try it out in your next AI app!

{% cta https://github.com/mayooear/gpt4-pdf-chatbot-langchain %} Star GPT-4 & LangChain ⭐️ {% endcta %}

---

## 18. [Amica](https://github.com/semperai/amica) - allows you to easily chat with 3D characters in your browser.

![amica](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2nvizcn717h3cteocft5.png)

&nbsp;

Amica is an open source interface for interactive communication with 3D characters with voice synthesis and speech recognition.

You can import VRM files, adjust the voice to fit the character, and generate response text that includes emotional expressions.

They use three.js, OpenAI, Whisper, Bakllava for vision, and many more. You can read on [How Amica Works](https://docs.heyamica.com/overview/how-amica-works) with the [core concepts](https://docs.heyamica.com/overview/core-concepts) involved.

You can clone the repo and use this to [get started](https://docs.heyamica.com/getting-started/installation).
```npm
npm i

npm run dev
```

You can read the [docs](https://docs.heyamica.com/) and see the [demo](https://amica.arbius.ai/) and it's pretty damn awesome :D

![demo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/92iv9y2auly6tvenee82.png)

You can watch this short video on what it can do.

{% embed https://www.youtube.com/watch?v=hUxAEnFiXH8 %}

Amica uses Tauri to build the desktop application. Don't worry, we have covered Tauri later in this list.

They have 400+ Stars on GitHub and it seems very easy to use.

{% cta https://github.com/semperai/amica %} Star Amica ⭐️ {% endcta %}

---

## 19. [Hugging Face Transformers](https://github.com/huggingface/transformers) - state-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

![Hugging Face Transformers](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c3acbf1f145jihy4pqar.png)

&nbsp;

Hugging Face Transformers provides easy access to state-of-the-art pre-trained models and algorithms for tasks such as text classification, language generation, and question answering. The library is built on top of PyTorch and TensorFlow, allowing users to seamlessly integrate advanced NLP capabilities into their applications with minimal effort. 

With a vast collection of pre-trained models and a supportive community, Hugging Face Transformers simplifies the development of NLP-based solutions.

These models can be deployed to perform text-related tasks like text classification, information extraction, question answering, summarization, translation, and text generation, in more than 100 languages. 
They can also handle image-related tasks such as image classification, object detection, and segmentation, as well as audio-related tasks like speech recognition and audio classification. 
They can also perform multitasking on various modalities, including table question answering, optical character recognition, information extraction from scanned documents, video classification, and visual question answering.

You can see plenty of [models](https://huggingface.co/models) that are available.

You can explore the [docs](https://huggingface.co/docs/transformers/task_summary) for complete objectives and samples showing you about the various tasks you can perform. 

For example, one way to use the pipeline is for image segmentation.

```python
from transformers import pipeline

segmenter = pipeline(task="image-segmentation")
preds = segmenter(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
)
preds = [{"score": round(pred["score"], 4), "label": pred["label"]} for pred in preds]
print(*preds, sep="\n")
```

Transformers are supported by the three most widely used deep learning libraries, namely Jax, PyTorch, and TensorFlow, with seamless integration among them. This integration enables easy training of models using one library, followed by loading them for inference with another library.

They have around 120k+ stars on GitHub and are used by a massive 142k+ developers. Try it out!

{% cta https://github.com/huggingface/transformers %} Star Hugging Face Transformers ⭐️ {% endcta %}

---

## 20. [LLAMA](https://github.com/facebookresearch/llama) - Inference code for LLaMA models.

![llama](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bia2hnh4i79w9ljj1c4l.png)

&nbsp;

Llama 2 is a cutting-edge technology developed by Facebook Research that empowers individuals, creators, researchers, and businesses of all sizes to experiment, innovate, and scale their ideas responsibly using large language models. 

The latest release includes model weights and starting code for pre-trained and fine-tuned Llama language models ranging from 7B to 70B parameters.

Get started with the [installation guide](https://github.com/facebookresearch/llama?tab=readme-ov-file#quick-start) covering the below steps.

- Clone and download the repository.
- Install the required dependencies.
- Register and download the model/s from the Meta website.
- Run the provided script to download the models.
- Run the desired model locally using the provided commands.

You can see this [YouTube video](https://www.youtube.com/watch?v=OqZ0CSKzu10) by ZeroToMastery on what a is llama.

You can also see the list of models and more info on [Hugging Face](https://huggingface.co/meta-llama) and [Meta official page](https://llama.meta.com/).

Ollama is based on llama and this has 50k+ stars on GitHub. See the docs and use this model for more research.  

{% cta https://github.com/facebookresearch/llama %} Star LLAMA ⭐️ {% endcta %}

---

## 21. [Fonoster](https://github.com/fonoster/fonoster) - The open source alternative to Twilio.

![fonoster](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pruup1a8yibepdi40fjk.png)

Fonoster Inc. researches an innovative Programmable Telecommunications Stack that will allow for an entirely cloud-based utility for businesses to connect telephony services with the Internet.

&nbsp;

There are various ways to start based on what you want to achieve.

Get started with the following npm command.
```npm
npm install @fonoster/websdk
// CDN is also available
```

For instance, this is how you can use Fonoster with Google Speech APIs. (you will need the key for the service account)

```npm
npm install @fonoster/googleasr @fonoster/googletts
```

This is how you can configure the Voice Server to use plugins.

```javascript
const { VoiceServer } = require("@fonoster/voice");
const GoogleTTS = require("@fonoster/googletts");
const GoogleASR = require("@fonoster/googleasr");
const voiceServer = new VoiceServer();
const speechConfig = { keyFilename: "./google.json" };

// Set the server to use the speech APIS
voiceServer.use(new GoogleTTS(speechConfig));
voiceServer.use(new GoogleASR(speechConfig));

voiceServer.listen(async(req, res) => {
  console.log(req);
  await res.answer();
  // To use this verb you MUST have a TTS plugin
  const speech = await res.gather();

  await res.say("You said " + speech);
  await res.hangup();
});
```

You can read the [docs](https://fonoster.com/docs/overview/).

They provide a free tier which is good enough to get started.

They have around 6k+ stars on GitHub and have more than 250 releases.

{% cta https://github.com/fonoster/fonoster %} Star Fonoster ⭐️ {% endcta %}

---

## 22. [DIPY](https://github.com/dipy/dipy) - paragon 3D/4D+ imaging library in Python.

![dipy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l2y1ztg24l2wc1kq5u0g.png)

&nbsp;

DIPY stands as the leading 3D/4D+ imaging library in Python. It contains various methods for spatial normalization, signal processing, machine learning, statistical analysis, and visualization of medical images.

Plus, it contains specialized methods for computational anatomy including diffusion, perfusion, and structural imaging.

You can get started with.
```
pip install dipy

// run this in python console
import dipy
print(dipy.get_info())
```

You can read the complete [installation guide](https://docs.dipy.org/stable/examples_built/quick_start/quick_start.html#sphx-glr-examples-built-quick-start-quick-start-py) if you're using anaconda or another system.

You can read the [docs](https://docs.dipy.org/stable/) and visit their [YouTube channel](https://www.youtube.com/c/diffusionimaginginpython).

You can see the [examples](https://docs.dipy.org/stable/examples_built/index.html) in detail.

![examples](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3b6x3jotln0chpoycmci.png)

They have 428k+ downloads and have 600+ Stars on the GitHub repository.

{% cta https://github.com/dipy/dipy %} Star DIPY ⭐️ {% endcta %}

---

## 23. [Elastic Search](https://github.com/elastic/elasticsearch) - Free and Open, Distributed, RESTful Search Engine.

![elastic search](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ouw3u41qdkfjvt999lnv.png)

![data fit](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tsn875yov9bmklfg9aqc.png)

&nbsp;

Elasticsearch is a distributed, RESTful search and analytics engine capable of addressing plenty of use cases. 

As the heart of the Elastic Stack, it centrally stores your data for lightning-fast search, fine‑tuned relevancy, and powerful analytics that scale with ease.

They have stated the use cases of using ElasticSearch.

![use cases](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sp4qf45yzulbi4c7dire.png)

Elasticsearch uses standard RESTful APIs and JSON. We also build and maintain clients in many languages such as Java, Python, .NET, SQL, and PHP.

This structure is as follows.

```javascript
const { Client } = require('@elastic/elasticsearch')
const client = new Client({ node: 'http://localhost:9200' })

client
  .search({
    index: 'social-*',
    body: {
      query: { match: { message: 'myProduct' } },
      aggs: {
        top_10_states: {
          terms: { field: 'state', size: 10 }
        }
      }
    }
  })
  .then(({ body }) => {
    const { hits } = body.hits
    console.log(hits)
  })
  .catch(console.error)
```

You can read the [docs]() and see the [list of features](https://www.elastic.co/elasticsearch/features).

Despite its useful features, the main drawback of Elastic Search is the absence of a free tier. However, you can still utilize the free trial to explore and learn about the open-source project's architecture. 

With over 67k+ stars on GitHub and nearly 1900 contributors, as well as being on the `v8` release, Elastic Search is continuously evolving and improving.

{% cta https://github.com/elastic/elasticsearch %} Star Elastic Search ⭐️ {% endcta %}

---

## 24. [Tauri](https://github.com/tauri-apps/tauri) - Build smaller, faster, and secure desktop applications with a web frontend.

![tauri](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7z6iilytnkaw5d3uj6zv.png)

&nbsp;

Tauri is a toolkit designed to assist developers in creating applications for major desktop platforms, utilizing virtually any frontend framework available. Its core is developed with Rust, while the CLI leverages Node.js, offering a truly polyglot approach to developing and maintaining exceptional apps.

The user interface in Tauri apps currently leverages Tao as a window-handling library on macOS, Windows, Linux, Android, and iOS. 

To render your application, Tauri utilizes WRY, a library that offers a unified interface to the system webview. It leverages WKWebView on macOS & iOS, WebView2 on Windows, WebKitGTK on Linux, and Android System WebView on Android.

You can use Vite, HTML/CSS/JS, Next.js, Svelte, and many more.

Get started with the following npm command.
```npm
npm create tauri-app@latest
```

You can read the [docs](https://tauri.app/v1/guides/getting-started/prerequisites) and see the [list of features](https://tauri.app/v1/guides/features/) that you get with Tauri.

You can even make your own CLI with Tauri, how cool is that :)

The team has provided a [YouTube video](https://www.youtube.com/watch?v=UxTJeEbZX-0&t=2s) for you to know more about Tauri.

They have 75k+ Stars on GitHub and have more than 800 releases.

{% cta https://github.com/tauri-apps/tauri %} Star Tauri ⭐️ {% endcta %}

---

## 25. [AutoGPT](https://github.com/Significant-Gravitas/AutoGPT) - more exciting than ChatGPT.

![auto gpt](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3hjamyxzkhy7luwsi9vp.png)

&nbsp;

At the core of AutoGPT lies its primary project, a semi-autonomous agent driven by large language models (LLMs), designed to perform any tasks for you.

The AutoGPT project consists of [four main components](https://docs.agpt.co/#agent):

- The Agent – also known as just "AutoGPT"
- The Benchmark – AKA agbenchmark
- The Forge
- The Frontend

Read on how you can [set up AutoGPT](https://docs.agpt.co/autogpt/setup/) using the OpenAI key.

You can see this [YouTube video by Fireship](https://www.youtube.com/watch?v=_rGXIXyNqpk) on what is AutoGPT.

{% embed https://www.youtube.com/watch?v=_rGXIXyNqpk %}

You can also watch this [AutoGPT tutorial](https://www.youtube.com/watch?v=FeIIaJUN-4A) by Sentral Media.

You can read the [docs](https://docs.agpt.co/) and check out the [project board](https://github.com/orgs/Significant-Gravitas/projects/1) on what things are under development right now.

Even if you don't know much about AI, you can try AutoGPT to understand how you can save time and build cool stuff.

They have around 159k+ stars on GitHub Repo due to such an excellent use case and automation capabilities. 

{% cta https://github.com/Significant-Gravitas/AutoGPT %} Star AutoGPT ⭐️ {% endcta %}

---

It's not over yet.
Now, let's explore some valuable resources that will help you learn new concepts and make a better AI app.
We will keep it simple. Hang on!

- [Artificial Intelligence (AI) courses, books, video lectures, and papers.](https://github.com/owainlewis/awesome-artificial-intelligence
)
- [Machine-Learning / Deep-Learning / AI + Web3 -Tutorials](https://github.com/TarrySingh/Artificial-Intelligence-Deep-Learning-Machine-Learning-Tutorials)
- [ML for Beginners](https://github.com/microsoft/ML-For-Beginners) - 12 weeks, 26 lessons, 52 quizzes, classic Machine Learning for all.
- [Machine learning frameworks, libraries, and software](https://github.com/josephmisiti/awesome-machine-learning)
- [How to Make an AI: A Step-by-Step Guide - Revelo](https://www.revelo.com/blog/how-to-make-an-ai)

Hopefully, this will help you learn more concepts!

---

I hope you found something useful in the list.

I've covered awesome open source projects that can take your AI app to the next level. 

AI is changing the world and it's better to keep AI as a friend rather than ignore it. 

Use it to be productive and take the chance to develop something extraordinary.

Some open source projects are more useful than others especially Taipy & AutoGPT if you're trying to improve your project in the best possible way. 

Comment down to let us know which project surprised you the most.

Have a great day! Till next time.

Follow me on GitHub.

{% embed https://github.com/Anmol-Baranwal %}

Follow Taipy for more content like this.

{% embed https://dev.to/taipy %}