First 15 Open Source Advent projects
Tags: opensource, llm, tutorial, programming
Published: 2023-12-16 03:33:41

##Just 10 days to go!
We launched [Open Source Advent](https://zilliz.com/open-source-advent) at the begininng of this month to celebrate 25 different open source projects. It has been fun sharing these projects and I thought I would reshare the first 15 projects! Take a look at the repo, try the tutorial and let us know what you build! 

Naturally, everyone who worked on these **Open-Source** projects would love a little Christmas 🎄💕 love by getting a GitHub star for their projects.

<img width="100%" style="width:100%" src="https://media.giphy.com/media/9YhFTjYhRYDXGpVBhJ/giphy.gif">

###1. Milvus by Zilliz | [Github](https://github.com/milvus-io/milvus) 
[Milvus](https://milvus.io/) is an open-source vector database that powers embedding similarity search and AI applications and strives to make vector search accessible to every organization. Milvus can store, index, and manage a billion+ embedding vectors generated by deep neural networks and other machine learning (ML) models. It is the project we all work on here at [Zilliz](https://zilliz.com/), so, of course it is on the list. 😇

###2. FiftyOne by Voxel51 | [Github](https://github.com/voxel51/) | [tutorial]()
FiftyOne is the open source toolkit for building high-quality datasets and computer vision models. With FiftyOne you can visualize, curate, manage, and QA data, and automate the workflows that make enterprise machine learning possible. They spoke at the last Unstructured Meetup and you can check out the recording [here](https://www.youtube.com/watch?v=3rURx8yGM7w) (29:10 - Speaker Jacob Marks, Vector search with computer vision data using Voxel51)

###3.  Quivr | [GitHub](https://github.com/StanGirard/quivr) | [tutorial](https://github.com/StanGirard/quivr)
Quivr is your personal productivity assistant to chat with your dumped files (PDF, CSV) & apps using GPT 3.5 / 4 turbo, Private, Anthropic, VertexAI, LLMs that you can share with users! Alternative to OpenAI GPTs.

###4.  Haystack by Deepset | [Github](https://github.com/deepset-ai/haystack) | [tutorial]()
Haystack is an end-to-end NLP framework that enables you to build applications powered by LLMs, Transformer models, vector search, and more. Whether you want to perform question answering, answer generation, semantic document search, or build tools capable of complex decision-making and query resolution, you can use state-of-the-art NLP models with Haystack to build end-to-end NLP applications to solve your use case. We have a [video](https://zilliz.com/event/retrieval-augmentation-haystack) on some examples of retrieval augmentation in Haystack.

###5. Proton by Timeplus | [Github](https://github.com/timeplus-io/proton) | [tutorial](https://docs.timeplus.com/proton#get-started)
Proton is a streaming analytics database, based on ClickHouse, written in C++. Fast. Powerful, Easy

###6. Ydata-synthetic and Ydata-profiling by YData | [Github](https://github.com/ydataai/ydata-profiling) | [tutorial](https://www.youtube.com/watch?v=ep0PhwsFx0A)
Ydata-profiling is a Python package for automated Data Quality profiling reports in a single line of code. Ydata-synthetic is a package to generate synthetic tabular and time-series data with state-of-the-art generative models.

###7. Apache Flink | [Github](https://github.com/apache/flink) | [tutorial](https://github.com/polyzos/advent-of-code-flink-paimon)
Apache Flink is the leading framework and distributed processing engine for stateful computations over unbounded and bounded data streams.


###8. LangChain RB | [Github](https://github.com/andreibondarev/langchainrb) | [tutorial](https://medium.com/@rushing_andrei/building-a-basic-rag-retrieval-augmented-generation-system-in-a-rails-app-247ccce5d1d2)
LangChain RB is an original Langchain-inspired Ruby framework. The goal is to abstract complexity and difficult concepts to make building AI/ML-supercharged applications approachable for traditional Ruby software engineers. If you are a Ruby fan, we have a [video](https://www.youtube.com/watch?v=78qfHVRq2Qg) to show you how to build a GenAI App End-to-End with Ruby

###9. Flyte by Union AI | [Github](https://github.com/flyteorg/flyte) | [tutorial](https://docs.flyte.org/projects/cookbook/en/latest/index.html)
Flyte is an open-source orchestrator that facilitates building production-grade data and ML pipelines. It is built for scalability and reproducibility, leveraging Kubernetes as its underlying platform. With Flyte, user teams can construct pipelines using the Python SDK and seamlessly deploy them in both cloud and on-premises environments, enabling distributed processing and efficient resource utilization.


###10. DVC by Iterative | [Github](https://github.com/iterative/dvc) | [tutorial](https://github.com/iterative/dvclive/tree/main/examples)
DVC is a command line tool to help you develop reproducible machine learning projects.

**But Wait!, There's More!**
<img width="100%" style="width:100%" src="https://media.giphy.com/media/11FRmJRii0I8iA/giphy.gif">

###11. Phoenix by Arize AI | [Github](https://github.com/Arize-ai/phoenix) | [tutorial](https://docs.arize.com/phoenix/notebooks )
Phoenix is Arize AI's open-source observability library designed for experimentation, fine-tuning, and troubleshooting your LLM, CV, and NLP models in a notebook.

###12. TruLens by TruEra | [Github](https://github.com/truera/trulens) | [tutorial](https://youtu.be/U3c0nOT4Cl4)
Observability of LLM and Multimodal apps with deep instrumentation and comprehensive evals.

###13. OpenLLM by BentoML | [Github](https://github.com/bentoml/OpenLLM) | tutorial
OpenLLM is an open-source platform designed to facilitate the deployment and operation of large language models (LLMs) in real-world applications. With OpenLLM, you can run inference on any open-source LLM, deploy them on the cloud or on-premises, and build powerful AI applications.

###14. LabelStudio by Human Signal | [Github](https://github.com/HumanSignal/label-studio/) | [tutorial]()
A flexible data labeling tool for all data types. Prepare training data for computer vision, natural language processing, speech, voice, and video models.


###15. LlamaIndex | [Github](https://github.com/run-llama/llama_index) | [tutorial](https://blog.llamaindex.ai/create-llama-a-command-line-tool-to-generate-llamaindex-apps-8f7683021191)
LamaIndex is a data framework for LLM-based applications to ingest, structure, and access private or domain-specific data.