NeuML/neuml-linkedin-202501 huggingface dataset:
LinkedIn post by David Mezzetti (07/25/2024):
🥴 Text extraction is a messy business!

There are libraries that don't work well, require numerous dependencies, have licenses that are commercially untenable (AGPL-3) and/or require sending data to a remote API for processing!

With txtai, we use Apache Tika as our main text extraction library. It works well for most formats but it does require Java. Keep in mind that a headless JRE is smaller than dependencies other text extraction libraries require such as LibreOffice.

txtai also plays nice with others. If you'd like to use an external library for PDF parsing, check this example out. Just be aware of the license for this library!

Link to code: https://lnkd.in/dt2z7W_j
LinkedIn post by David Mezzetti (07/21/2024):
Did you know that a txtai embeddings database is a file format? 

While txtai can integrate with a number of external components, it's base components all save content locally. The entire database can be saved to a singled compressed file. There is built-in support for saving these indexes to cloud storage (i.e. S3) and the HF Hub.

Learn more about the file formats behind this here: https://lnkd.in/e3j5Pf-n
LinkedIn post by David Mezzetti (07/20/2024):
📈 Let's talk about Graph RAG. We've been looking at graph-based approaches for context generation since 2022. The best use case we've seen for Graph RAG is for more complex questions and research. For example, think of a problem as a road trip with multiple stops. A graph path traversal is a great way to pick up various concepts as context, concepts which may not be directly related and not picked up by a simple keyword/vector search.

The attached image shows two graph path traversal examples. The first shows the path between a squirrel and the Red Sox winning the world series. The 2nd shows an image path from a person parachuting and someone holding a french horn. Note the progression of both the text and images along the way. There is also another example of traversing history from the end of the Roman Empire to the Norman Conquest of England.

For problems like this, graphs do a great job. If the answer is a simple retrieval of a single entry, Graph RAG doesn't add much value. Like all things, Graph RAG isn't the be-all and end-all.

Read more in the articles below.

Semantic Graph Intro: https://lnkd.in/eMsAXarn
Graph RAG: https://lnkd.in/d-BSjuj7
LinkedIn post by David Mezzetti (07/19/2024):
🔒 Want to work with vectors, LLMs and RAG but worried about security?

Did you know that txtai has full Postgres integration (dense vectors, sparse vectors, content and graph)? This can be combined with standard row level security to limit what content a user can utilize for GenAI processes.

Learn more at the links below.

Article: https://lnkd.in/eFeFNgYK
Postgres docs: https://lnkd.in/e9aFks5Y
LinkedIn post by David Mezzetti (07/16/2024):
🚀 New in txtai 7.3: Streaming RAG. This feature builds off streaming LLMs which iteratively return chunks of content as a stream vs waiting for the entire generation call. Streaming is supported with Transformers, llama.cpp and LLM APIs (i.e. GPT-4, Claude, Ollama)

Try it out with txtai's new RAG application. Plug in a new txtai embeddings index to use this application with your own data.

Docker image: https://lnkd.in/d-fJiVXR
RAG Application: https://lnkd.in/e82fddyY
LinkedIn post by David Mezzetti (07/16/2024):
txtai is a grassroots project dedicated to the developer-experience. We're passionate about building a quality project. GitHub ⭐'s and 🔥 comments go a long way.

NeuML is building a sustainable and profitable company from the start. We provide consulting services in situations where it's an interesting business problem. It's not about generating buzz to get the next funding round or rubbing elbows with big names in Silicon Valley.

The majority of our work is open-source and from our desk to yours with ❤️ Thank you!
LinkedIn post by David Mezzetti (07/15/2024):
🚀 txtai 7.3 is out! 

This release adds a new RAG front-end template, streaming LLM and streaming RAG support along with significant text extraction improvements.

From local to remote vectorization, model inference and data storage - txtai has you covered. It's the easiest way to build vector search, LLM and RAG systems without the bloat. 

See below for more.

GitHub:https://lnkd.in/dxWDeey
Release Notes: https://lnkd.in/e_4kYXma
PyPI: https://lnkd.in/eE_Jvft
Docker Hub: https://lnkd.in/e598zTHb

API Clients:
Python: https://lnkd.in/eqVx_nqt
JavaScript​: https://lnkd.in/dM8ua2y
Rust: https://lnkd.in/d2MAae2
Java: https://lnkd.in/dqmmjTw
Go: https://lnkd.in/dq7Ujv4
LinkedIn post by David Mezzetti (07/12/2024):
😎 All the cool kids have a RAG framework in 2024. Why? Because it's really easy. A simple RAG framework can be a couple lines of code. If all you do is call API services and stitch the results together, what value does this add?

txtai is much more than this. It's a local vector database that can also store data in Postgres. A LLM framework that works with multiple LLM backends local and remote. A sophisticated RAG pipeline. Not to mention components for graphs, BM25 and other traditional ML model pipelines. It does all this without creating unnecessary complexity and abstraction.

And it's open-source, check it out: https://lnkd.in/dxWDeey
LinkedIn post by David Mezzetti (07/11/2024):
We often hear one say they have a LLM and they want to solve a problem. LLMs aren't always the best tool for the job. Let's take text classification using a sentiment dataset.

Running LLM prompts for this dataset only leads to 58% accuracy! Training with a 4.4M parameter model has 91% accuracy. BERT is 93%. Sure we can fine-tune the LLM for this task but why spend an hour vs 8 minutes?

Be willing to accept the simpler solution.

Link to code: https://lnkd.in/d38xBm6X
LinkedIn post by David Mezzetti (07/09/2024):
🤔 Machine translation just ask a LLM to do it?

While LLMs can translate that doesn't mean they should. What if we could utilize smaller models that were trained to translate between specific languages? What if there was a pipeline that automatically loads models based on the source to target language? 

🚀 Enter txtai's translation pipeline! The Translation pipeline automatically detects languages and searches the Hugging Face Hub for the best specialized model to perform the translation. These specialized models are often smaller than LLMs and much faster. 

Link to code: https://lnkd.in/eefCwRDu
LinkedIn post by David Mezzetti (07/08/2024):
No GPU available? Only using external API services? Want llama.cpp GPU models with txtai? 

Did you know that Torch has a CPU-only install that brings a significantly smaller dependency chain (no PyPI CUDA libraries). The txtai-cpu Docker image employs this same strategy. It reduces the image size from 3.1 GB to 700 MB.

Link to code: https://lnkd.in/exnasWHf
LinkedIn post by David Mezzetti (07/07/2024):
🗎 Want to summarize webpages, word documents, PDFs and more? Did you know there are models pre-built for summarization that pre-date the latest LLMs? And that they do a decent job and are faster? 

txtai supports pre-trained summarization models and LLMs for summarization. Either can be run as Python workflows or FastAPI services.

Link to code: https://lnkd.in/emg4QUfm
LinkedIn post by David Mezzetti (07/07/2024):
✨ Let's build on our previous BM25 post and take tokenization into account. We'll compare LangChain's BM25 retriever, the recently released bm25s library (built with Scipy sparse matrices) and txtai. We'll use the same tokenization method for all 3, the Unicode Text Segmentation algorithm (UAX 29). Keep in mind that this is relevant to those using hybrid search (vector + keyword).

1.6M ArXiv abstracts were evaluated. txtai's index time was slower but search times were significantly faster. txtai used almost 6x less RAM.

Link to code: https://lnkd.in/ekSxN9pJ
LinkedIn post by David Mezzetti (07/06/2024):
✂️ Let's talk tokenization, an underappreciated part of the NLP pipeline! Naive methods like splitting on whitespace work for European languages but not with others. Stop words were once a common pattern but have since fallen out of favor.

Modern keyword tokenizers split using the Unicode Text Segmentation algorithm (UAX 29). This enables broader language support. Many Transformers models use either word/subword or BPE tokenizers.

txtai has a built-in tokenizer that implements UAX 29. This functionality is similar to what's found in systems like Elasticsearch/Apache Lucene and it's used with txtai's sparse keyword indexing.

Link to the code: https://lnkd.in/egVRgEAT
LinkedIn post by David Mezzetti (07/05/2024):
⚡ LangChain and LlamaIndex both use the Rank-BM25 library to provide in-line BM25 document retrieval. Rank-BM25 is a great way to quickly stand up a BM25 search index for a small number of documents. But it doesn't scale as it's built to run in memory.

txtai has it's own BM25 implementation in Python. Term vectors are built harnessing the native performance of the Python arrays package. These term vectors are stored in a SQLite database. LRU caching stores frequently used vectors in memory. This combination of factors enables a highly performant index.

For this comparison, 2.3M ArXiv abstracts were used. LangChain ran out of memory (32 GB of RAM). The test was scaled to 1.6M abstracts. txtai had 2x slower index times but 13x faster search times than LangChain. LangChain used 25 GB of RAM, txtai used 3.8 GB of RAM.

Link to code: https://lnkd.in/eD7qn6qn
LinkedIn post by David Mezzetti (07/05/2024):
💥⚾ BM25 continues to be a heavy hitter in the information retrieval space. Did you know that txtai has a BM25 component built for speed💨? 

BM25 term vectors are built harnessing the native performance of the Python arrays package. These term vectors are stored in a SQLite database. LRU caching stores frequently used vectors in memory. This combination of factors enables a highly performant index. 

https://lnkd.in/eA3ui6cQ
LinkedIn post by David Mezzetti (07/04/2024):
💡 Retrieval Augmented Generation (RAG) is one of the most practical use cases of the Generative AI era. An LLM when presented with a bounding context often will generate factually grounded answers.

txtai makes RAG with your documents easy. It has pipelines to extract text from Office and PDF documents while preserving structured formatting (i.e. tables, lists). It has an easy-to-use LLM pipeline that automatically loads models from Hugging Face, llama.cpp and APIs (OpenAI, Ollama etc).

See how this compares to RAG with LangChain (txtai was able to generate the correct answer given it preserves table formatting): https://lnkd.in/evv_7gi6
LinkedIn post by David Mezzetti (07/03/2024):
💥 Hnswlib is a great vector indexing library. It's integrated into a number of vector databases.

txtai utilizes the same core pipeline for generating embeddings and storing vectors regardless of the end components. There has been a careful focus in building a highly performant and efficient vector database implementation that runs great locally.

txtai uses mmap-ing and other techniques to ensure that memory limits are respected. Streaming vector generation and offloading those vectors during index creation allows txtai to build large local indexes whereas other implementations run out of memory. 

See how this compares to Chroma DB (txtai is 3x faster for the same dataset): https://lnkd.in/eY_nMA85
LinkedIn post by David Mezzetti (07/01/2024):
⚡ Faiss is a great vector indexing library. It has so many features past just a flat index. txtai automatically creates a performant Faiss index scaled by the size of the incoming data. The index type can also be fully customized with configuration. This shows the power of a full-featured and long-standing integration.

See how this compares to LlamaIndex: https://lnkd.in/eWqU5z3U
LinkedIn post by David Mezzetti (06/30/2024):
What do we get with txtai out of the box? txtai vector indexes use SQLite + Faiss by default. This enables search with SQL and dynamic columns. Results are standard Python dictionaries and that allows direct integration with Pandas/Polars DataFrames.

See how this compares to LangChain: https://lnkd.in/esX88rPR
LinkedIn post by David Mezzetti (06/30/2024):
A fundamental part of any RAG solution is the data source.

txtai is an all-in-one embeddings database with support for storing data as local file-based indexes. Did you know that txtai has built-in support for storing these indexes as a Hugging Face model and cloud storage such as with AWS S3 buckets? These composable indexes can be built and shared for RAG.

https://lnkd.in/eMGY7uRB

A couple example datasets are linked below.

txtai-wikipedia: https://lnkd.in/eQz5dKtG
txtai-arxiv: https://lnkd.in/eSCCs-Jz
LinkedIn post by David Mezzetti (06/27/2024):
Curious about RAG? Not a programmer and want to experiment? Well check out these easy-to-use series of RAG applications packaged as Docker images! Everything needed is built in.

Wikipedia: https://lnkd.in/d-fJiVXR
ArXiv: https://lnkd.in/dMx8Sfk2

All code and configuration used to build these images can be found on txtai's GitHub repo: https://lnkd.in/dxWDeey
LinkedIn post by David Mezzetti (06/22/2024):
🤔 Curious about how Retrieval Augmented Generation (RAG) works? Then check out this easy-to-understand article covering how txtai RAG works!

This article shows how to create RAG processes in Python. It also covers standing up low code RAG API services with FastAPI and Docker.

https://lnkd.in/eExBX_3A
LinkedIn post by David Mezzetti (06/17/2024):
Why are so many AI projects failing? Unrealistic expectations has to be at the top. But another often overlooked item is picking too complex a stack. Many of the popular AI frameworks try to support integrating everything leading to unnecessary complexity.

txtai follows the KISS principle with it's architecture. It's designed to get up and running fast but also scale to production

https://lnkd.in/dxWDeey
LinkedIn post by David Mezzetti (05/31/2024):
🚀 txtai 7.2 is out! 

This release adds Postgres integration for all components, LLM Chat Messages and vectorization with llama.cpp/LiteLLM

From local to remote vectorization, model inference and data storage - txtai has you covered. It's the easiest way to build vector search, LLM and RAG systems without the bloat. 

See below for more.

GitHub: https://lnkd.in/dxWDeey
Release Notes: https://lnkd.in/eKe56dwE
PyPI: https://lnkd.in/eE_Jvft
Docker Hub: https://lnkd.in/e598zTHb

API Clients:
Python: https://lnkd.in/eqVx_nqt
JavaScript: https://lnkd.in/dM8ua2y
Rust: https://lnkd.in/d2MAae2
Java: https://lnkd.in/dqmmjTw
Go: https://lnkd.in/dq7Ujv4
LinkedIn post by David Mezzetti (05/12/2024):
txtai has a unique feature where it can persist indexes to object storage (i.e. S3) along with other systems such as the Hugging Face Hub. This adds a large level of customizability with the same code.

From Postgres to S3 and the Hugging Face Hub, txtai has you covered.

https://lnkd.in/eMGY7uRB
LinkedIn post by David Mezzetti (05/07/2024):
Is Postgres all you need? Is a vector just a data type? That's a tough question. On one hand, dedicated vector databases have a lot of catching up to do in terms of almost 30 years of functionality. On the other, there are advantages to reimagining the architecture factoring in all we know in 2024. 

The good news with txtai is that it's capable of working with multiple setups. It can persist data to Postgres. It can store data in Faiss + SQLite. It can also integrate content with other vector databases. The idea is to have everything needed to get started fast and be flexible to change as the requirements and landscape evolves.

https://lnkd.in/dxWDeey
LinkedIn post by David Mezzetti (04/30/2024):
One unique feature of txtai is it's ability to mix and match vector, content, graph and keyword index systems together. Out of the box, local defaults are set to get up and running fast. But txtai provides a high level of flexibility in integrating different components together. It also provides it's own SQL dialect for querying regardless of the underlying choices made. The architecture is designed to make it easy to add new file formats and integrations.

https://lnkd.in/edem7iNX
LinkedIn post by David Mezzetti (04/25/2024):
🚀 Big news! We're excited to release this new Postgres + pgvector integration for txtai. It's now possible to fully persist txtai content, vectors and graph data to Postgres. From there it can be queried through txtai and/or directly with any Postgres client!

From prototyping to production, txtai has you covered.

https://lnkd.in/eFeFNgYK
LinkedIn post by David Mezzetti (04/19/2024):
🎉 We're excited to release txtai 7.1

This release adds dynamic embeddings vector support along with semantic graph and RAG improvements.

See below for more.

GitHub: https://lnkd.in/dxWDeey
Release Notes: https://lnkd.in/eiJWhTA6
PyPI: https://lnkd.in/eE_Jvft
Docker Hub: https://lnkd.in/e598zTHb

API Clients:
Python: https://lnkd.in/eqVx_nqt
JavaScript: https://lnkd.in/dM8ua2y
Rust: https://lnkd.in/d2MAae2
Java: https://lnkd.in/dqmmjTw
Go: https://lnkd.in/dq7Ujv4
LinkedIn post by David Mezzetti (04/02/2024):
Want to build agent workflows? Then take a look at txtai.

txtai has long (since 2021) had a framework for connecting different pipelines into unified workflows. 

This can be used to connect LLM prompts and/or specialized models for translation/summarization/text extraction.

Read this to learn more: https://lnkd.in/em2ew5ia
LinkedIn post by David Mezzetti (04/01/2024):
🚀 We're excited to announce a new 500M parameter model 🌌 Space Time LLM. 

Recent breakthroughs in LLMs have resulted in an uncanny and game changing ability to predict future outcomes. Impressive advances in quantization and compression such as 1-bit LLMs have contributed to this phenomenal breakthrough in predictive capabilities. This model redefines our understanding of what and how LLMs learn.

Check out this model and see what you can predict today!

https://lnkd.in/exdy4R5G
LinkedIn post by David Mezzetti (03/28/2024):
txtai is developed in the open. The full project history and design decisions are documented. What tests run and all code is easily accessible. Documentation is a priority and examples are provided for all major features. There are no secrets. It takes courage to develop with this level of transparency.

We're proud of our code quality, design decisions and code consistency. There is no "just get it done and we'll fix it later" mentality here.

https://lnkd.in/dxWDeey
LinkedIn post by David Mezzetti (03/17/2024):
We're seeing a lot of progress with methods to improve the efficiency of vector embeddings generation. For example, vector models trained with Matryoshka Representation Learning, push the most important information to the front of the vector, enabling us to only keep a portion of an embeddings vector.

Another method, quantization, compresses the number of values that can be represented by each bucket in an embeddings vector. With binary quantization, the values can even be reduced down to a single bit.

While all this is exciting, how do we know that these methods will work well enough for our requirements? We have to test it of course! The BEIR evaluation framework has a number of sources and it's easy to add new custom sources to test. Check out this article for a full example on this topic.

https://lnkd.in/db5Jx_fE
LinkedIn post by David Mezzetti (03/11/2024):
⚡ The new mxbai-embed-2d-large-v1 embeddings model from mixedbread is impressive! Performance with 384 dimensions using 2D Matryoshka Sentence Embeddings is superb. This model works out of the box with txtai - see below.

Model: https://lnkd.in/dnGAMnaH
Gist: https://lnkd.in/e5PeHxBB
LinkedIn post by David Mezzetti (03/02/2024):
The best compliment on txtai is that it's easy to get up and running. The goal is to have a well-documented project that has a number of reliable, easy-to-use features of the box. It's great to see the number of positive comments over on Reddit's r/LocalLLaMA subreddit. 

https://lnkd.in/ePMrZXi6
LinkedIn post by David Mezzetti (03/01/2024):
Would you trade 1% of accuracy to only have to store 1% of the data?

Exciting to see the innovation happening in the vector space and we're not talking about 1.58-bit LLMs.

With Matryoshka Embeddings, we can drastically reduce vector dimensionality while maintaining a strong level of accuracy. Check out this example that combines Matryoshka Embeddings with Faiss 4-bit scalar quantization 🚀
LinkedIn post by David Mezzetti (02/24/2024):
🚀 Excited to release an updated version of PubMedBERT Embeddings with Matryoshka Representation Learning support! With this model, dynamic embeddings sizes of 64, 128, 256, 384 and 512 can be used in addition to the full size of 768. It's a great way to save space with a relatively low level of accuracy tradeoff.

Thank you to Tom Aarsen, Philipp Schmid and the Hugging Face team for adding this feature to Sentence Transformers!

https://lnkd.in/edNYMyrF
LinkedIn post by David Mezzetti (02/22/2024):
🚀 🌕 We're excited to release txtai 7.0 🎉

This major release introduces the next generation of the semantic graph. It adds support for graph search, advanced graph traversal and graph RAG. It also adds binary support to the API, index format improvements and training LoRA/QLoRA models.

See below for more.

GitHub: https://lnkd.in/dxWDeey
Release Notes: https://lnkd.in/eD_vkBFu
Article: https://lnkd.in/efuQipp9
PyPI: https://lnkd.in/eE_Jvft
Docker Hub: https://lnkd.in/e598zTHb

API Clients:
Python: https://lnkd.in/eqVx_nqt
JavaScript: https://lnkd.in/dM8ua2y
Rust: https://lnkd.in/d2MAae2
Java: https://lnkd.in/dqmmjTw
Go: https://lnkd.in/dq7Ujv4
LinkedIn post by David Mezzetti (02/17/2024):
If you're looking to get started in RAG, vector search and/or with LLMs, txtai-wikipedia is a great datasource. It's a txtai index with all Wikipedia article abstracts organized by popularity (views). 

It's one of the easiest ways to have vector search across a broad range of topics.

https://lnkd.in/eQz5dKtG
LinkedIn post by David Mezzetti (02/16/2024):
Simple RAG systems run a vector query and use that as context. Graph RAG enables much more complexity and nuance.

Let's say you're a geopolitical analyst and you want to study the events from WW2 to the 1989 revolutions in Eastern Europe. Just searching for "Cold War" isn't going to pull everything you need. With Graph RAG you can draw a path between WW2->Cold War->1989 and pull in other concepts along the way. Like having a shopping cart and going down the aisles. You can also study the first two decades of the 2000s. 

Keep in mind that each node here is a Wikipedia article. Imagine what this could do with your data.
LinkedIn post by David Mezzetti (02/16/2024):
The new Graph RAG feature from txtai is a gamechanger. It's a whole new way to pull information as context, a composable graph. Check out these graphs of knowledge built using Wikipedia articles (txtai-wikipedia).

Keep in mind that these graphs are automatically built using vector similarity and path finding queries. Think of it like taking a highlighter and circling a series of concepts to research.

More to come soon.
LinkedIn post by David Mezzetti (02/14/2024):
RAG in it's simplest form is a vector search as context with a LLM prompt.

With the next version of txtai, we'll have a series of new graph-based RAG techniques. Think of this like a road trip with a number of pit stops. 

Say you're researching the early medieval history of England. Sure we can run a vector search for that. But what if we can instruct a query to traverse a number of concepts we're interested in? 

Let's take the example below. This is a network of Wikipedia articles (via txtai-wikipedia). The query traverses paths of history between the Roman Empire, Anglo-Saxon period, Viking period and ends with the Norman conquest. This rich dataset is then available as a library of context to downstream LLM prompts.

Graph databases aren't new. The difference here is that txtai builds a vector store and uses that to automatically build a graph network weighted by vector similarity. Load your data and you get this for free. 😀