Why FAISS Works | Towards Data Science

Accurate, fast, and memory-efficient similarity search is a hard thing to do — but something that, if done well, lends itself very well to our huge repositories of endless (and exponentially growing) data.
The reason that similarity search is so good is that it enables us to search for images, text, videos, or any other form of data — without getting too specific in our search queries — which is something that we humans are not so great at.
I will use the example of image similarity search. We can take a picture, and search for similar images. This works by first converting every image into a set of automatically generated features — which are represented as a numerical vector.
When we then compare the vectors of two similar images — we will find that they are very similar.
Now, if we took our picture, and searched for other similar images — we don’t really want to compare our query vector to every single vector in the database. Imagine trying to do that with a Google Image search — you could be waiting some time.
Instead, we want to find a more efficient approach — and that is exactly what Facebook AI Similarity Search (FAISS) offers.
Our story of efficient similarity search begins with vectors — many of them in fact.
If we were to take three vectors:
We can say quite confidently that vectors a and b are closer to each other than c. And this can be visualized with a simple 3-dimensional chart:
In our case, our vectors have many more dimensions — too many to visualize. But we can still use the same distance metrics to calculate the proximity and/or similarity between two highly-dimensional vectors. A few of those are:
Euclidean distance (measures magnitude)
Dot product (measures direction and magnitude)
Cosine similarity (measure direction)
FAISS makes use of both Euclidean distance and dot product for comparing vectors.
Given our vectors a, and b. We calculate the Euclidean distance as:
Given our Euclidean distance metric, we can move onto creating the nearest neighbors (NN) graph using a set of vectors.
NN is an essential component of FAISS, it is how we build the core ‘distance’ property in our index. However, NN-search is computationally heavy due to the curse of dimensionality.
The process consists of calculating the Euclidean distance between two vectors, and then another two, and so on — the nearest neighbors are those with the shortest distance between them.
The most basic approach to NN-search is the brute-force, exhaustive search — where we calculate the distance between all elements.
Now, if we have a few million vectors indexed — or a billion (as is the number that FAISS is built for) — this can become a heavy operation where we sacrifice speed and memory efficiency, for accuracy:
Brute ForceGood AccuracyBad SpeedBad Memory (eg a lot of memory is required)
So, we can instead perform a non-exhaustive search that does not search all elements in our index and/or transforms the vectors to make them smaller (and therefore faster to compare against).
Because this non-exhaustive (approximate) NN-search uses slightly modified vectors or a restricted search area — it returns an approximate best result, which is not necessarily the absolute best result.
Approximate NNReasonable AccuracyGood SpeedGood Memory
This degradation in accuracy however is very slight and viewed as a fair trade-off for the performance benefits.
First, FAISS uses all of the intelligent ANN graph-building logic that we’ve already learned about. But there’s more to FAISS.
The first of those efficiency savings comes from efficient usage of the GPU, so the search can process calculations in parallel rather than in series — offering a big speed-up.
Additionally, FAISS implements three additional steps in the indexing process. A preprocessing step, followed by two quantization operations — the coarse quantizer for inverted file indexing (IVF), and the fine quantizer for vector encoding.
For many of us, that last paragraph is nonsensical gibberish — so let’s break each step down and understand what each means.
We enter this process with the vectors that we would like FAISS to index. The very first step is to transform these vectors into a more friendly/efficient format. FAISS offers several options here.
PCA — use principal component analysis to reduce the number of dimensions in our vectors.
L2norm — L2-normalize our vectors.
OPQ — rotates our vectors so they can be encoded more efficiently by the fine quantizer — if using product quantization (PQ).
Pad — pads input vectors with zeros up to a given target dimension.
This operation means that when we do get around to comparing our query vector against already embedded vectors, each comparison will require less computation — making things faster.
The next step is our inverted file (IVF) indexing process. Again, there are multiple options — but each one is aiming to partition data into similar clusters.
This means that when we query FAISS, and our query is converted into a vector — it will be compared against these partition/cluster centroids.
We compare similarity metrics against our query vector and each of these centroids — and once we find the nearest centroid, we then access all of the full vectors within that centroid (and ignore all others).
Immediately, we have significantly reduced the required search area — reducing complexity and speeding up the search.
This is also called the ‘non-exhaustive’ search component — eg the component that allows us to avoid an ‘exhaustive’ (compare everything) search.
The final step is a final encoding step for each vector before it is indexed. This encoding process is carried out by our fine quantizer. The goal here is to reduce index memory size and increase search speed.
There are several options:
Flat — Vectors are stored as is, without any encoding.
PQ — Applies product quantization.
SQ — Applies scalar quantization.
It’s worth noting that even with the Flat encoding, FAISS is still going to be very fast.
All of these steps and improvements combine to create an incredibly fast similarity search engine — which on GPU is still unbeaten.
That’s all for this article on FAISS! FAISS is an incredibly cool tool, and you can read more about it here, and the FAISS repo here.
You can find Python implementations of FAISS in this notebook.
I hope you’ve enjoyed the article. Let me know if you have any questions or suggestions via Twitter or in the comments below. If you’re interested in more content like this, I post on YouTube too.
Thanks for reading!
🤖 NLP With Transformers Course
*All images are by the author except where stated otherwise