Understanding FAISS. ....And the world of Similarity Searching | by Vedashree Patil | Towards Data Science

A few weeks back, I stumbled upon FAISS — Facebook’s library for similarity search for very large datasets. My interest piqued, and a few hours of digging around on the internet led me to a treasure trove of knowledge. In this post, I hope to pen down (or rather type down) few basic concepts associated with the library. And in my subsequent post, I will dig a little deeper and explore some more advanced concepts.
Usually in similarity searching, there is often a query record that is compared against a stored database of records (documents or images etc). The main aim is to retrieve a set of database records that are similar to the query record. So, if you have a picture of a dog, a similarity search should give you a list of pictures with dogs (not rainbows!) in them.
When Machine learning comes into picture, the database corresponds to a collection of vectors. Vectors can be seen as high dimensional representations of the input data generated by machine learning algorithms. Similarity searching in this context means searching for similar vectors for a given query vector based on some similarity or distance measure.
A naive way for searching based on similarity is to compare the query vector with every other vector in the database. But what if the database has more than a million vectors? Enter FAISS....
FAISS is a C++ library (with python bindings of course!) that assures faster similarity searching when the number of vectors may go up to millions or billions.
At its very heart lies the index. Mind you, the index is everywhere!(albeit in different forms and names). In this post, I’ll elaborate on one: “the inverted file index ” or “IVF”
Let me tell you a little story (Bear with me over here, I promise its important...)
The queen of Someland had just conquered a new territory and discovered that the natives of this land were segregated in three different tribes who simply couldn’t tolerate each other. So, to avoid feuds, she decided to build three separate cities for each of the tribes. Interestingly, each tribe had a distinctive set of skills that they practised. This helped the queen identify which tribe a person belonged to. The Greens, it seemed, were born with a green thumb and spent most of their time tending to plants. The Tinks were the smart, analytical bunch and mostly comprised of architects, builders and scientists. And finally there were the creative Whims, who were known for their proficiency in the fine arts. They were mostly poets, dancers or artists.
The cities were built and the tribe chiefs were appointed as the representatives for their respective tribes. For administrative purposes, the queen’s ministers maintained a “master-book” that recorded the name of the tribe chief along with names of the citizens for each city.
One day, a group of travellers arrived in the kingdom and asked for refuge. The queen now had a problem. She had to find suitable volunteers who would agree to accommodate the travellers in their homes. (Apparently there were no inns in the kingdom). She knew that the citizens were quite guarded and only warmed up to people who shared their interests.
Finally, a solution was found. For each traveller, the chiefs decided which tribe he or she would belong to depending on how closely the traveller’s characteristics matched the tribe’s characteristics. When the travellers arrived in their respective cities, a few citizens came forth and volunteered to accommodate one of the travellers since they felt that the traveller’s characteristics matched theirs. And well.... Problem solved! (The queen could now sleep in peace).
Viola! I just gave you a bird’s eye view of how the inverted index works.
Let’s use the story above as an analogy.
All the citizens of the kingdom are the vectors in a database and the three tribes correspond to three separate “clusters” or “cells”. The vectors are then assigned to one of these three clusters depending upon a certain similarity measure .(Remember the tribes practised different set of skills that set them apart from each other?). Usually the L2 distance measure along with a clustering algorithm like K-means is used for this.
Like the tribe chiefs in our story, each cluster is represented by a cluster centroid or “code” . And just like the ministers’ “master book” , a separate “codebook” is maintained that keeps track of the code (or cluster centroid) and its corresponding vectors for each cluster. This is essentially the “Inverted File” or index.
A Quantiser is used to decide which cluster the vector belongs to (I guess this was primarily the queen’s job). So when a query vector comes in (like the travellers in our story), a suitable cluster (or clusters) is found for the query vector based on its similarity with the cluster centres (just like the tribe chiefs who selected the travellers based on their characteristics). Finally, a select number of similar vectors within the selected cluster(s) that are returned as the query result (like the citizens of the city who volunteered to house a traveller). This can be seen as a very basic working of the inverted index.
Firstly, install the FAISS library with the python bindings. Just follow the instructions given at : https://github.com/facebookresearch/faiss/blob/master/INSTALL.md
Then, import the library and other dependencies in python using:
import numpy as np import faiss  # this will import the faiss library
Now, let’s create some vectors for the database. FAISS requires the dimensions of the database vectors to be predefined. We create about 200 vectors with dimension size 128. This creates a (200 * 128) vector matrix. Note that all vector values are stored in the float 32 type.
dimension = 128    # dimensions of each vector                         n = 200    # number of vectors                   np.random.seed(1)             db_vectors = np.random.random((n, dimension)).astype('float32')
We use the ‘IndexIVFFlat’ index type for our vectors. The ‘Flat’ here signifies that the vectors are stored as is without any compression or quantisation (more on that later). The IVF index takes two parameters:
nlist : to specify the number of clusters to be formed
quantizer : to assign the vectors to a particular cluster. This is usually another index that uses the L2 distance metric (we use the FlatL2 index)
nlist = 5  # number of clustersquantiser = faiss.IndexFlatL2(dimension)  index = faiss.IndexIVFFlat(quantiser, dimension, nlist,   faiss.METRIC_L2)
The index has to be trained initially to create ‘nlist’ number of clusters and then the vectors are added to these clusters. the ‘is_trained’ flag denotes whether the index is trained or not and the ‘ntotal’ attribute shows the total number of vectors added to the index.
print(index.is_trained)   # Falseindex.train(db_vectors)  # train on the database vectorsprint(index.ntotal)   # 0index.add(db_vectors)   # add the vectors and update the indexprint(index.is_trained)  # Trueprint(index.ntotal)   # 200
Next, we perform a search on the index for 10 query vectors. The ‘nprobe’ parameter specifies the number of clusters to visit during the search operation. This can be seen as hyper-parameter which can be tuned to get different results. Note that ‘nprobe’ cannot exceed ‘nlist’.
‘k’ specifies the number of similar vectors to be returned from the visited clusters.
nprobe = 2  # find 2 most similar clustersn_query = 10  k = 3  # return 3 nearest neighboursnp.random.seed(0)   query_vectors = np.random.random((n_query, dimension)).astype('float32')distances, indices = index.search(query_vectors, k)
The search operation will return the ids (row numbers or index in the vector store) of the k most similar vectors for each query vector along with their respective distances.
distances: [[15.770459 16.773014 17.17131  17.439615 17.443993] [16.476107 18.52229  18.811913 18.974785 19.02668 ] [15.520995 16.500256 17.069542 17.483345 17.742079] [16.842718 17.712341 17.828487 18.699772 19.345257] [18.325392 18.495464 18.684456 18.70239  18.904179] [17.531885 18.18179  18.331263 19.429993 19.700233] [16.840158 17.03664  17.091755 17.34306  17.487806] [15.984037 16.380917 17.270592 17.620832 17.663855] [18.018503 18.0761   18.766172 18.956903 18.985767] [17.113918 17.385284 17.65757  18.122086 18.170212]]indices: [[185  35  96  80  75] [118  51 122 108  31] [148 149 173  27  84] [175 177  50  38  81] [ 44 144 174 105  70] [156  74 151 167 182] [ 57 144  18 174  74] [ 82  12  46  64 127] [ 52  73  59 138 131] [ 82  46  90  37 179]]
The index can be saved on disk using the write_index() function and can be loaded later using the using the read_index() function
faiss.write_index(index,"vector.index")  # save the index to diskindex = faiss.read_index("vector.index")  # load the index
There! A rudimentary code to understand faiss indexes!
FAISS has a handful of features including:
GPU and multithreaded support for index operations
Dimensionality reduction: vectors with large dimensions can be reduced to smaller dimensions using PCA
Quantisation: FAISS emphasises on product quantisation for compressing and storing vectors of large dimensions
Batch processing i.e searching for multiple queries at a time
Similarity searching and information retrieval are old pals! Image retrieval or document retrieval and even recommender systems use similarity searching. Say, for example, when you are shopping online for a watch of a particular brand, you see all kinds of watches similar in nature in your recommended list.
Another route to explore is classification. For example, if we take the cliched ‘cats and dogs’ image recognition example, we can actually predict if the given query image is of a cat or a dog, depending on the most similar images returned from a datastore of cat and dog images(hmm....definitely another post for this!)
I feel like I have barely scraped the surface! FAISS is an interesting library and theres definitely a lot more to explore. Squeezing everything in one post might make you scroll for ages! Hence, my next post further dives into the library and explains an advanced concept called product quantisation.
github.com
Thanks for reading!!!