# Leveraging MongoDB Atlas Vector Search with LangChain

## Introduction to Vector Search in MongoDB Atlas

Vector search engines — also termed as vector databases, semantic search, or cosine search — locate the closest entries to a specified vectorized query. While the conventional search methods hinge on keyword references, lexical match, and the rate of word appearances, vector search engines measure similarity by the distance in the embedding dimension. Finding related data becomes searching for the nearest neighbors of your query. 

Vector embeddings act as the numeric representation of data and its accompanying context, preserved in high-dimensional (dense) vectors. There are various models, both proprietary (like those from OpenAI and Hugging Face) and open-source ones (like FastText), designed to produce these embeddings. These models can be trained on millions of samples to deliver results that are both more pertinent and precise. In certain situations, the numeric data you've gathered or designed to showcase essential characteristics of your documents might serve as embeddings. The crucial part is to have an efficient search mechanism, like MongoDB Atlas.

), choose _Search_ and _Create Search Index_. Please also visit the official MongoDB documentation to learn more.

 in your user settings.

To install LangChain, you'll first need to update pip for Python or npm for JavaScript, then use the respective install command. Here are the steps:

For Python version, use:

```
pip3 install pip --upgrade
pip3 install langchain
```

We will also need other Python modules, such as ``pymongo`` for communication with MongoDB Atlas, ``openai`` for communication with the OpenAI API, and ``pypdf` `and ``tiktoken``  for other functionalities.

```
pip3 install pymongo openai pypdf tiktoken
```

### Start using Atlas Vector Search 

In our exercise, we utilize a publicly accessible PDF document titled "MongoDB Atlas Best Practices" as a data source for constructing a text-searchable vector space. The implemented Python script employs several modules to process, vectorize, and index the document's content into a MongoDB Atlas collection.

In order to implement it, let's begin by setting up and exporting the environmental variables. We need the Atlas connection string and the OpenAI API key.

```
export OPENAI_API_KEY="xxxxxxxxxxx"
export ATLAS_CONNECTION_STRING="mongodb+srv://user:passwd@vectorsearch.abc.mongodb.net/?retryWrites=true"
```

Next, we can execute the code provided below. This script retrieves a PDF from a specified URL, segments the text, and indexes it in MongoDB Atlas for text search, leveraging LangChain's embedding and vector search features. The full code is accessible on GitHub.

```
import os
from pymongo import MongoClient
from langchain.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import MongoDBAtlasVectorSearch

# Define the URL of the PDF MongoDB Atlas Best Practices document
pdf_url = "https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE4HkJP"

# Retrieve environment variables for sensitive information
OPENAI_API_KEY = os.getenv('OPENAI_API_KEY')
if not OPENAI_API_KEY:
    raise ValueError("The OPENAI_API_KEY environment variable is not set.")

ATLAS_CONNECTION_STRING = os.getenv('ATLAS_CONNECTION_STRING')
if not ATLAS_CONNECTION_STRING:
    raise ValueError("The ATLAS_CONNECTION_STRING environment variable is not set.")

# Connect to MongoDB Atlas cluster using the connection string
cluster = MongoClient(ATLAS_CONNECTION_STRING)

# Define the MongoDB database and collection names
DB_NAME = "langchain"
COLLECTION_NAME = "vectorSearch"

# Connect to the specific collection in the database
MONGODB_COLLECTION = clusterDB_NAME][COLLECTION_NAME]

# Initialize the PDF loader with the defined URL
loader = PyPDFLoader(pdf_url)
data = loader.load()

# Initialize the text splitter
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=150)

# Split the document into manageable segments
docs = text_splitter.split_documents(data)

# Initialize MongoDB Atlas vector search with the document segments
vector_search = MongoDBAtlasVectorSearch.from_documents(
    documents=docs,
    embedding=OpenAIEmbeddings(),
    collection=MONGODB_COLLECTION,
    index_name="default"  # Use a predefined index name
)
# At this point, 'docs' are split and indexed in MongoDB Atlas, enabling text search capabilities.
```

Upon completion of the script, the PDF has been segmented and its vector representations are now stored within the ``langchain.vectorSearch`` namespace in MongoDB Atlas.

![embedding results][5]

### Execute similarities searching query in Atlas Vector Search

"`MongoDB Atlas auditing`"  serves as our search statement for initiating similarity searches. By utilizing the  `OpenAIEmbeddings` class, we'll generate vector embeddings for this phrase. Following that, a similarity search will be executed to find and extract the three most semantically related documents from our MongoDB Atlas collection that align with our search intent.

In the first step, we need to create a  ``MongoDBAtlasVectorSearch`` object:

```
def create_vector_search():
    """
    Creates a MongoDBAtlasVectorSearch object using the connection string, database, and collection names, along with the OpenAI embeddings and index configuration.

    :return: MongoDBAtlasVectorSearch object
    """
    vector_search = MongoDBAtlasVectorSearch.from_connection_string(
        ATLAS_CONNECTION_STRING,
        f"{DB_NAME}.{COLLECTION_NAME}",
        OpenAIEmbeddings(),
        index_name="default"
    )
    return vector_search
```

Subsequently, we can perform a similarity search.

```
def perform_similarity_search(query, top_k=3):
    """
    This function performs a similarity search within a MongoDB Atlas collection. It leverages the capabilities of the MongoDB Atlas Search, which under the hood, may use the `$vectorSearch` operator, to find and return the top `k` documents that match the provided query semantically.

    :param query: The search query string.
    :param top_k: Number of top matches to return.
    :return: A list of the top `k` matching documents with their similarity scores.
    """

   # Get the MongoDBAtlasVectorSearch object
    vector_search = create_vector_search()
    
    # Execute the similarity search with the given query
    results = vector_search.similarity_search_with_score(
        query=query,
        k=top_k,
    )
    
    return results

# Example of calling the function directly
search_results = perform_similarity_search("MongoDB Atlas auditing")
```

The function returns the most semantically relevant documents from a MongoDB Atlas collection that correspond to a specified search query. When executed, it will provide a list of documents that are most similar to the query "`MongoDB Atlas auditing`".  Each entry in this list includes the document's content that matches the search along with a similarity score, reflecting how closely each document aligns with the intent of the query. The function returns the top k matches, which by default is set to 5 but can be specified for any number of top results desired. Please find the [code on GitHub. 

## Summary

MongoDB Atlas Vector Search enhances AI applications by facilitating the embedding of vector data into MongoDB documents. It simplifies the creation of search indices and the execution of KNN searches through the ``$vectorSearch``  MQL stage, utilizing the Hierarchical Navigable Small Worlds algorithm for efficient nearest neighbor searches. The collaboration with LangChain leverages this functionality, contributing to more streamlined and powerful semantic search capabilities
. Harness the potential of MongoDB Atlas Vector Search and LangChain to meet your semantic search needs today!

In the next blog post, we will delve into LangChain Templates, a new feature set to enhance the capabilities of MongoDB Atlas Vector Search. Alongside this, we will examine the role of retrieval-augmented generation (RAG) in semantic search and AI development. Stay tuned for an in-depth exploration in our upcoming article!

Questions? Comments? We’d love to continue the conversation over in the Developer Community forum.

  [1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blte7a2e75d0a8966e6/6553d385f1467608ae159f75/1.png
  [2]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltdc7192b71b0415f1/6553d74b88cbdaf6aa8571a7/2.png
  [3]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltfd79fc3b47ce4ad8/6553d77b38b52a4917584197/3.png
  [4]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blta6bbbb7c921bb08c/65a1b3ecd2ebff119d6f491d/atlas-search-create-search-index.png
  [5]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt627e7a7dd7b1a208/6553d7b28c5bd6f5f8c993cf/4.png