Package: aws-rag-bot
A Serverless RAG Chatbot Framework using AWS OpenSearch, AWS Bedrock and LangChainThis is a RAG chatbot application based on AWS components and designed to be optimized for a serverless architecture
and cost optimized for a high-volume mobile application use case.  The AWS Services being used are:AWS Bedrock with the Titan LLM and Titan embeddingsOpenSearch (aka Elasticsearch) vector database - this is instance based but can be serverless (not implemented yet)The frameworks I created here abstract out a variety of components to enable easily testing variations.  This makes it easier to tune the implementation based on the
content being use in my applications.  The goal is to play with different LLM's, different embeddings, parameters like temperature, top_p and more.
Content loading also needed to be highly refined allowing me to control each source easily.This library is meant more to provide a complete working set of examples and less to be an out-of-the-box library to use as-is.Features and aspects of interestLangChain- Great framework building Generative AI applications.LLM Model Support- Defaults to AWS Bedrock Titan LLM, but supports other Bedrock models (LLama 2, Jurassic) OpenAI (GTP-3.5 and GTP-4), Google GeminiLangChain LLM Callbacks- Example of using LLM Callbacks, provided here for custom costingConversational Memory-- Designed to manage chat history memory from the client side - to support a serverless modelOpenSearch- an AWS hosted semantic search service which has a vector database that is valuable in the retrieval feature of RAGOpenSearch loading library- making it easier to load multiple content sources into an OpenSearch vector database.Langchain LCEL Chains- for more flexible chaining of steps in the RAG appMultiple embedding models- Defaults to Bedrock Titan, but supports OpenAI and Hugging FaceWeb and directory crawlers- for loading content into the vector DB.  Lots of fine-tuning to document selection features - whitelisting, black listing, etc.Prompt library management- making it easier to implement query routing to optimize for specific domains of questionsLangSmith logging- for logging and debuggingHow to usePrerequisitesPython 3.11 (may work on older, but this was the target version used)AWS Account with keys defined in .aws/credentials fileAWS OpenSearch Domain created and accessible to the AWS account credentials aboveAWS Bedrock with Titan Express LLM (or other LLM supported by this code) - you may need to request accessIdeally a LangChain (LangSmith) API Key defined in .env file (or removed if not using LangSmith) for logging and monitoringIf using google gemini or openai, you will need to have an account and keys for those services and have them defined in the .env filePython dependencies installed (dependency manager for this project coming soon)Setting up the project locallyThis project uses Poetry to manage dependencies.  You can install it with the following command:pipinstallpoetryThen you can install the dependencies with the following command:poetryinstallAlternatively, you can install the package from PyPi with the following command:pipinstallaws-rag-botSample CodeThis is a very simple, high-level example.  Check out the rag_bot_code_samples.ipynb for a more.
First step is to have content in your vector database.fromopen_search_vector_db.aws_opensearch_vector_databaseimportOpenSearchVectorDBLoadercontent_sources=[{"name":"Internal KB Docs","type":"PDF","location":"kb-docs"}]vectordb_loader=OpenSearchVectorDBLoader(domain_name=my_open_search_domain_name,index_name=my_index_name,data_sources=content_sources)vectordb_loader.load()Then you can start asking questions of itfromrag_chatbotimportRagChatbot,LlmModelTypesfromprompt_libraryimportDefaultPromptschatbot=RagChatbot(my_open_search_domain_name,LlmModelTypes.BEDROCK_TITAN_EXPRESS,prompt_model=NasaSpokespersonPrompts)chat_history=[]question="What...?"# Ask a question related to the content you loadedresponse=chatbot.ask_question(question,chat_history,verbose=True)print(response)Running chatbot_client.pyA very simple command line client program has been created as an example and tool to test.  It is called chatbot_client.py.It is a simple command line program that will ask a question and then print the response while retaining the chat history for context.pythonchatbot_client.pymy-opensource-domain-nameReferencesVector Database:May references show using Chroma and FAISS, but I needed a solution that worked well in a Lambda serverless environment.Ideally it would be at AWS keeping my stack uniform.https://aws.amazon.com/what-is/vector-databases/Ultimately I chose OpenSearch because of cost, support by LangChain and a serverless version I plan to evaluateVector Database Loader:LangChain has a great library of DataLoaders for loading data into OpenSearch.  I wanted an effective way
to scrape a website with help from this article chose to use Selenium.https://www.comet.com/site/blog/langchain-document-loaders-for-web-data/I also used the directory loader and plan to implement cloud based directory loaders in the future.  Primarily S3 and Google Drive.Langsmith Logging and Debugging:I used LangSmith for logging and debugging.  It is a great tool for this purpose.https://docs.smith.langchain.comWhat's next (Roadmap):Add vector database stub for FAISS and easy testingExpand range of support for Embeddings - including Hugging Face cloud service for embeddings