Building a Productized AI Chatbot for Credit Card Business
Source: tai_blog

IntroductionImagine youre a customer with an urgent question about your credit card. You call customer support  but the wait time is long  and the process is frustrating. This is where our AI chatbot comes in  transforming how customer support is handled in the credit card business.   When building a smarter  faster  and more secure customer support system  I realized the need for a chatbot that isnt just a novelty but a practical  production-ready tool. This chatbot uses AI technologies to ensure its efficient  secure  and easy to deploy.   I used Azure OpenAI to create the chatbot because its the best tool for understanding and responding to customer questions. However  keeping customer information safe is also important. To do this  I added Amazon Comprehend Moderation to protect personal data.   To make the chatbot more practical  I added a PostgreSQL database for specific data queries in the credit card business.   Deploying this chatbot is straightforward thanks to Docker containers  making it easy to scale and manage. Using tools like ChainLit and ConversationBufferWindowMemory  the chatbot can maintain a conversational history. This provides an excellent and personalized customer experience.   In this post  I want to show how these technologies combine to create a powerful AI chatbot that transforms customer support for credit cards. This setup can also be easily adapted for other businesses  like retail.   Items and FrameworkLets dive into the AI chatbot for credit card customer support by breaking down the entire framework  piece by piece.   AzureOpenAI for Intelligent Responses:   I chose Azure OpenAI as the brain of our chatbot because it fits perfectly with our needs as an enterprise application. It works better with other Azure services like AZURE_AI_SEARCH_SERVICE and data storage  making integration smooth and efficient. In addition  Azure OpenAI has robust security features and meets industry standards  necessary for keeping our data safe and secure.   Embedding Models and Data Chroma for Information Retrieval:   We need to store and find all the information the chatbot might retrieve. Embedding models and data Chroma acts like a well-organized library where each piece of information has a unique code. This makes it easy to find and ensures coherent conversations.   I tested Azure AI Search Retriever with Azure data storage and found it performs excellently. However  using Chroma with local data provides better protection for sensitive information  crucial for enhanced data privacy in business.   This database PostgreSQL contains detailed information about our credit card products. Whenever the chatbot answers a customers question  it also provides real-time promotions about credit cards with no annual fee from our database. Additionally  you can extend many functionalities by using database queries  such as recommending specific credit cards to customers based on the information they provide.   Amazon Comprehend Moderation for Data Privacy:   Privacy is the priority in business AI applications  especially with financial info. Amazon Comprehend Moderation scans and protects sensitive data like social security numbers and addresses. I tested it  and its great at keeping information safe. This ensures we comply with privacy laws  and make users feel secure sharing their information.   ChainLit and ConversationBufferWindowMemory for Conversational Management:   ChainLit and ConversationBufferWindowMemory act like the chatbots memory  helping it keep track of ongoing chats. This is very important for customer support since it often involves follow-up questions. These tools let the chatbot remember the context  making interactions more personal and coherent.   Docker for Deployment and Management:   Finally  we need a reliable way to deploy and manage our chatbot. Docker is chosen for its ability to deploy the chatbot in containers. It can isolate environments and can be easily scaled while maintaining security.   Imagine our chatbot can only initially handle 100 user requests per day. As our user base grows and we receive 1 000 requests  Docker may quickly scale container instances to meet this increased demand without altering the underlying code.   To put it all together  imagine a flowchart that starts with a customer query. This query gets processed by Azure OpenAI (the brain)  which then retrieves the needed information from our organized library (embedding models and data Chroma). Before responding  our security guard (Amazon Comprehend Moderation) checks for any sensitive data. The chatbot with its memory (ChainLit and ConversationBufferWindowMemory) delivers a coherent response. And overseeing everything is Docker  helping the system run smoothly and grow as needed.   Code Explanation for the AI ChatbotIll walk through the code that powers our AI chatbot. I use Chainlit for the user interface  LangChain for the conversational flow and memory  and Amazon Comprehend for ensuring data security.   Lets break down the code block by block to understand how each component works together.   1. Setting Up Environment VariablesFirst  we set up the necessary environment variables for Azure OpenAI and Amazon Comprehend. These keys and endpoints are essential for authenticating our API requests.   import os # Azure OpenAI credentials os.environ[AZURE_OPENAI_API_KEY] = your_azure_openai_api_key os.environ[AZURE_OPENAI_ENDPOINT] = https://your_openai_endpoint/ OPENAI_API_KEY = your_azure_openai_api_key OPENAI_DEPLOYMENT_NAME = gpt4 MODEL_NAME = gpt-4 OPENAI_API_VERSION = 2024-03-01-preview2. Initializing the Chatbot ModelThen initialize the Azure OpenAI model which will generate responses to user queries. This model uses the credentials set earlier.   from langchain_openai import AzureChatOpenAI # Set up the Azure Chat OpenAI model chat_model = AzureChatOpenAI( openai_api_version=OPENAI_API_VERSION  azure_deployment=OPENAI_DEPLOYMENT_NAME  temperature=0 )3. Embedding Model and RetrieverNext  I set up the embedding model and the retriever using LangChain and Chroma. This enables the chatbot to search through a vector database. In addition  I created a function to fetch promotional credit card products from the table credit_card_products in PostgreSQL.   from langchain_openai import AzureOpenAIEmbeddings from langchain_chroma import Chroma from sqlalchemy import create_engine # Postgresql Database connection conn_str = 'postgresql://#####' engine = create_engine(conn_str) # Obtain promotional credit card products from Postgresql def fetch_promotional_cards(): try: query = SELECT * FROM credit_card_products WHERE annual_fee < 1 df_promotional_cards = pd.read_sql(query  engine) return df_promotional_cards except Exception as e: print(fError fetching promotional cards: {e}) return pd.DataFrame() # Initialize the embedding model emb_model = AzureOpenAIEmbeddings( deployment='textembedding3large'  model='text-embedding-3-large'  openai_api_key=OPENAI_API_KEY  azure_endpoint=https://#####.com/  openai_api_type=azure ) # Define the function to load the retriever def get_retriever(): loaded_vectordb = Chroma(persist_directory=path_to_chroma_db  embedding_function=emb_model) retriever = loaded_vectordb.as_retriever() return retriever chat_retriever = get_retriever()4. Managing Conversation ContextWe use ConversationBufferWindowMemory from LangChain to maintain conversational context. It allows the chatbot to keep track of previous interactions.   from langchain.memory import ConversationBufferWindowMemory # Set up the conversation memory chat_memory = ConversationBufferWindowMemory( k=5  memory_key=chat_history  input_key=question  output_key='answer'  return_messages=True )5. Amazon Comprehend for Data SecurityAmazon Comprehend Moderation is configured to scan and protect sensitive data. This ensures that any personally identifiable information (PII) in user queries is handled securely.   import boto3 from langchain_experimental.comprehend_moderation import ( AmazonComprehendModerationChain  BaseModerationConfig  ModerationPiiConfig  ModerationPromptSafetyConfig  ModerationToxicityConfig ) # Set up the Amazon Comprehend Moderation os.environ[AWS_ACCESS_KEY_ID] = #### os.environ[AWS_SECRET_ACCESS_KEY] = ###### # Initialize the Amazon Comprehend client comprehend_client = boto3.client(comprehend  region_name=us-east-1) # Define moderation configurations pii_labels = [SSN  DRIVER_ID  ADDRESS  'EMAIL'  'PHONE'  'CA_SOCIAL_INSURANCE_NUMBER'] pii_config = ModerationPiiConfig(labels=pii_labels  redact=True  mask_character=X) toxicity_config = ModerationToxicityConfig(threshold=0.5) prompt_safety_config = ModerationPromptSafetyConfig(threshold=0.5) moderation_config = BaseModerationConfig( filters=[pii_config  toxicity_config  prompt_safety_config] ) comp_moderation_with_config = AmazonComprehendModerationChain( moderation_config=moderation_config  client=comprehend_client  verbose=True )6. Defining System and Human Message TemplatesThen define templates for system and human messages to guide the chatbots interactions. This enables it to follow a structured approach.   from langchain.prompts import ChatPromptTemplate  HumanMessagePromptTemplate  SystemMessagePromptTemplate # Define system and human message templates system_template =  You are a virtual assistant for the Help Desk. Only answer questions related to credit card business. Include URL in your reply  and return the full URL for your source document. Do not include any email. Use the answers from the retrieved document first. If you cannot find the answer from the pieces of context  just say that sorry you don't know nicely. Do not try to make up an answer. All the personally identifiable information will be redacted with X. Ignore the personally identifiable information and answer generally. --------------- {context}  human_template = Previous conversation: {chat_history} New human question: {question}  messages = [ SystemMessagePromptTemplate.from_template(system_template)  HumanMessagePromptTemplate.from_template(human_template) ] qa_prompt = ChatPromptTemplate.from_messages(messages)7. Setting Up the Conversational Retrieval ChainThis step sets up the conversational retrieval chain that ties everything together  using the model  retriever  memory  and moderation components.   from langchain.chains import ConversationalRetrievalChain # Initialize the conversational retrieval chain qa = ConversationalRetrievalChain.from_llm( llm=chat_model  chain_type='stuff'  retriever=chat_retriever  memory=chat_memory  return_source_documents=True  combine_docs_chain_kwargs={prompt: qa_prompt} )8. Chainlit Integration for Chatbot UIFinally  integrate Chainlit to handle user interactions. This provides a user-friendly interface.   import chainlit as cl @cl.on_chat_start async def on_chat_start(): msg = cl.Message(content=Hello  this is AI powered helpdesk  feel free to ask me any questions!) await msg.send() cl.user_session.set(chain  qa) @cl.on_message async def main(message: cl.Message): chain = cl.user_session.get(chain) cb = cl.AsyncLangchainCallbackHandler( stream_final_answer=True ) # Force final answer if necessary cb.answer_reached = True res = await chain.acall(message.content  callbacks=[cb]) answer = res[answer] source_documents = res[source_documents] # type: List[Document] text_elements = [] # type: List[cl.Text] if source_documents: for source_idx  source_doc in enumerate(source_documents): source_name = fsource_{source_idx} text_elements.append( cl.Text(content=source_doc.page_content  name=source_name  display=side) ) source_names = [text_el.name for text_el in text_elements] answer += f\n\nSources: {'  '.join(source_names)} else: answer += \n\nNo sources found # Fetch promotional credit card products df_promotional_cards = fetch_promotional_cards() promotional_cards_info = df_promotional_cards.to_dict(orient='records') # Append promotional cards information to the answer if promotional_cards_info: answer += \n\nPromotional Credit Cards with No Annual Fee:\n for card in promotional_cards_info: answer += f- {card['card_name']} (Credit Limit: {card['credit_limit']}  Cashback: {card['cashback']}%  Sign-Up Bonus: {card['sign_up_bonus']} points)\n await cl.Message(content=answer  elements=text_elements).send()By combining Chainlit for the UI  LangChain for conversation management  a PostgreSQL database for detailed credit card information  and Amazon Comprehend for data security  we can create a professional and robust chatbot solution.   Deploying the AI Chatbot Using DockerTo make our AI chatbot professional and production-ready  we need to deploy it using Docker. Docker allows us to package our application with all its dependencies into a container that can run consistently on any system. Heres a detailed guide on how to build and deploy our chatbot using Docker.   Dockerfile and RequirementsFirst  lets look at the Dockerfile and the requirements.txt file  which are essential for building our Docker image.   Dockerfile:   # Stage 1 - Install build dependencies FROM python:3.11-slim AS builder WORKDIR /app ENV PYTHONDONTWRITEBYTECODE=1 ENV PYTHONUNBUFFERED=1 RUN apt-get update && apt-get install -y \ build-essential \ curl \ software-properties-common \ git \ libpq-dev \ && rm -rf /var/lib/apt/lists/* RUN python -m venv /opt/venv ENV PATH=/opt/venv/bin:$PATH COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt # Stage 2 - Copy only necessary files to the runner stage FROM python:3.11-slim ARG FILENAME ARG PORT=8000 ENV FILENAME=${FILENAME} ENV PORT=${PORT} WORKDIR /app COPY --from=builder /opt/venv /opt/venv ENV PATH=/opt/venv/bin:$PATH COPY $FILENAME . COPY chainlit.json . COPY static ./static EXPOSE ${PORT} CMD [sh  -c  python -m chainlit run ${FILENAME} --port=${PORT} -w]requirements.txt:   chainlit langchain boto3 langchain_experimental langchain_openai langchain_community langchain_chroma flask psycopg2 SQLAlchemyBasics of Dockerfile and requirements.txtStage 1  Build Dependencies:   Base Image: We use the python:3.11-slim image as the base.Working Directory: Sets the working directory to /app .Environment Variables: Disables bytecode generation and ensures unbuffered output.System Dependencies: Installs build-essential tools  curl  and git.Virtual Environment: Creates a virtual environment and updates the PATH.Dependencies Installation: Copies requirements.txt and installs the Python dependencies.Stage 2  Runner Stage:   Base Image: Uses the same python:3.11-slim base image.Arguments and Environment Variables: Sets up arguments and environment variables for the filename and port.Working Directory: Sets the working directory to /app .Copy Files: Copies the virtual environment  application files  and necessary directories.Expose Port: Exposes the specified port.Command: Runs the Chainlit application using the specified filename and port.requirements.txt:   Lists all the necessary Python packages for the chatbot  ensuring they are installed in the Docker image.Steps to Build and Deploy the Docker ContainerCreate Dockerfile and requirements.txt:   Ensure both files are in your project root directory.Build the Docker Image:   Use the Docker build command to create the Docker image.docker build -t ai-chatbot --build-arg FILENAME=your_chatbot_script.py .Run the Docker Container:   Use the Docker run command to start a container from the image.docker run -d -p 8000:8000 --name ai-chatbot-container ai-chatbotAccess the Chatbot:   Once the container is running  you can access the chatbot by navigating to http://localhost:8000 in your web browser.Deploying the AI chatbot using Docker is necessary for ensuring it is production-ready  scalable  and consistent across different environments.   Testing Azure Cognitive Search for Enhanced PerformanceIn this project  I also tested Azure Cognitive Search as an alternative to the embedding-based Chroma search. Azure Cognitive Search is a cloud search service that provides indexing and querying capabilities.   Azure Cognitive Search Code:   from langchain_community.vectorstores.azuresearch import AzureSearch # Set up the Azure AI Search Retriever if using AZURE_COGNITIVE_SEARCH_SERVICE os.environ[AZURE_COGNITIVE_SEARCH_SERVICE_NAME] = your_search_service_name os.environ[AZURE_AI_SEARCH_SERVICE_NAME] = your_ai_search_service_name os.environ[AZURE_COGNITIVE_SEARCH_INDEX_NAME] = your_index_name os.environ[AZURE_AI_SEARCH_INDEX_NAME] = your_index_name os.environ[AZURE_COGNITIVE_SEARCH_API_KEY] = your_search_api_key os.environ[AZURE_AI_SEARCH_API_KEY] = your_ai_search_api_key # Initialize the Azure Cognitive Search retriever search_retriever = AzureSearch( service_name=os.getenv(AZURE_COGNITIVE_SEARCH_SERVICE_NAME)  index_name=os.getenv(AZURE_COGNITIVE_SEARCH_INDEX_NAME)  api_key=os.getenv(AZURE_COGNITIVE_SEARCH_API_KEY) )Advantages:   Performance: Azure Cognitive Search shows better performance in speed and accuracy when retrieving relevant documents compared to the embedding-based Chroma search.Scalability: Azure Cognitive Search can handle large-scale search queries efficiently. This makes it better for enterprise applications.Integration: Azure Cognitive Search integrates better with other Azure services. This provides a cohesive environment for querying data.Drawbacks:   Cost: Azure Cognitive Search can be more expensive compared to using local embedding-based solutions like Chroma.Complexity: Setting up and configuring Azure Cognitive Search can be more complex  requiring additional knowledge and management.Data Storage:   Azure: Storing RAG (Retrieval-Augmented Generation) data in Azure provides benefits such as high availability  redundancy  and security. However  it may also incur higher costs and dependency on cloud infrastructure.Chroma: Using Chroma with local storage can be more cost-effective and allows greater control over data. However  it may not scale as efficiently as Azure Cognitive Search.As usual  you can find the relevant code in the following GitHub repository: https://github.com/datalev001/chatbot_chainlit   Final ThoughtsUsing Docker to deploy our AI chatbot with tools like Chainlit  LangChain  and Amazon Comprehend makes it professional  scalable  and secure. This setup handles complex interactions  keeps track of conversations  and protects sensitive info.   Adding a PostgreSQL database lets the chatbot give personalized responses and real-time promotions  making it a great business tool. It uses specific data to provide tailored support and recommendations.   Testing Azure Cognitive Search showed its great for big queries  though its more complex and costly than local solutions like Chroma.   By setting up the environment  integrating advanced models  and deploying with Docker  we can create a solid AI helpdesk system.