Build a SMS Bot that Answers questions over Docs with LangChain in Python
Tags: machinelearning, langchain, ai, nlp
Published: 2023-06-15 20:33:43

---
title: Build a SMS Bot that Answers questions over Docs with LangChain in Python
published: true
description: 
tags: machinelearning, langchain, ai, nlp
cover_image: https://assets.cdn.prod.twilio.com/images/Copy_of_C01_Blog_Text_2_eiYLn4j.width-1616.format-webp.webp
# Use a ratio of 100:42 for best results.
# published_at: 2023-06-15 19:45 +0000
---
This blog post was written for [Twilio](https://www.twilio.com/) and [originally published on the Twilio blog](https://www.twilio.com/blog/qa-over-docs-bot-langchain-python).

With [Natural Language Processing (NLP)](https://www.twilio.com/docs/glossary/what-natural-language-processing-nlp), you can chat with your own documents, such as a text file, a PDF, or a website. Read on to learn how to build a generative question-answering SMS chatbot that reads a document containing [Lou Gehrig's Farewell Speech](https://www.si.com/mlb/2009/07/05/gehrig-text) using [LangChain](https://python.langchain.com/en/latest/index.html), [Hugging Face](https://huggingface.co/), and Twilio in Python.
![sms example where I text what would Lou Gehrig give to beat the New York Giants and the bot replies with he would give his right arm to beat them](https://assets.cdn.prod.twilio.com/images/Screenshot_2023-06-15_at_11.44.44_AM.format-webp.webp)
## LangChain Q&A
LangChain is an open-source tool that wraps around many large language models (LLMs) and tools. It is the easiest way (if not one of the easiest ways) to interact with LLMs and build applications around LLMs.

LangChain makes it easy to perform question-answering of those documents. Picture feeding a PDF or maybe multiple PDF files to a machine and then asking it questions about those files. This could be useful, for example, if you have to prepare for a test and wish to ask the machine about things you didn’t understand.
## Prerequisites
1. A Twilio account - [sign up for a free Twilio account here](https://www.twilio.com/try-twilio)
2. A Twilio phone number with SMS capabilities - [learn how to buy a Twilio Phone Number here](https://support.twilio.com/hc/en-us/articles/223135247-How-to-Search-for-and-Buy-a-Twilio-Phone-Number-from-Console)
3. Hugging Face Account – [make a Hugging Face Account here](https://huggingface.co/join)
4. Python installed - [download Python here](https://www.python.org/downloads/)
5. [ngrok](https://ngrok.com/download), a handy utility to connect the development version of our Python application running on your machine to a public URL that Twilio can access.
> :warning: **ngrok is needed for the development version of the application because your computer is likely behind a router or firewall, so it isn’t directly reachable on the Internet. You can also [choose to automate ngrok as shown in this article](https://www.twilio.com/blog/automating-ngrok-python-twilio-applications-pyngrok).**: 
## Configuration
Since you will be installing some Python packages for this project, you will need to make a new project directory and a virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:
```bash
mkdir lc-qa-sms 
cd lc-qa-sms 
python3 -m venv venv 
source venv/bin/activate 
!pip install langchain
!pip install requests
pip install flask
pip install faiss-cpu
pip install sentence-transformers
pip install twilio
pip install load_dotenv
```
If you're following this tutorial on Windows, enter the following commands in a command prompt window:
```bash
mkdir lc-qa-sms  
cd lc-qa-sms 
python -m venv venv 
venv\Scripts\activate 
pip install langchain
pip install requests
pip install flask
pip install faiss
pip install sentence-transformers
pip install twilio
pip install load_dotenv
```
## Set up Hugging Face Hub
The Hugging Face Hub offers over 120k models, 20k datasets, and 50k demos people can easily collaborate in their ML workflows. As mentioned earlier, this project needs a Hugging Face Hub Access Token to use the LangChain endpoints to a Hugging Face Hub LLM. After making a Hugging Face account, you can [get a Hugging Face Access Token here](https://huggingface.co/settings/tokens) by clicking on <strong>New token</strong>. Give the token a name and select the <strong>Read</strong> Role.
![hugging face hub with access tokens page](https://assets.cdn.prod.twilio.com/images/huggingfacehub.format-webp.webp)
Alternatively, you could use models from, say, OpenAI.

On the command line in your root directory, run the following command on a Mac to set the token as an environment variable.
```bash
export HUGGINGFACEHUB_API_TOKEN=replace-with-your-huggingfacehub-token
```
For the Windows command, [check out this blog post on environment variables](https://www.twilio.com/blog/how-to-set-environment-variables-html).

Now let's build that LangChain question-answering chatbot application.
## Answer Questions from a Doc with LangChain via SMS
Inside your `lc-qa-sms` directory, make a new file called <em>app.py</em>.

At the top of the file, add the following lines to import the required libraries.
```python
import requests
from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains.question_answering import load_qa_chain
from langchain import HuggingFaceHub
from flask import Flask, request, redirect
from twilio.twiml.messaging_response import MessagingResponse
```
Beneath those import statements, make a helper function to write to a local file from a URL, and then load that file from the local file storage with LangChain's `TextLoader` library. Later the file will be passed over to the `split` method to create chunks.
```python
def loadFileFromURL(text_file_url):
    output_file = "lougehrig.txt"
    resp = requests.get(text_file_url)
    with open(output_file, "w",  encoding='utf-8') as file:
      file.write(resp.text)

    # load text doc from URL w/ TextLoader
    loader = TextLoader('./'+output_file)
    txt_file_as_loaded_docs = loader.load()
    return txt_file_as_loaded_docs
```
In this tutorial, the file will be Lou Gehrig's famous speech, which is why the `output_file` is named "lougehrig.txt".
![lou gehrig farewell speech on field image](https://assets.cdn.prod.twilio.com/images/lougehrigfarewell.format-webp.webp)
You could alternatively use a local file.

Next, add the following code for a helper function to split the document into chunks. This is important because LLMs can't process inputs that are too long. LangChain's `CharacterTextSplitter` function helps us do this–setting `chunk_size` to 1000 and `chunk_overlap` to 10 keeps the integrity of the file by avoiding splitting words in half.
```python
def splitDoc(loaded_docs):
    # split docs into chunks
    splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=10)
    chunked_docs = splitter.split_documents(loaded_docs)
    return chunked_docs
```
Now convert the chunked document into embeddings (numerical representations of words) with Hugging Face and store them in a FAISS Vector Store. [Faiss](https://faiss.ai/) is a library for efficient similarity search and clustering of dense vectors.
```python
def makeEmbeddings(chunked_docs):
    # Create embeddings and store them in a FAISS vector store
    embedder = HuggingFaceEmbeddings()
    vector_store = FAISS.from_documents(chunked_docs, embedder)
    return vector_store
```
That `makeEmbeddings` function will make it more efficient to retrieve and manipulate the stored embeddings to conduct a similarity search to get the most semantically-similar documents to a given input which the LLM needs to best answer questions in the following helper function.
```python
def askQs(vector_store, chain, q):
    # Ask a question using the QA chain
    similar_docs = vector_store.similarity_search(q)
    resp = chain.run(input_documents=similar_docs, question=q)
    return resp
```
The final helper function defines and loads the Hugging Face Hub LLM to be used with your Access Token and starts the request on a similarity search embedded with some input question to the selected LLM, enabling a question-and-answer conversation.
```python
def loadLLM():
    llm=HuggingFaceHub(repo_id="declare-lab/flan-alpaca-large", model_kwargs={"temperature":0, "max_length":512})
    chain = load_qa_chain(llm, chain_type="stuff")
    return chain
```
declare-lab/flan-alpaca-large is a LLM. You could use others from Hugging Face Hub such as google-flan-t5-xl. They are trained on different corpuses and this tutorial uses flan-alpaca-large because it's faster.

Lastly, make a Flask app using the Twilio REST API to call the helper functions and respond to inbound text messages with information pulled from the text file.
```python
app = Flask(__name__)
@app.route('/sms', methods=['POST'])
def sms():
    resp = MessagingResponse()
    inb_msg = request.form['Body'].lower().strip() #get inbound text body
    chain = loadLLM()
    LOCAL_ldocs = loadFileFromURL('https://raw.githubusercontent.com/elizabethsiegle/qanda-langchain-sms-lougehrig/main/lougehrig.txt')
    LOCAL_cdocs = splitDoc(LOCAL_ldocs) #chunked
    LOCAL_vector_store = makeEmbeddings(LOCAL_cdocs)
    LOCAL_resp = askQs(LOCAL_vector_store, chain, inb_msg)
    resp.message(LOCAL_resp)
    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)
```
Your complete `app.py` file should look like this:
```python
import requests
from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains.question_answering import load_qa_chain
from langchain import HuggingFaceHub
from flask import Flask, request, redirect
from twilio.twiml.messaging_response import MessagingResponse

def loadFileFromURL(text_file_url): #param: https://raw.githubusercontent.com/elizabethsiegle/qanda-langchain-sms-lougehrig/main/lougehrig.txt
    output_file = "lougehrig.txt"
    resp = requests.get(text_file_url)
    with open(output_file, "w",  encoding='utf-8') as file:
      file.write(resp.text)

    # load text doc from URL w/ TextLoader
    loader = TextLoader('./'+output_file)
    txt_file_as_loaded_docs = loader.load()
    return txt_file_as_loaded_docs

def splitDoc(loaded_docs):
    # split docs into chunks
    splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=10)
    chunked_docs = splitter.split_documents(loaded_docs)
    return chunked_docs

def makeEmbeddings(chunked_docs):
    # Create embeddings and store them in a FAISS vector store
    embedder = HuggingFaceEmbeddings()
    vector_store = FAISS.from_documents(chunked_docs, embedder)
    return vector_store

def askQs(vector_store, chain, q):
    # Ask a question using the QA chain
    similar_docs = vector_store.similarity_search(q)
    resp = chain.run(input_documents=similar_docs, question=q)
    return resp

def loadLLM():
    llm=HuggingFaceHub(repo_id="declare-lab/flan-alpaca-large", model_kwargs={"temperature":0, "max_length":512})
    chain = load_qa_chain(llm, chain_type="stuff")
    return chain


app = Flask(__name__)
@app.route('/sms', methods=['GET', 'POST'])
def sms():
    resp = MessagingResponse()
    inb_msg = request.form['Body'].lower().strip() #get inbound text body
    chain = loadLLM()
    LOCAL_ldocs = loadFileFromURL('https://raw.githubusercontent.com/elizabethsiegle/qanda-langchain-sms-lougehrig/main/lougehrig.txt')
    LOCAL_cdocs = splitDoc(LOCAL_ldocs) #chunked
    LOCAL_vector_store = makeEmbeddings(LOCAL_cdocs)
    LOCAL_resp = askQs(LOCAL_vector_store, chain, inb_msg)
    resp.message(LOCAL_resp)
    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)
```
On the command line, run `python app.py` to start the Flask app.
## Configure a Twilio Number for the SMS Chatbot
Now, your Flask app will need to be visible from the web so Twilio can send requests to it. ngrok lets you do this. With ngrok installed, run `ngrok http 5000` in a new terminal tab in the directory your code is in.
![ngrok http 5000 terminal with forwarding URLs opening up your local server to the web](https://assets.cdn.prod.twilio.com/images/ngrok_zGN7AVJ.format-webp.webp)
You should see the screen above. Grab that ngrok <strong>Forwarding URL</strong> to configure your Twilio number: select your Twilio number under <strong>Active Numbers </strong>in your Twilio console, scroll to the Messaging section, and then modify the phone number’s routing by pasting the ngrok URL with the <em>/sms</em> path in the textbox corresponding to when <strong>A Message Comes In</strong> as shown below:
![configure twilio phone number webpage with ngrok url when a message comes in](https://assets.cdn.prod.twilio.com/images/twiliosmsconfigphone.format-webp.webp)
Click <strong>Save</strong> and now your Twilio phone number is configured so that it maps to your web application server running locally on your computer and your application can run. Text your Twilio number a question relating to the text file and get an answer from that file over SMS!
![sms example where i text what kind of break was lou given and the bot/llm responds lou was given a bad break](https://assets.cdn.prod.twilio.com/images/Screenshot_2023-06-15_at_11.49.59_AM.format-webp.webp)
The [complete code can be found here on GitHub](https://github.com/elizabethsiegle/qanda-langchain-sms-lougehrig).
## What's Next for Twilio and LangChain
![langchain logo](https://assets.cdn.prod.twilio.com/images/langchainlogo.format-webp.webp)
There's so much you can do with document-based question-answering. You can use a different LLM, use a longer document than a text file containing Lou Gehrig's famous speech, use other types of documents like a PDF or website ([here's LangChain's docs on documents](https://python.langchain.com/en/latest/modules/indexes/document_loaders.html)), store the embeddings elsewhere, and more. Other than a SMS chatbot, you could create an AI tutor, search engine, automated customer service agent, and more.

Let me know online what you're building!
- Twitter: [@lizziepika](http://twitter.com/lizziepika)
- GitHub: [elizabethsiegle](http://github.com/elizabethsiegle)
- Email: lsiegle@twilio.com