Building a Question and Answer Chatbot with Amazon Kendra and AWS Fargate | by Yi Ai | Towards Data Science

Amazon announced the general availability of Amazon Kendra a few weeks ago, Kendra is a highly accurate and easy to use enterprise search service powered by machine learning.
In this post I will build a question and answer chatbot solution using React with Amplify, WebSocket API in AWS API Gateway, AWS Fargate and Amazon Kendra.
The solution provides a conversational interface for Questions and Answers. This chatbot allows users to ask their questions and get relevant answers quickly.
Create an Amazon Kendra index, extract questions and answers from semi-structured document to Kendra FAQ.
Deploy a WebSocket API in API Gateway to process questions and answer messages.
Create a React application and use AWS Amplify to connect & interact with this chatbot through WebSocket.
Create a service in AWS Fargate that let our bot call Kendra’s API to provide answer and send it back to user.
The following diagram shows the architecture of the above steps:
An easier approach to building a chatbot would be to use a lambda function to query Amazon Kendra without using Fargate. However, with AWS Fargate or EC2, we are able to extend the chatbot with custom AI models to make our bot more human, eg. We can build a chatbot based on the Hugging Face State-of-the-Art Conversational AI model and query Kendra only for Specific data.
If your program is a long compute job which requires more GBs of memory and higher performance, Fargate is probably the better option.
Setup an AWS account
Install latest aws-cli
Install Amplify cli
Basic understanding of React
Basic understanding of Docker
Basic understanding of CloudFormation
Install or update the Serverless Framework to latest version
jq installed (optional)
Let’s create a Kendra index. Amazon Kendra is a highly accurate and easy to use enterprise search service that’s powered by machine learning. Kendra supports unstructured and semi-structured documents like FAQs stored in S3, we will use FAQS in our case.
First, let’s download a QnA dataset and upload it to S3. We can use Microsoft Research WikiQA Corpus for our chatbot.
Once downloading the dataset, let’s transform to Kendra supported csv format like below:
Use following script to transform the dataset and upload the transformed csv file to existing S3 bucket my-kendra-index:
!wget https://download.microsoft.com/download/A/A/9/AA9C9802-B448-4361-89E0-DBF24D5A5C2C/WikiQACodePackage.zip
!unzip WikiQACodePackage.zip

import json
import boto3
import pandas as pd
import os

def create_faq_format(input_path):
    faq_list = []
    
    with open(input_path) as f:
        lines = [line.strip('\n') for line in f]
    for i in range(2, len(lines)):
        l = lines[i].split('\t')
        if l[2]=="1":
            faq_list.append({"Question":l[0],"Answer":l[1]})
                
    return faq_list
    
qa_list = create_faq_format("WikiQACodePackage/data/wiki/WikiQASent-train.txt")
df = pd.DataFrame(qa_list, columns=["Question","Answer"])
df = df.drop_duplicates(subset='Question', keep="first")
df.to_csv('faq.csv', index=False)

s3 = boto3.resource('s3')
s3.meta.client.upload_file("faq.csv", 'my-kendra-index', 'faq/faq.csv')

Now, we are now ready to create a Kendra index. To create a Kendra index, complete the following steps:
On the Amazon Kendra console, choose Launch Amazon Kendra.Create index and enter an Index name, such as my-aq-index.For IAM role, choose Create a new role to create a role to allow Amazon Kendra to access CloudWatch Logs.Create Index.
On the Amazon Kendra console, choose Launch Amazon Kendra.
Create index and enter an Index name, such as my-aq-index.
For IAM role, choose Create a new role to create a role to allow Amazon Kendra to access CloudWatch Logs.
Create Index.
After the Kendra index has been created, we can add our FAQ document:
Add FAQ from Amazon Kendra console.For S3, browse S3 to find your bucket, and select the FAQ csv file. Here we use s3://my-kendra-index/faq/faq.csv.For IAM role, select Create a new role to allow Amazon Kendra to access FAQ content object in S3 bucket.Add FAQ.
Add FAQ from Amazon Kendra console.
For S3, browse S3 to find your bucket, and select the FAQ csv file. Here we use s3://my-kendra-index/faq/faq.csv.
For IAM role, select Create a new role to allow Amazon Kendra to access FAQ content object in S3 bucket.
Add FAQ.
Now that we have a working Kendra index, let’s move to the next step.
In this section we will build a 1) WebSockets API in AWS API Gateway, 2) create lambda functions to manage WebSockets routes ($connect, $disconnect, sendMessage) and 3) create DynamoDb to store WebSockets connection Ids and user names.
We will use Serverless Framework to build and deploy all required resources. let’s create a new Serverless project and add the following config to serverless.yml :
Note that the Cognito App Client Id (/chatbot/dev/app_client_id) and Cognito User Pool Id (/chatbot/dev/user_pool_id) in serverless.yml has not been created yet, We only reference Cognito details as SSM Parameters here, in the next step, we will create Cognito User Pool using Amplify Cli and then we can modify related SSM parameters from theSystem Storage Manager console.
Once serviceless.yml has been modified, update handler.js to create the lambda functions for WebSockets routes: $connect with custom authorizer, $disconnect, sendMessage:
Run the following commands to deploy the WebSocket API:
$sls deploy --stage dev --region YOUR_REGION
In this section, We will build a web app using React and AWS Amplify with an authentication feature.
The complete project is in my Github repo, you can find the following folders in the project directory:
amplify/.config/, and amplify/backend/.project-config.json in .config/ folder.backend-config.json in backend/ folder.CloudFormation files in the backend/ folder.
amplify/.config/, and amplify/backend/.
project-config.json in .config/ folder.
backend-config.json in backend/ folder.
CloudFormation files in the backend/ folder.
Let’s download the source code and re-initialise the existing Amplify project by running:
$amplify init
then push changes:
$amplify push 
and deploy:
$amplify publish
We will get the web application URL after the project has been deployed:
Now, log in to the AWS Cognito service console and you can now see AWS Cognito User Pool has been created . Copy the User Pool Id and App Client Id and use them as SSM Parameters we already created in previous step.
In this section, we will create a bot task to run our chatbot service in AWS Fargate. First, a chatbot task connects to a websocket API, Then, when the user asks a question, the bot can query the Kendra index, and Kendra will surface a relevant answer, and send it back to the user who asked the questions.
To deploy the Fargate service, perform the following steps:
Download chatbot script and Dockerfile here.Build Docker, tag an Amazon ECR Repository and push the image to ECR. For more details, please refer to AWS official tutorial.Download the CloudFormation templates and bash scripts here.If using the Fargate launch type, the awsvpc network mode is required. We need to deploy VPC and Security Groups:
Download chatbot script and Dockerfile here.
Build Docker, tag an Amazon ECR Repository and push the image to ECR. For more details, please refer to AWS official tutorial.
Download the CloudFormation templates and bash scripts here.
If using the Fargate launch type, the awsvpc network mode is required. We need to deploy VPC and Security Groups:
$bash create-infra.sh -d dev
5. Create task definition.
$bash create-task.sh -d dev
6. Deploy chatbot service in AWS Fargate.
$bash deploy-service.sh -d dev
The main logic can be found here:
To extend the chatbot with ConvAI model, you can try below sample script, note that you will need to put more effort to train the model and put it in the docker or Amazon EFS.
Once the service has been deployed, we should be able to ask the bot questions. Let’s visit the React application and try it out live!
As evident above, using keywords alone is sufficient for the system to respond with the correct answer.
If you would like to learn more about Amazon Kendra, there is an official tutorial on how to build a chatbot using Lex and Kendra here.
I hope you have found this article useful, The source code for this post can be found in my GitHub repo.