How to Easily Build a PDF Chatbot with RAG (Retrieval-Augmented Generation) Using Azure AI Studio's Prompt Flow
Tags: azure, openai, rag
Published: 2024-01-25 00:38:20

Developing a chatbot that can answer questions about PDF documents might seem like a task requiring extensive time and effort. However, with Azure AI Studio's Prompt Flow, the implementation becomes surprisingly straightforward. In this blog, we will delve into the process of creating a chatbot with RAG (Retrieval-Augmented Generation) that can respond to queries related to sample PDF data. We’ll also guide you through testing this chatbot using the chat feature in Azure AI Studio.

---
This post is a part of a series that serves as a step-by-step guide to developing a chatbot with RAG:

- **Step 1** : [How to Easily Build a PDF Chatbot with RAG (Retrieval-Augmented Generation) Using Azure AI Studio's Prompt Flow](https://dev.to/aykhara/how-to-easily-build-a-pdf-chatbot-with-rag-retrieval-augmented-generation-using-azure-ai-studios-prompt-flow-boc) **← YOU ARE HERE!**
- **Step 2** : [How to Evaluate a PDF Chatbot Response with Prompt Flow](https://dev.to/aykhara/how-to-evaluate-a-pdf-chatbot-response-with-prompt-flow-c56)
- **Step 3** : [How to Deploy a PDF Chatbot as a REST Endpoint and Test with Postman](https://dev.to/aykhara/how-to-deploy-a-pdf-chatbot-as-a-rest-endpoint-and-test-with-postman-4jp3)

---

### Table of Contents
- [Prerequisites](#prerequisites)
- [1. Create a project on Azure AI Studio](#1-create-a-project-on-azure-ai-studio)
  - [1-1. Build your own copilot](#11-build-your-own-copilot)
  - [1-2. Configure project details](#12-configure-project-details)
  - [1-3. Create an Azure AI resource for your projects](#13-create-an-azure-ai-resource-for-your-projects)
  - [1-4. Review and create a project](#14-review-and-create-a-project)
- [2. Deploy Azure OpenAI models](#2-deploy-azure-openai-models)
  - [2-1. Create a new deployment](#21-create-a-new-deployment)
  - [2-2. Select a model](#22-select-a-model)
  - [2-3. Deploy model](#23-deploy-model)
- [3. Create an index on Azure AI Search](#3-create-an-index-on-azure-ai-search)
  - [3-1. Select your dataset](#31-select-your-dataset)
  - [3-2. Configure index storage](#32-configure-index-storage)
  - [3-3. Configure search settings](#33-configure-search-settings)
  - [3-4. Configure index settings](#34-configure-index-settings)
  - [3-5. Review and create an index](#35-review-and-create-an-index)
- [4. Configure Prompt Flow](#4-configure-prompt-flow)
  - [4-1. Create runtime](#41-create-runtime)
  - [4-2. Configure parameters in each node](#42-configure-parameters-in-each-node)
  - [4-3. Save all configuration changes](##43-save-all-configuration-changes)
- [5. Try Chat!](#try-chat)
- [Conclusion](#conclusion)

## Prerequisites
- Prepare PDF data
As a demo I'm going to use sample data from [azure-search-openai-demo](https://github.com/Azure-Samples/azure-search-openai-demo/tree/main/data).
- Create Azure AI Search

## 1. Create a project on Azure AI Studio
### 1-1. Build your own copilot
Access to [ai.azure.com](ai.azure.com) and make sure that the right directory (subscription) is selected.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2c6j0xq3m9sawov79qla.png)

Select "Build your own copilot".
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g57n9vw4rnjqy6yayq8j.png)

Then, click "Create a new project".
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mtbwhdbhbtp6f7621443.png)

### 1-2. Configure project details
Input project name and select "Create a new resource", then click next.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lh9fp5l8rgd0ga7oqp3a.png)
### 1-3. Create an Azure AI resource for your projects
Input resource name(Your Azure AI resource name must be different from your project name) and select an appropriate Azure location, then next.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4yf929uvefxf67srflci.png)
### 1-4. Review and create a project
Click "Create a project" in Review and finish pane
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4jww37zmt0dhrikidh5b.png)

It will take around 2 minutes to deploy all required resources to Azure.

## 2. Deploy Azure OpenAI models
Once deploying all resources to Azure, you will automatically navigate to Playground in Azure AI Studio.
At the same time, you will find the information message "No deployment exists: You need a deployment to work in the playground. Navigate to the Deployment page to create a deployment." Please click the link "Deployment page".
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/01yk5du95lx3eghsw5ck.png)

You will need at least two models including one for embedding.
### 2-1. Create a new deployment
Move to "Deployment" pane and click "Create".
### 2-2. Select a model
Select a model:text-embedding-ada-002, then Confirm
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oaxcxuygoxbvmo1397qw.png)
### 2-3. Deploy model
Finally deploy a selected model.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b80o8dptxm9tkuom4dfm.png)

Repeat 2-1 to 2-3 above for other model (e.g. gpt-35-turbo) as well.

Once done with deploying models, they are listed in the Deployment pane.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wzvbwt62xlo72qkhus4b.png)

## 3. Create an index on Azure AI Search
### 3-1. Select your dataset
Select "Upload files/folders" as Data source,
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pazmmozgfkf4srmr9vbt.png)
and then click "Upload" > "Upload files"
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lrgp0yb1z6ak3lgetooc.png)

### 3-2. Configure index storage
Select the following settings:
- "Connect other Azure AI Search resource"
- Azure subscription which you deployed the Azure AI Search on
- Azure AI Search service which you deployed already.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fgnfwlfgq0pa1udwd6yj.png) 

### 3-3. Configure search settings
Confirm the acknowledgement
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/30y9hum7moe0w4ljjaam.png)

### 3-4. Configure index settings
Insert index name and select virtual machine* (e.g. auto select) 
*Selected virtual machine will be used to run indexing jobs.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/svn01ien74ynlyjszfzk.png)

### 3-5. Review and create an index
Finally click "Create"
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w6reux87275kd1ed7bnl.png)

It will take around 10 minutes to get done with all jobs.
If you want to know what is happening behind, "job details" navigates you to Azure ML Studio for more details.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ptq1fh8qw5h3t6u3qlo6.png)
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sokz5qwungsy99luh2wy.png)

Once all jobs are completed you can find the index with "ready" sign.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yztzltuacfosh7us920b.png)

## 4. Configure Prompt Flow
If you move to Prompt flow pane you can find "(your-index-name)-sample-flow" in the flow list.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a04d8z4kcagcp32puqqf.png)

Once you select the flow you may notice that the basic flow is already prepared. However, there are still some manual configuration required.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/od5j71cdv65d0cgka86x.png)
### 4-1. Create runtime
Create runtime by simply selecting "automatic runtime start".
It will take 5 minutes or so.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/71e9aru1x6ewwftfp8yo.png)

### 4-2. Configure parameters in each node
- **modify_query_with_history** : Select deployment name and set max token (e.g. 1000), then click "Validate and purse input"
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0i5a27llhk8j4h6pqew5.png)

- **embed_the_question** : Click "Validate and purse input"

>Note : Perhaps a deployment name will be removed due to some reasons after clicking validate button. Please make sure that the right deployment name for embedding is set.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ne03d3ixgba7887udxma.png)

- **search_question_from_indexed_docs** : Click "Validate and purse input"
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a75mh7vddr20jyx532ab.png)

- **generate_prompt_context** : Click "Validate and purse input"
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/55293mepvmj2i9ns96k7.png)

- **Prompt_variants** : : Click "Validate and purse input"
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/liz3cv92spr0i9l98gd0.png)

- **answer_the_question_with_context** : Select deployment name and set max token (e.g. 1000), then click "Validate and purse input"
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9zgoktaj712aoatu5mvk.png)

### 4-3. Save all configuration changes
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p2z1hjz69trqitpu0s85.png)

## 5. Try Chat!
Finally it's time to chat with your PDFs.
Click "Chat" button,
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vs4a7d0hwpvy2sa2nyfs.png)
and then put your question in the chat!
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hdjtrkl7i57k0ml2ejtm.png)

The answer will be returned in the same language with your input.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vx0bwddaplffy7gndhd3.png)

## Conclusion
In this blog, we have introduced how to implement a chatbot with RAG (Retrieval-Augmented Generation) that answers questions about sample PDF data using Azure AI Studio's Prompt Flow, and how to practically test it within Azure AI Studio's chat feature. Next in our series, ["How to Evaluate a PDF Chatbot Response with Prompt Flow"](https://dev.to/aykhara/how-to-evaluate-a-pdf-chatbot-response-with-prompt-flow-c56), we will delve into methods for evaluating the performance of the chatbot we've just implemented.