How to Deploy a PDF Chatbot as a REST Endpoint and Test with Postman
Tags: azure, openai, rag
Published: 2024-01-25 00:39:11

Utilizing Azure AI Studio's Prompt Flow, we were able to easily implement a chatbot that answers questions about PDF documents. Furthermore, we conducted an evaluation of this chatbot to verify its accuracy. In this blog post, we will take you through the next step: moving beyond just testing the chatbot within Azure AI Studio's chat feature. We will demonstrate how to deploy this chatbot as a REST endpoint and test it using Postman, a popular tool for API development and testing by developers and test engineers.

---
This post is a part of a series that serves as a step-by-step guide to developing a chatbot with RAG:

- **Step 1** : [How to Easily Build a PDF Chatbot with RAG (Retrieval-Augmented Generation) Using Azure AI Studio's Prompt Flow](https://dev.to/aykhara/how-to-easily-build-a-pdf-chatbot-with-rag-retrieval-augmented-generation-using-azure-ai-studios-prompt-flow-boc) 
- **Step 2** : [How to Evaluate a PDF Chatbot Response with Prompt Flow](https://dev.to/aykhara/how-to-evaluate-a-pdf-chatbot-response-with-prompt-flow-c56)
- **Step 3** : [How to Deploy a PDF Chatbot as a REST Endpoint and Test with Postman](https://dev.to/aykhara/how-to-deploy-a-pdf-chatbot-as-a-rest-endpoint-and-test-with-postman-4jp3) **← YOU ARE HERE!**

---

### Table of Contents
- [Prerequisites](#prerequisites)
- [1. Modify a flow](#1-modify-a-flow)
  - [1-1. Replace with vector db lookup](#11-replace-with-vector-db-lookup)
  - [1-2. Delete an unnecessary node](#12-delete-an-unnecessary-node)
  - [1-3. Change the value of search result node](#13-change-the-value-of-search-result-node)
  - [1-4. Try chat](#14-try-chat)
- [2. Deploy a flow](#2-deploy-a-flow)
  - [2-1. Deploy a flow from Prompt Flow tab](#21-deploy-a-flow-from-prompt-flow-tab)
  - [2-2. Test the deployed endpoint](#22-test-the-deployed-endpoint)
- [3. Test the endpoint with Postman](#3-test-the-endpoint-with-postman)
  - [3-1. Copy Rest endpoint and Primary key](#31-copy-rest-endpoint-and-primary-key)
  - [3-2. Configure on Postman](#32-configure-on-postman)
  - [3-3. Test the endpoint](#33-test-the-endpoint)
- [Conclusion](#conclusion)

## Prerequisites
- Complete to create a flow (see [How to Easily Build a PDF Chatbot with RAG (Retrieval-Augmented Generation) Using Azure Prompt Flow](https://dev.to/aykhara/how-to-easily-build-a-pdf-chatbot-with-rag-retrieval-augmented-generation-using-azure-ai-studios-prompt-flow-boc))
- Download [Postman](https://www.postman.com/downloads/)
- Optional : Complete to evaluate a flow (see [How to Evaluate a PDF Chatbot Response with Prompt Flow](https://dev.to/aykhara/how-to-evaluate-a-pdf-chatbot-response-with-prompt-flow-c56))

## 1. Modify a flow
### 1-1. Replace with vector db lookup
Need to replace the "index lookup tool" with "vector db lookup".
Add "Vector DB Lookup" from More tools.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fncv6z1pppevintq3r0s.png)

Then, enter a node name (e.g. search) and add it to the flow
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rzzm5qufytsy8o478gda.png)

Finally, input values for the node and Save the change.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8qu31vm328c4bgba2fmw.png)

If you want to know where these information come from, you can check them in Azure AI Search on Azure portal and navigate to indexes as below.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j9plihf4omcrynx28uzf.png)
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/400buqxl41rhd65b4kgy.png)

After creating the "Vector DB Lookup" node your graph will look like the screenshot below.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vk5fyhj3jzcxe4zhs3x6.png)

### 1-2. Delete an unnecessary node
Delete the "Vector Index Lookup - search_question_from_indexed_docs" and Save the change
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1nv0i1w1tbukii9ak0ir.png)

### 1-3. Change the value of search result node
Change the value of search_result to "$(search.output)" in generate_prompt_context node and Save the change

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/94k9eeo7e0ktoh1ekwgq.png)

After completing all the steps to replace the "index lookup tool" with "vector db lookup" your graph will look like the screenshot below.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qxbvurpzxe9b2edyiq1s.png)

### 1-4. Try chat
Try the chat with your PDFs to make sure if it works as expected.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jt6xpa4zfqvp09uc5xe5.png)

## 2. Deploy a flow
### 2-1. Deploy a flow from Prompt Flow tab
Click "Deploy",
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sa7jy87u1y9buklnh57d.png)

and change basic settings if you wish.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/183fwqqk6c7b3w7d7si3.png)

Review the deployment settings and Create it.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o6lr08zr7p95oo7ko59p.png)

It will take around 10 minutes to complete the deployment.

### 2-2. Test the deployed endpoint
Move to Deployments tab and select the deployed endpoint,
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/id2ezbc2u56x5xdyrmc0.png)

then test the endpoint.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3vkara4xdsxuzvbdkfxz.png)

## 3. Test the endpoint with Postman
### 3-1. Copy Rest endpoint and Primary key
Copy REST endpoint and Primary key from Consume tab

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/obqbvfgb11x0yj0tth1k.png)

### 3-2. Configure on Postman
- POST : REST endpoint
- Headers

|KEY|VALUE|
|-|-|
|Authorization|Bearer {Primary key}|
|Content-Type|application/json|

- Body : raw (json)

```json
{"question":"who are you","chat_history":[]}
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oco6665iznzmzxbvxox4.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d5pbhqy9hm8s9vqgiwso.png)

### 3-3. Test the endpoint
Click "Send" and get the response!

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w00er4yse4evn6znoduz.png)

## Conclusion
In this blog, we have explored how to deploy the chatbot, implemented using Azure AI Studio's Prompt Flow, as a REST endpoint and how to test it using Postman. Next, we plan to introduce a method for implementing a simple application that utilizes the REST endpoint we've just deployed.