How to build chatbots with Contentful and Azure functions
Tags: azure, contentful, bots, javascript
Published: 2020-08-27 11:18:31

---
title: How to build chatbots with Contentful and Azure functions
published: true
date: 2020-08-26 09:00:00 UTC
tags: azure, contentful, bots, javascript
canonical_url: https://www.contentful.com/blog/2020/08/26/powering-chatbots-with-contentful/
---

In 2019, I was boarding a plane for London when I realized that I forgot my shaving kit. I quickly opened the app for the hotel that I would be staying in. Thanks to its chatbot feature, I dropped them a request. They were able to immediately solve my shaving problem and, in addition, booked me a taxi from the airport.

The benefits of chatbots are pretty evident: they make customers happy, they operate as a sales tool and allow you to maintain a conversational relationship. I didn’t get frustrated, I stayed connected with the hotel and now we’ve built a conversational history.

Because conversational bots offer an important channel to reach customers, most brands are developing expert bots that provide specific solutions. Some chatbots provide FAQ solutions, others recommend specific products depending on the customer’s needs, and still others quickly reply to questions about billing and account information. Contentful can handle the content layer of your chatbot, and this post walks you through the architecture, implementation and results.

Specific context and tech stack for chatbot
-------------------------------------------

One of our enterprise customers wanted to build an FAQ chatbot with Contentful after they used the content platform to successfully power their web products. They were already using Microsoft cloud services, so we decided to build a quick proof-of-concept using Microsoft Azure functions and knowledge base. Contentful works with these tools, but can also be used with other cloud and bot services like Amazon Web Services, Google Cloud Platform, or IBM Watson thanks to its APIs. After some initial investigation, we developed the architecture presented here, which uses Contentful webhooks, Azure cloud functions and Microsoft’s knowledge base service.

Architecture
------------

![The Contentful chatbot publishing flow ](https://images.ctfassets.net/fo9twyrwpveg/1m7uZ9141mcgr2FPGpe0yy/c9df075a3c44bcf81528154f2dfd65ce/BLOG_PoweringChatbots_ArchitectureOverview.png)

### Publishing flow

*   Editors will publish the FAQ in Contentful
    
*   Publishing of the FAQ will trigger a webhook in Contentful
    
*   The webhook will call our publish cloud function
    
*   The function will push the content to our knowledge base
    

### Unpublishing flow

*   Editors will unpublish the FAQ in Contentful
    
*   Unpublishing of the FAQ will trigger a webhook in Contentful
    
*   The webhook will call our unpublish cloud function
    
*   The function will remove the FAQ from our knowledge base
    

Implementation
--------------

### Contentful

We used Contentful to store questions and answers for the chatbot. In the Contentful web app, editors can easily write, edit and publish that content. 

#### Content Model

We’re keeping the content model very simple for this demo — just a single content type holds our question and answer pairs, as well as an internal title for content editors to have as a reference. 

![Contentful FAQ content model](https://images.ctfassets.net/fo9twyrwpveg/5biAKVZbj7JUiecCL4t2br/c665fe01e49c7a7ec43ab9bbc2070db5/Explaining_Content_Models_-_ChatBots.jpg?q=1)

#### Webhooks

We use webhooks for publish and unpublish events, triggering our cloud functions.

![Using Contentful webhooks to build a chatbot](https://images.ctfassets.net/fo9twyrwpveg/6RqTPRlpbRC7LxGD0O2yw5/0e4c36d9694a6c7e5f956fafb9a12e2a/Screenshot_2020-04-27_at_11.44.59.png?q=1)

When creating the publish event webhook, make sure to set the trigger of the webhook for the entry publish event as shown below. For the URL, we will need to add the endpoint of our cloud function, as we have not configured any cloud function yet. We can use the [DummyWebhook](https://webhook.site/#!/a74d2708-dd74-49e2-8bf2-ee616bcf6fa0) website or [ngrok](https://ngrok.com/) to create dummy endpoints and add them.

![Creating dummy end points and adding them](https://images.ctfassets.net/fo9twyrwpveg/3R7YcQ9VX94jdDpUBkFs9A/4c54f9f1bf5740b77011dda8e8bf43fe/unnamed.png?q=1)

When creating the unpublish event webhook, make sure to set the trigger of our webhook for the entry unpublish event as shown below. Just like last time, we’ll populate the URL with a dummy value from DummyWebhook or ngrok.

![Creating the Contentful chatbot](https://images.ctfassets.net/fo9twyrwpveg/1ruUdOy3OGt2hQHTeHZa6U/b3fcd6e8658f1f55e86c7f67b9fab45e/Screenshot_2020-04-27_at_11.51.52.png?q=1)

Once done, you can create an entry and test our workflow by publishing and unpublishing it and checking the webhook payload being sent. You can then view the payloads in our webhook logs.

Publish entry webhook payload:

    {
      "sys": {
        "type": "Entry",
        "id": "1ssZF4Lqq1Wpi9KSHvq9jE",
        "space": {
          "sys": {
            "type": "Link",
            "linkType": "Space",
            "id": "bwn0phmhnub6"
          }
        },
        "environment": {
          "sys": {
            "id": "master",
            "type": "Link",
            "linkType": "Environment"
          }
        },
        "contentType": {
          "sys": {
            "type": "Link",
            "linkType": "ContentType",
            "id": "faqQuestion"
          }
        },
        "revision": 2,
        "createdAt": "2020-04-13T23:06:07.225Z",
        "updatedAt": "2020-04-13T23:07:05.635Z"
      },
      "fields": {
        "title": {
          "en-US": "First question"
        },
        "question": {
          "en-US": "What is contentful? \n\n"
        },
        "answer": {
          "en-US": {
            "data": {},
            "content": [
              {
                "data": {},
                "content": [
                  {
                    "data": {},
                    "marks": [],
                    "value": "Contentful is Headless CSM",
                    "nodeType": "text"
                  }
                ],
                "nodeType": "paragraph"
              }
            ],
            "nodeType": "document"
          }
        }
      }
    }

Unpublish entry webhook payload:

    {
      "sys": {
        "type": "DeletedEntry",
        "id": "1ssZF4Lqq1Wpi9KSHvq9jE",
        "space": {
          "sys": {
            "type": "Link",
            "linkType": "Space",
            "id": "bwn0phmhnub6"
          }
        },
        "environment": {
          "sys": {
            "id": "master",
            "type": "Link",
            "linkType": "Environment"
          }
        },
        "contentType": {
          "sys": {
            "type": "Link",
            "linkType": "ContentType",
            "id": "faqQuestion"
          }
        },
        "revision": 2,
        "createdAt": "2020-04-13T23:27:30.971Z",
        "updatedAt": "2020-04-13T23:27:30.971Z",
        "deletedAt": "2020-04-13T23:27:30.971Z"
      }
    }

For now, that’s all we have to do within Contentful. 

### Creating the FAQ chatbot in Microsoft Knowledge Base

#### QnA service

1\. Go to [https://www.qnamaker.ai/](https://www.qnamaker.ai/) and create a free account.

![Creating an account at QnA](https://images.ctfassets.net/fo9twyrwpveg/287CZO8MXz55uu1sUmRMXm/8abd914ace70d5a9e65412d5b15497d9/unnamed__1_.png?q=1)

2\. Create a knowledge base.

![How to create a knowledgebase in QnAmaker](https://images.ctfassets.net/fo9twyrwpveg/Y767pq5Udo6VfIJ4297py/3c7b8257e9c0c7e944598e4fb6ed4aa7/unnamed__2_.png?q=1)

3\. Create a QnA service.

![Create a QnA service
](https://images.ctfassets.net/fo9twyrwpveg/4YYH3nvtJMYq9FBwbHABCx/6198d86530070e5f5188f1206c088673/unnamed__3_.png?q=1)

4\. Go back to [https://www.qnamaker.ai/](https://www.qnamaker.ai/) and refresh to see your subscription name and Azure QnA service.

![Go back and refresh to see your subscription name and Azure QnA service.
](https://images.ctfassets.net/fo9twyrwpveg/1QyvFGK88dhGT2zHZfbbHT/ec809ba1475cae93640edc4f0c0d3b6a/Screenshot_2020-04-29_at_14.49.15.png?q=1)

5\. Complete the rest of the steps to create a knowledge base. Once done, you will have a knowledge base similar to the one shown below.

![Complete the rest of the steps to create a knowledge base. Once done, you will have a knowledge base similar to the one shown below.](https://images.ctfassets.net/fo9twyrwpveg/7lnju7tC37f770QU4FRNDX/98830774c35a3704dcbe5582dfa805dd/unnamed__4_.png?q=1)

#### APIs to integrate

Now that we’re done with creating our knowledge base, we need to populate it with questions and answers coming from Contentful for the chatbot to use. Let’s explore which APIs we can use to add, update or delete information from our knowledge base.

We will be using the [update endpoint](https://docs.microsoft.com/en-us/rest/api/cognitiveservices/qnamaker/knowledgebase/update) to add and remove content. We will also use the [download endpoint](https://docs.microsoft.com/en-us/rest/api/cognitiveservices/qnamaker/knowledgebase/download) to get all questions and answers and then filter them.

#### Azure functions

Azure functions are a powerful way to develop and deploy serverless applications. For this use case we chose Azure functions as we can quickly develop and test our proof of concept.

#### Setup

This [video tutorial from Microsoft](https://www.youtube.com/watch?v=F0dJz8LLF4Q) gives you a guide to set up an Azure function project using Visual Studio Code.

We used the following settings for this chatbot project.

*   **Language**: you can select [multiple languages](https://docs.microsoft.com/en-us/azure/azure-functions/supported-languages), but we used JavaScript  
    
*   **Function trigger**: http
    
*   **Name**: the function we used **publishWebhookHandler** for the first function and **unpublishWebhookHandler** for the second function
    
*   **Access Rights**: set it to anonymous for now. For production, make sure to implement proper security mechanisms 
    

We created a new Azure function project with two functions: publishWebhookHandler and unpublishWebhookHandler.

#### publishWebhookHandler

This function will be used to **push content** into our **knowledge** **base** using the **webhook** coming from Contentful whenever the **content** is **published**.

**Algorithm**

1.  Retrieve the following from the webhook: entry ID, question and answer
    
2.  Make an [API](https://westus.dev.cognitive.microsoft.com/docs/services/5a93fcf85b4ccd136866eb37/operations/knowledgebases_download) call to our knowledge base to get all the questions and the answers. Once you have all questions, filter the question using metadata and check for the entry ID. We’re doing this to check if we have duplicate questions in our **knowledge base,** so we **update** them instead of **adding** a **duplicate** entry.
    
3.  If the question is **present** we will **update** that **question** using the [update endpoint](https://westus.dev.cognitive.microsoft.com/docs/services/5a93fcf85b4ccd136866eb37/operations/5ac266295b4ccd1554da7600). Use the same endpoint to add new questions. The only difference is the body of the payload. You can see in the metadata that we are adding key **contentfulid** and its **value**. This helps us **to map** entries in **Contentful** with entries in the **knowledge base** and will help us for **deleting** and **updating** **entries**.
    

Adding a new question:

    {
      "add": {
        "qnaList": [
          {
            "answer": "answer",
            "source": "source",
            "questions": [
              "question"
            ],
            "metadata": [{"name":"contentfulid","value":"1ssZF4Lqq1Wpi9KSHvq9jE"}]
          }
        ]
      }
    }

Updating a question:

    {
      "update": {
      	"name": "QnA Maker FAQ Prompts Bot",
        "qnaList": [
          {
          	"id":87,
            "answer": "answer 2",
            "source": "source",
            "questions": {
              "add":["questions 1"],
              "delete":[]
            }
          }
        ]
      }
    }

Sample function for adding content to our knowledge base:

    module.exports = async function(context, req) {
       if (req.body) {
           const contentfulID = req.body.sys.id;
           const contentfulQuestion = req.body.fields.question['en-US'];
           const contentfulAnswer = req.body.fields.answer['en-US'].content[0].content[0].value;
     
           const questionFromKB = await getQuestionIdFromKB(contentfulID);
           const response = await  upsertQuestionInKB(context, questionFromKB, contentfulID, contentfulAnswer, contentfulQuestion);
          
           context.res = {
               status: 200,
               body: JSON.stringify(response),
           };
       } else {
           context.res = {
               status: 400,
               body: 'Please pass a name on the query string or in the request body',
           };
       }
    };

#### unpublishWebhookHandler

This function **removes content** from our **knowledge** **base** using the unpublish **webhook**, whenever **content** is **unpublished**.

    module.exports = async function(context, req) {
     if (req.body) {
         const contentfulID = req.body.sys.id;
         const questionId = await getQuestionIdFromKB(contentfulID);
         const response = await removeQuestionFromKB(questionId);
         context.res = {
             body: JSON.stringify(response),
         };
     } else {
         context.res = {
             status: 400,
             body: 'Please pass a name on the query string or in the request body',
         };
     }
    };

**Algorithm**

1.  Retrieve the **entry id** from the webhook
    
2.  Make an [API](https://westus.dev.cognitive.microsoft.com/docs/services/5a93fcf85b4ccd136866eb37/operations/knowledgebases_download) call to our knowledge base to get all the questions, and filter the question using the **entry id**
    
3.  Make an [API](https://westus.dev.cognitive.microsoft.com/docs/services/5a93fcf85b4ccd136866eb37/operations/5ac266295b4ccd1554da7600) call to remove the question from our **knowledge base**. The payload will look similar to the following:
    
```
{
   "delete":{
      "ids":[
         29
      ]
   }
}
```

### Deploy

Once both functions are ready, they can be deployed to Azure Cloud.

![](https://images.ctfassets.net/fo9twyrwpveg/33g3RiFqq7D1f8K1T9HiR8/2a1c442f9e062f20f0df4472e755cabb/Screenshot_2020-08-26_at_11.04.16.png?q=1)

When the deployment is complete, get the **publishWebhookHandler** and **unpublishWebhookHandler** **URLs** from Microsoft Azure portal. Add them to the **webhooks,** which we created in the **first step**.

Testing our chatbot end to end
------------------------------

### Populate content

We **published** three **Q&As** for testing in Contentful. Each of the publish events triggered a **webhook** and they were pushed into our knowledge base. To check, we logged into [https://www.qnamaker.ai](https://www.qnamaker.ai/) to see if the questions are published.

On the screenshot below you can see the three published Q&As on Contentful as well as added on our knowledge base.

![](https://images.ctfassets.net/fo9twyrwpveg/1WPsAcKmNGrrgzK2hnvC6E/dd4fedf6fab1c0985f77acf4d4e0d1c3/Screenshot_2020-08-26_at_11.05.28.png?q=1)

Now, our knowledge base will train the content. Inside QnA Service, clicking on **Test** you can test your chatbot.

![](https://images.ctfassets.net/fo9twyrwpveg/20tj2zEsnpv58Ol6KhNptJ/92ca3a7b5a2732ca3291072fa4484c60/Screenshot_2020-08-26_at_11.50.56.png?q=1)

Results
-------

Now you have an intelligent chatbot customers can talk to. This way customers do not need to go through the extensive list on a website searching for a simple answer. You can also measure the sentiments of the customers and their frequent questions. With this additional information, we are empowered to improve our FAQs. 

### Future enhancements for our chatbot

1.  Our content model can be extended to add multiple questions for a single answer, and for every entry, additional metadata could be added. For this we have to update the sync functionality within the **publishWebhookHandler** function.
    
2.  Contentful also provides different [localization](https://www.contentful.com/developers/docs/concepts/locales/) patterns, which can be used to create content in multiple languages, allowing us to power multilingual bots. 
    
3.  In this article, we explored using the Microsoft tech stack, but similar things can be achieved using [AWS Lambda](https://aws.amazon.com/lambda/) and [AWS Lex](https://aws.amazon.com/lex/). Any chatbot platform that provides API access to their knowledge base can be used with Contentful. Using webhooks and cloud functions allow us to push content into a knowledge base and train it afterward.
    

If you’re looking to implement your own conversational chatbot, you can [sign up](https://www.contentful.com/get-started/) and try it for yourself. Feel free to contact us. Our solution services team works with enterprise customers to accelerate their digital journey.