Decoding Dialogflow by coding your own chatbot
By: Ovo Ojameruaye
Tags: Blog Post, Machine Learning, Chatbots, NLP, Dialogflow

Retrieval based chatbots: Similar to the rule-based models, all responses are predefined with retrieval chatbots. The difference, however, is in the process of going from the user input to returning a response. While rule-based models make use of pattern matching in providing a response, retrieval-based models utilize classification algorithms to obtain intent from the user input.

Classification algorithms naturally infer that a model must be trained to help it learn how to classify user input. The number of classes are limited and have been defined as labels from training logs. Once the intent has been classified and entity extracted, this is used to pick the appropriate response. The context can also be considered when forming a response. The rules are implicit — they are learned by the AI system.

Generative Models: Unlike the rule based and retrieval models, these models generate response based on the user input and previous context of the conversation. They generate a reply one word at a time by computing probabilities over the whole vocabulary of the model. They are also capable of continuously learning from interacting with the user.

As deep learning advanced, generative models built with end-to-end trainable neural nets began replacing rule-based and retrieval models around 2015. More specifically, the recurrent encoder-decoder model and variations of it are seen to have dominated the task of conversational modeling.

Intents, entities, recurrent neural networks. What are these really about?

Let’s first understand some jargon before we get our hands dirty:

Utterances: Anything that a user says can be regarded as utterances. Think of it as the input.

Responses: The reply to user’s utterances. Anything that a bot says is a response.

Intent: The user ‘intent’ refers to what the user ‘intends’ to do while interacting with the chatbot. Think of them to be like ‘topics’ in a particular chapter of a book. For example, the customer says ‘I want to order a pizza’, then the intent is order_pizza for the bot. The number of intents a bot has depends upon the application the bot is made for.

Entity: Entities are one level deeper than intents. They act as metadata for intents, describing in detail what in particular the user’s intention is. They can represent quantities, counts, names etc. For the above pizza example, the user can say “I want to order two large vegetarian pizzas with alfredo base”, it says more about what kind of pizza the user wants. There can be multiple entities inside an intent. Here, ‘two’, ’large’, ’vegetarian’, ’alfredo base’ can be treated as entities.

Seq2Seq Models: The generative chatbot is powered by AI. This approach is typically based on recurrent neural networks. This tends to be quite an advanced topic but we will do our best to illustrate with a basic example. A variation currently used are Sequence to Sequence models which can be implemented as two LSTMS (called an encoder and decoder).

Source — link

The encoder’s job is to take the user’s input and previous message as a sequence of word vectors and produces something known as a thought vector which essentially captures the meaning and represents the whole input sequence. The decoder converts the thought vector into a sequence of word vectors (the response). Each word in the sequence is generated based on the previous word and thought vector. At each step, the model outputs the word with the highest probability.

A cool resource on the nitty gritties of Seq2Seq models can be found here. Training the model gives the chatbot the ability to learn the process of generating relevant and grammatically correct responses to input utterances.

This all sounds so interesting, how can I actually build one?

Building a ChatBot

Chatbots have traditionally been challenging to understand and build. Google has simplified this process with its tool Dialogflow. Dialogflow lets you build conversational interfaces by providing a powerful natural language understanding (NLU) engine to process and understand what your users are looking for.

Before we dive right into it, some of these terms come up often while we talk about chatbots in Dialogflow, so why not get that out of the way.

Context: Context refers to the user’s intent in general. Related intents put together make up context. Contexts are used to determine the flow of the conversation.

Agents: In DialogFlow, an agent is the heart of the chatbot. It is similar to a human call centre agent. Agents are NLU modules that process inputs given by a user. They are top-level containers for all intents, entities and contexts. An advantage of using DialogFlow is the set of pre-built agents provided for generic use-cases.

Confidence Score: Chatbots input utterances and calculate intent. Their classification of intent outputs a confidence score demonstrating how confident the machine learning model is of correctly understanding the user’s intent. We can threshold the confidence score to handle cases and optimize intent classification.

Intent classification: In order to classify the user’s input as a particular intent, the text is first pre-processed to convert it into numerical format which can be understood by the ML model. Pre-processing could include but are not limited to conversion to lower case, tokenization, stopword removal, stemming and vectorization of tokens.

Pipeline for Intent Classification

The classification can be done with various algorithms ranging from Multinomial Naïve Bayes, SVMs to using Neural Networks. The classifier is trained using labelled conversational logs where different sentences have been mapped to intents.

Entity recognition: Extracting the entity encompasses a technique known as Named Entity Recognition. NER locates and classifies named entities in text into predefined categories such as the name of a person, location, organization etc. It is very strongly dependent on datasets and heuristics (e.g capital letters, question marks).

Enough theories!!! Show me how to code!

Really….Building a Chatbot

Finally we get to the part where we build a chatbot using DialogFlow, Node.js and React.js. The basic architecture of our app would look something like this.

We will have front-end user interface where the end-user inputs their queries, which is passed on to the backend-app which communicates with Dialogflow using an API. The responses are then passed on from Dialogflow to the backend and finally displayed to the end-user.

The process of building our chatbot would follows these steps: