How to make "smarter" chatbots, Pt 1
Tags: ai, chatbot, machinelearning, nlp
Published: 2019-06-19 18:34:05

Often time when people think about using AI in their chatbots, they think they need to create a complex algorithm to understand what their user is trying to say, and they can't start anywhere. I have created many chatbots, some of which have over a million users, by following a simple idea.

The idea here is simple, each conversation has - at least - two sides. People tend to overanalyze things and so with a little help, your chatbot can appear smart, if it leaves room for interpretation. Consider the following dialog with ELIZA:

```
Men are all alike.
IN WHAT WAY
They're always bugging us about something or another.
CAN YOU BE MORE SPECIFIC
Well, my boyfriend made me come here.
YOUR BOYFRIEND MADE YOU COME HERE
He says I'm depressed much of the time.
I AM SORRY TO HEAR YOU ARE DEPRESSED
It's true. I am unhappy.
DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPY
I need some help, that much seems certain.
WHAT WOULD IT MEAN TO YOU IF YOU GOT SOME HELP
Perhaps I could learn to get along with my mother.
TELL ME MORE ABOUT YOUR FAMILY
```

The text in Caps is ELIZA and the text in the mixed case is the user. As you can see, ELIZA appears smart. It seems to know what the user is saying and seems to engage in conversation with them. But that's not the case. ELIZA operates by finding patterns in its input and converting them to reply. When she sees the word *alike* or *same*, she replies with *in what way*. Another pattern is if it sees something matching "I need X" it will reply with "What does it mean for you to get X". If X is *some help* or *vacation*, it makes perfect sense and ELIZA doesn't even know what vacation or some help means, and it doesn't need to know! But consider the following dialog:

```
I need this like a hole in the head
WHAT DOES IT MEAN FOR YOU TO GET THIS LIKE A HOLE IN THE HEAD
```

So, although this method will work 80% of the time, it is not that effective.

Another example to consider is Woebot, a chatbot that will make you feel better by spending 10 minutes a day talking to it. Now, Woebot actually uses some advanced NLU to understand what the user says, but not always. The only time they use it is when the bot has asked *how was your day* and then it will try to understand if the user is sad, happy or something else and respond appropriately. But once that step is done, it goes on a script-like mode, that seems to be really smart unless you respond badly. It even limits the user's response by using buttons instead of text input.

This type of response, where you present the user with a button, is helpful in so many ways. It makes the user's experience smoother, they don't like typing, and the developer's life easier, we don't like NLP/NLU that much.

This works 90% of the time, but sometimes we need a more specific input, or the user needs to provide more information. In these scenarios, I tend to do the following, prompt the user for text input, try to validate it somehow if you can, then respond accordingly.

Consider asking the user for her name, it should be an easy task. First we ask the user her name, then we see if it meets certain conditions, for example, we don't expect to see special characters in a name, then we just welcome them with this template `Hi NAME!`

Another example is asking the user how was his day. Here, we prompt the user, *How was your day?* and then we follow this pattern to create a response:

* Set ans to an empty string
* If NLU detected the user is feeling good, ans += "Glad to hear that!"
* If NLU detected the user is feeling bad, ans += "Sorry about that."
* If NLU detected the user is asking how are we, ans += "I'm okay, thanks for asking."
* ans += "So, let's discuss....."

This is a pretty simple pattern, you can make it much more complex, and it would work most of the times. We also don't need to worry, if the user says he had a good time and our NLU doesn't happen to catch it, so what, we will continue the dialog anyway. But if we happen to catch it, GREAT! The user will feel even better.

I'll try to cover some other techniques in my next post. Thank you for reading!

What do you use to make your user feel special? How do you communicate with them to make your bot feel smart?