Make Your Bot Understand the Context of a Discourse
By: Yuli Vasiliev
Tags: Chatbots, Chatbot Development, Python, NLP

Photo by Igor Shabalin

One of the most challenging tasks in chatbot development is to ‘teach’ the bot to recognize the context of a sentence being processed. Many good designed bots are supposed to understand what stands for pronoun in the current sentence, based on what was said in a discourse previously. Say, a smart bot should understand that in the phrase: ‘What do you know about it?’ ‘it’ is a substitute for a word or a phrase mentioned in one of the previous sentences.

Understanding Substitutes

It’s quite common in natural language to use pronouns in place of nouns, proper nouns, or even entire phrases, which were mentioned earlier in a discourse. For those who develop bots, this makes things a bit more complicated. Not so much though, as you’ll see in the article example.

To understand what the problem is, let’s look at a simple example. The following screenshot shows a fragment of a discourse with a bot that can understand the meaning of a certain question and respond accordingly, but absolutely does not remember a preceding question:

As you might guess, this bot doesn’t remember a previous question.

In other words, it doesn’t maintain the context of the discourse. In this particular example, a smart bot should have replaced the question:

What did he draw?

With the following one:

What did Raphael Santi draw?

The latter should be then sent to the underlying layers of the bot application for further processing and generating a meaningful answer:

Using Linguistic Features to Catch Necessary Pronouns

This is where tools like spaCy may come in handy. spaCy automatically assigns so called linguistic features (aka linguistic annotations) to each token in a text being processed. Thus, using part of speech tags allows you to easily recognize necessary pronouns in a sentence. The following script provides an illustration of how both coarse-grained and fine-grained part-of-speech tags can be obtained with spaCy:

import spacy

nlp = spacy.load('en')

doc = nlp(u'What did he draw?')

for token in doc:

print(token.text, token.pos_, token.tag_)

This should give you the following output:

What PRON WP

did AUX VBD

he PRON PRP

draw VERB VB

? PUNCT .

As you can see, the sample sentence includes two pronounces, but only one of them is a personal pronoun — tag PRP indicates this.

Let’s now look at how this can be used in practice. Here is a simplified example that illustrates how two consecutive questions might be processed using the linguistic features available in spaCy, assuming the second question contains a pronoun whose antecedent can be found in the first one. The code is supposed to replace this pronoun in the second question with its antecedent mentioned in the first question (be warned that the following code is a simplification — just to illustrate the point).

… doc = nlp(u’Who was Raphael Santi? What did he draw?’)

doc_sents = list(doc.sents)

sent_prev = doc_sents[0]

sent_cur = doc_sents[1]

sent_upd = sent_cur.text

prons = ['he', 'she', 'him', 'his', 'her', 'they', 'them', 'their']

for token in sent_cur:

if (token.tag_ == ‘PRP’ or token.tag_ == ‘PRP$’) and token.text.lower() in prons:

ent = find_antecedent(sent_prev)

if ent:

pron_i = token.i — sent_cur[0].i

sent_upd = sent_cur[:pron_i].text + ' ' + ent + ' ' + sent_cur[pron_i + 1:].text

print(sent_upd)

In this particular example, the updated second question is simply printed out. In a real-world implementation, it would be submitted for further processing instead of its original version.

Looking through the above code, you may notice that it relies on the find_ antecedent() function, which, as you might guess, looks into the preceding sentence and finds a word or phrase that likely is the antecedent for the pronoun in the current sentence. The implementation of this function is based on using the spaCy’s named entity recognizer, as explained in the next section.

Using Named Entity Recognition for Finding Antecedents

The find_ antecedent() function invoked in the snippet shown in the previous section might be implemented as follows:

def find_antecedent(sent_prev):

for ent in sent_prev.ents:

if ent.label_ == 'PERSON':

return ent.text

return False

For simplicity, the above implementation is specific only to finding the name of a person. Note that a named entity recognized by spaCy may consist of more than one token. In this particular example, the name ‘Raphael Santi’ will be recognized as a single entity labeled as the PERSON entity type.

So, the sample discussed here should produce the following output:

What did Raphael Santi draw?

thus, replacing pronoun ‘he’ with the name that stands for it:’Raphael Santi’ , according to the context of this particular discourse.

After incorporating this technique into your bot implementation, it should correctly recognize what stands for some of pronounces in a discourse, thus getting an opportunity to generate more relevant answers to user questions. Turning back to the screenshot of a bot you saw in the beginning of this article, this now might look as follows: