Understand Virtual Assistants (Post 1)
By: Trinh Gia Thanh
Tags: Naturallanguageprocessing, Blog, Virtual Assistant, Machine Learning, Artificial Intelligence

Introduction

Have you ever picked up your phone and asked the virtual assistant (VA) to do something? Have you lived in a SMART house where people can tell the system to open the light or make coffee for them? Or even have you seen Tony commanded Jarvis to control ? But have you wondered how these assistant work? In this series of blog, I will walk through a project which I have worked on, where we will discover the technology behind these interesting assistants.

This first blog will be about the general goal of the project and analysis on the data that I used to train my “small” VAs system.

Goal

Assuming that the VA is already able to analyze human voice and translate those audios into sentences, all of our work will mainly focus on analyzing human’s commands.

Clearly, we cannot predict all the things the speakers might say and program the VA system to accomplish the requests based on every different sentence. All we can do is that we can define possible tasks that user may demand and design our system to recognize and complete the tasks. Therefore, it is essential for us to implement Natural Language Processing (NLP) algorithms on these system. We will use NLP in order to help our VAs to figure out the speaker’s intent and all important information related to the intent.

For example, if we command the virtual assistant “Book a table for six at Intercontinental for this Sunday night”, we will expect the final prediction from VAs in this form:

{

'intent': 'BookRestaurant',

'slots': {

'party_size_number' : 'six'

'restaurant_name' : 'Intercontinental'

'timeRange' : 'Sunday night'

}

}

Data

I. Predefined Classes

In this project, we will use a voice command prepared and published by SNIPS.ai, a French startup which was bought last November by Audio device manufacturer Sonos. We are using a preprocessed version of this dataset with token-level BIO-tagging, which was prepared by Su Zhu.

The dataset contains 5 main files: “train”, “valid”, “test”, “vocab.intent”, and “vocab.slot”. Now, let’s discover what it is in it.

In the “vocab.intent” file, there are 7 predefined possible intents for our VAs system, which include AddToPlaylist, BookRestaurant, GetWeather, PlayMusic, RateBook, SearchCreativeWork, and SearchScreeningEvent.

AddToPlaylist

BookRestaurant

GetWeather

PlayMusic

RateBook

SearchCreativeWork

SearchScreeningEvent

This dataset is only for us to discover about virtual assistant, in real world, VAs definitely need to be able to solve more tasks.

Corresponding to each intent, VAs also need to identify related information. These information can be found through a process called Named Entity Recognition (NER), which identifies and classifies all name entities from the text into intent-related categories such as person names, times, and locations. The information then will be tagged with BIO format (we will explain this later). We also defined some possible types of related information, which are stored inside “vocab.slot”

B- album

B- artist

B- best_rating

B- city

B- condition_description

B- condition_temperature

B- country

B- cuision

...

I- album

I- artist

I- best_rating

...

O

As we can see, the data is BIO-tagging format, which is common in NER.

B: beginning of a chunk

I: inside of a chunk

O: outside of a a chunk, which means this token relates to no chunk.

Using the previous sentence “Book a table for six at Intercontinental for this Sunday night”, the token-level BIO tagging would be:

Book : O

a : O

table : O

for : O

two : B-party_size_number

at : O

Intercontinental : B-restaurant_name

for : O

this : O

Sunday : B-timeRange

night : I-timeRange

II. Training Data

Now, let’s look into training data in the “train” file. We write a function to parse lines of training data into JSON format to get each elements of a commands, which includes the intent, command, length of the command, word_level BIO tags and then store the results in a pandas DataFrame.

Let’s take a deep analysis of this training data. To begin with, we count the number of examples of commands in each intent_label.

intent_labels | words

1. AddToPlaylist | 1842

2. BookRestaurant | 1873

3. GetWeather | 1900

4. PlayMusic | 1900

5. RateBook | 1856

6. SearchCreativeWork | 1854

7. SearchScreeningEvent | 1859

The numbers of each intent_labels’ sentence is pretty even, therefore, the training data is not necessarily biased toward any specific intents.

III. Validation and Test Data

We have separated datas in order for our system not “cheat” during developing. The Training Data set is used to train and fit our system. On the other hand, the Validation Data and Test Data Set are used to unbiasedly evaluate our model, to test if the model has been overfitted or not. The system may perform well on the Training Data Set but may not on the other twos, which is why we split our dataset.

Same with Training Data, we also need to create a pandas DataFrame for both Validation and Test Data. Taking a quick look over the two DataFrame, we also see that there is an exact even spread of number of 100 sentences in 7 intent_labels in both sets.

Conclusion

Until this point, we can feel confident in our chosen datasets for training our VAs. This is also the end of the first blogs in the series. In the next blog, we will focus on a more interesting part, which is classifying voice commands.

Hope you enjoy reading this blog and See you soon in the next one!

Thanh Trinh is a Student Ambassador in the Inspirit AI Student Ambassadors Program. Inspirit AI is a pre-collegiate enrichment program that exposes curious high school students globally to AI through live online classes. Learn more at https://www.inspiritai.com/.