# Multitask Learning with Knowledge Base for Joint Intent Detection and Slot Filling
## 1 Introduction  
With the development of the task-oriented dialog system, nature language understanding (NLU), as a critical component of the task-oriented dialog system, has attracted great research attention. We can capture context information to identify the user's intent by using intelligent interactive devices that talk to humans in different scenarios, and extracting the semantic constituents from the text that the user inputs into the semantic slots that were previously defined [1]. These two modules, namely intent detection and slot filling, can convert the text into its semantic representation, which provides the task information for supporting the dialog system and helps users achieve their demands. Intent detection is the task of classifying natural language utterances into semantic intent classes that have been previously defined. Take Siri as an example. A sentence spoken by a user, such as, \"Tell me about the weather\" should be classified as a weather-query subtype. Several classifiers, such as the support vector machine (SVM) and convolutional neural network (CNN) have been applied to detect a user's intent. Generally, the intent-detection method constructs a classifier based on a lexicon and semantic feature training dataset [2, 3]. However, since the text that a user inputs into the task-oriented dialog system is generally short and a large-scale corpus cannot be found that can be used to train a high-quality model, the intent-detection task cannot be adequately fulfilled. Slot filling is used to allocate appropriate semantic tags to each input word by extracting the semantic concept. For instance, for the sentence \"I want the cheapest airfare from Tacoma to Orlando\", the slot-filling task should tag Tacoma as the departure city and Orlando as the arrival city. It can be treated as a sequence-labeling task that maps an input word sequence to thecorresponding slot-label sequence. Popular approaches to solve the sequence-labeling problem include generative models [4], such as composite hidden Markov models (HMM), and discriminant or conditional models, such as conditional random fields (CRF) [5] and the support vector machine (SVM) [6; 7]. Despite many years of research, the slot-filling task is still a challenging problem, and a few neural-network and deep-learning approaches have been used to solve this problem in recent studies. Intent detection and slot filling have always been adopted as the pipeline modules in traditional task-oriented dialog system research. However, the pipeline methods propagate errors easily. Differently from the pipeline methods, the joint models have the advantages of utilizing the dependency between intents and slots. These works are mainly divided into two categories. One is the use of semantic analysis [8; 9]. Another type of joint model is deep learning, which integrates both feature design and classification into the learning procedure [10; 11]. But little research has made full use of additional information, such as the incidence relations and the shared resources between intent-detection and slot-filling modules. Moreover, the above studies have a common problem, i.e., that the standardized and high-quality public datasets are very insufficient, because collecting and labelling datasets require a lot of time and effort. Therefore, we attempt to add an external knowledge base to the existing datasets to solve the problem of data scarcity [12]. A knowledge base is a knowledge cluster engineered so that it is structured, easy to operate and use, and contains comprehensive and organized knowledge. In addition, it is a collection of interrelated knowledge pieces that are stored, organized, managed, and used in computer memory by a certain knowledge representation mode for solving problems in some fields. The latest researches show that the effects of NLP can be improved by introducing a knowledge base [13; 14].