# Multitask Learning with Knowledge Base for Joint Intent Detection and Slot Filling
## 2 Related Work  
Both the intent-detection and slot-filling modules are usually used to convert the text that a user inputs into the task-specific user's intent and task-specific semantic representation [16]. Previous work mainly viewed intent detection as an utterance classification task and explored various classification methods. A common approach is to build a multiclass classifier that is trained using the lexical and semantic features of the utterances. In addition, with the development of deep learning, deep belief nets (DBNs) have been used for natural language call routing's intent-detection task for the first time, and they have produced better results compared with the traditional model [17]. Although standard classifiers have produced good results for the intent-detection module in several domains, it is not enough to rely on intention labels solely. Some studies look for auxiliary tasks to promote the intent-detection model's implementation. [PERSON] presented a probabilistic topic model for identifying hidden semantic intent classes and intent-bearing constituents from spoken language utterances [18]. [PERSON] proposed a variational Bayesian approach for modeling the latent intents of user queries and clicked URLs when available [19]. They used this model to enhance the supervised intent classifications of user queries from conversational interactions. All the above tasks use auxiliary tasks to help the implemented intent detection, but it is easy to incur unnecessary overheads. [PERSON] constructed an intention-detection model that can identify students' learning styles by using the integrated classification method. It combines three classifiers: SVM, naive Bayes (NB), and K-nearest neighbor (KNN), which are based on majority voting rules to effectively utilize personal characteristics (i.e., age and gender) and cognitive characteristics as the minimum amount of input for model training to determine learning style [20]. [PERSON] proposed a binary classification framework based on deep neural networks (DNN) and extracted the most important features that had the positive or negative impact on the final prediction [21]. Slot filling can be treated as a sequence-labeling task. Traditional approaches including the HMM and CRF can solve the problem to some extent, but the F values are poor. Recently, neural network models, such as recurrent neural networks (RNNs), LSTM, and CNNs have been applied to the slot-filling module [22; 23]. [PERSON] tried to use a bidirectional-LSTM-CRF (BiLSTM-CRF) in the sequence labeling for the slot-filling task and showed that the BiLSTM-CRF model provides a significant improvement compared with other models [24]. Joint models for intent detection and slot filling have also been explored. These works are mainly divided into two categories. One is the use of semantic analysis. [PERSON] presented a dependency parsing-based sentence simplification approach that extracts a set of keywords from natural language sentences and uses those in addition to entire utterances for completing NLU tasks [8]. [PERSON] showed that a semantic tree can be parsed by recursively using discriminative semantic classification models whose outputs were used to recursively construct a semantic tree, resulting in both slot and intent labels [9]. Although great progress has been made, these methods require good feature engineering and even additional semantic resources. Another type of joint model for intent detection and slot filling is deep learning, which integrates both feature design and classification into the learning procedure. [PERSON] proposed a triangular CRF (TriCRF) that coupled an additional random variable for intent detection with a standard CRF. This structure both encodes the dependencies of intents and slots and preserves the uncertainty between them [10]. [PERSON] improved the RNN by using the transition features and the sequence-level optimization criterion of the CRF to explicitly model the dependencies of output labels [11].