# Multitask Learning with Knowledge Base for Joint Intent Detection and Slot Filling
## 2 Related Work
[PERSON] put forward a CNN-TriCRF model for improving the TriCRF model by using a CNN to automatically extract features to simultaneously handle intent detection and slot filling [25]. However, these works lacked the ability to acquire the long-term memory, which is crucial for sequence labeling. [PERSON] proposed an RNN-LSTM framework that completely estimated the semantic frame by leveraging public data from multiple domains [26]. [PERSON] and [PERSON] consolidated information about the hidden states from an RNN model for slot filling and then generated intent information by using an attention model [27]. This work applied a joint loss function to link the two tasks implicitly. [PERSON] connected and fed the output vector of BiLSTM and the CNN to the CRF layer to accomplish this task [28]. However, this research neglects the shared features of the intent detection and slot-filling modules. [PERSON] proposed a slot-gated model that introduces an additional gate that leverages the intent context vector for improving slot-filling performance [29]. This model considers the effect of intents on slots, but ignores the influence of the mutual promotion between these two modules. In recent studies, the application of incidence relations has been gradual. [PERSON] simultaneously optimized both the tasks via joint learning in an end-to-end manner [30]. [PERSON] proposed a bi-directional interrelated model for joint intent detection and slot filling [31]. [PERSON] proposed a multihead self-attention joint model with a conditional random field (CRF) layer and a prior mask [32]. The incidence relations between these modules have already been introduced to some extent. In addition, the pre-trained BERT model provides a powerful context-dependent sentence representation, which can serve to obtain semantic information easily. Semantic information is useful to the joint model. There are several studies that have focused on the BERT application. Namely, [PERSON] implements the BERT pre-trained model to address the poor generalization capability of NLU [33; 34]. [PERSON] adapted the original BERT fine-tuning method to define a new joint-learning framework [35]. Although the joint model can solve the problems in the current research to some extent, previous studies have not made full use of the shared resources between these modules. Furthermore, the knowledge base is of significance for various NLP applications. It can process information of types other than text and may serve to complete a task in the case of limited data. For example, [PERSON] proposed a knowledge-based topic model to extract more meaningful phrases and coherent topics [13]. [PERSON] introduced SemCluster, a clustering-based unsupervised key phrase extraction method that is used to mitigate the coverage limitation problem by considering knowledge of a wider background [14]. However, knowledge base has not been applied to intent-detection and slot-filling modules in recent studies. We consider that the current modeling methods do not make full use of the information, which can be extracted from a knowledge base. Since intents and slots are usually highly dependent on each other, our work focuses on how to establish the relationship between intents and slots based on MTL. At the same time, our model introduces a knowledge base to facilitate the performance improvement of intent detection and slot filling.