# Multitask Learning with Knowledge Base for Joint Intent Detection and Slot Filling
## 4 Experimental Analysis
### Results  
The main task of the MTL with knowledge base for joint intent detection and slot filling that was proposed in this paper is to identify the intent and slot of the text that a user inputs in the dialog system. In this paper, the accuracy is used to evaluate the intent-detection task, and the F value is used to evaluate the slot-filling task. The experimental results are shown in Table 2. With the ATIS dataset, the accuracy values of intent detection and the F values of slot filling obtained by our model are 98.83% and 97.06% respectively, and with the Snips dataset, the accuracy value and F value are 98.79% and 97.31%, respectively. For comparison, the RNN-LSTM [26], attention-based RNN [27], slot-gated [28], joint BERT [33], and CAPSULE-NLU [37] models have demonstrated their performance on the ATIS and Snips datasets. It can be seen from Table 2 that, according to the accuracy index, our model outperforms the best joint BERT model [33] by 1.33% and 0.19% among the group of comparative models. Meanwhile, with respect to the F index, our model outperforms the best joint BERT [33] by 0.94% and 0.31% among the group of comparative models. As shown in Table 2, we performed ablation analysis on the ATIS dataset. In the ATIS dataset, without joint learning, the accuracy of intent detection drops to 96.19% from 97.40%, and the F1 score of slot filling drops to 94.90% from 96.16%. Without a knowledge base, the accuracy of intent detection drops to 98.37% from 98.83%, and the F1 score of slot filling drops to 96.42% from 97.06%. The Snips dataset demonstrates the same trends. In other words, strong associations and mutual promotion between tasks may improve the intent-detection and slot-filling performance. Moreover, integrating knowledge base facilitates the performance improvement of intent-detection and slot-filling modules.  
Three reasons may contribute to the superiority of our joint model, namely, the shared parameters, knowledge base, and joint promotion. First, our joint model uses the LSTM-CNN shared representation layer to obtain shared resources, which can be used to better learn more common information from the dataset and can lead to high scalability. Second, a knowledge base is introduced into the model to improve its performance. Third, we designed a weighted-loss function to build the joint optimization of these modules. In other words, strong associations and external knowledge bases between tasks promote the model's performance.  
Here, we discuss an error analysis of the proposed methods. This error analysis will help improve our model. Two factors lead to error in our model. The first is lexical ambiguity. Ambiguity is a natural and common linguistic phenomenon in English. Some nouns were incorrectly identified as verbs, such as in the following sentence: \"I would give this current book a rating of five and a best rating of six.\" If we consider the word \"book\" as a verb, it should be marked as \"O\". However, this word is a noun and should be marked as \"object_type\". The second factor is the entity recognition problem. Some phrases cannot be recognized as an entity; e.g., \"Find a photograph called 'call on me'.\" Here, \"call on me\" is an entity but cannot be recognized, which means we could not find the correct referred entity for the given mention.