# Multitask Learning with Knowledge Base for Joint Intent Detection and Slot Filling
## 5 Discussion  
In this section, we demonstrate the influencing factors in our joint model, namely, shared representation, the BERT coding method, the knowledge base, and the learning-weighted loss function, respectively. First, we compared our MTL model based on the one-hot coding method with its pruning version that executes independent tasks on the \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**ATIS**} & \multicolumn{2}{c}{**Snips**} \\ \cline{2-5}  & **Intent** & **F1 Score** & **Intent Accuracy** & **F1 Score** \\  & **Accuracy (\%)** & **(\%)** & **(\%)** & **(\%)** \\ \hline RNN-LSTM [26] & 94.09 & 95.42 & 96.90 & 87.30 \\ Joint BERT [33] & 97.50 & 96.10 & 98.60 & 97.00 \\ Attention-based RNN [27] & 91.10 & 94.20 & 96.70 & 87.80 \\ Slot-gated [28] & 94.10 & 95.20 & 97.00 & 88.80 \\ CAPSULE-NLU [37] & 95.00 & 95.20 & 97.30 & 91.80 \\ MTL (One-hot) & 97.40 & 96.16 & 97.66 & 92.32 \\ MTL (BERT) & 98.37 & 96.42 & 98.52 & 97.03 \\ MTL with knowledge base & 98.83 & 97.06 & 98.79 & 97.31 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance comparison. ATIS dataset. The experimental results in Table 3 show that our joint model substantially outperforms both independent intent-detection and slot-filling models. Then, we analyzed the effects of the BERT pre-train language model on this model. BERT is helpful for textual encoding and suitably manages tasks with deep semantic features. In general, applying this method can greatly enhance the performance of the NLP tasks with just a few words. Compared with the one-hot encoding, the effectiveness increased by 0.97% and 0.26% separately. Furthermore, we further analyzed the key to the success of our joint model. From the shared resources standpoint, the shared parameters and features have been learned to represent the high correlation between intents and slots, which can fully determine the relationship between these two tasks. We introduced the WordNet knowledge base as an external knowledge source to the Bi-LSTM model. The experimental results demonstrate that this method solves the problem of unknown words, and the model performance improves. This is because some knowledge existing in the knowledge base can help identify words that do not appear in the dataset. Furthermore, we selected a case from ATIS. Here, for example, if the dataset contains only the word \"American\", the phrase \"the United States\" cannot be identified by relying solely on this dataset. However, if a knowledge base is introduced, by determining that \"America\" and \"the United States\" are synonyms, the phrase \"the United States\" can be identified. In addition, we analyze the mechanism of the weighted-loss function in our joint model, which simultaneously affects the intent-detection and the slot-filling task. We define \(\alpha\) as the weight coefficient, and the value of \(\alpha\) obtains the optimal configuration by using the gradient descent algorithm to solve the problem that the fixed-value weights cannot be reasonably distributed due to subjective factors. Furthermore, we show the influence of the self-learning weight based on the gradient descent algorithm. Figure 3 illustrates the change of \(\alpha\) in a certain iteration, which indicates that the weight can be acquired through learning. The value of \(\alpha\) continuously changes as the number of iterations increases. The value of \(\alpha\) is used for making the weighted-loss function converge faster than the original loss function. Figure 4 shows the comparison of the changes of the weighted-loss function,  
Figure 3: Self-learning weight \(\alpha\) value changes. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Intent Accuracy (\%)** & **F1 Score (\%)** \\ \hline Independent training intent & 96.19 & - \\ Independent training slot & - & 94.90 \\ Joint model & 97.40 & 96.16 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of independent and joint models. loss1, with the changes of the original loss function, loss2.