# A Kazakh-Chinese Cross-Lingual Joint Modeling Method for Question Understanding
## 3 Proposed Method
### Bidirectional Propagation Module  
The subtask interaction module is a core component of the Bi-XTM model, designed to match the token-level intent encoding approach. Based on the collaborative interaction attention mechanism, a subtask interaction layer is added to mitigate the error propagation that may occur during the transfer of subtask vectors. The collaborative interaction attention layer in the Collaborative Interaction Transformer framework has demonstrated significant effectiveness in capturing the local interdependence between intent and slot vector representations. To further leverage this advantage, it is integrated into the subtask interaction module for the joint task of intent recognition and slot filling. Through the collaborative interaction attention mechanism, the model establishes a clear interaction between the two subtasks, enabling bidirectional synchronous updating and transfer of  
Figure 1: Bi-XTM model architecture. intent and slot vector representations. After encoding the input text, the intent vector representation \(H_{I}\) and the slot vector representation \(H_{s}\) are obtained. Then, linear functions are used to map \(H_{I}\) and \(H_{s}\) to their corresponding \(Query\), \(Key\), and \(Value\) matrices, as shown in Equations (2) and (3). \[Q_{I},K_{I},V_{I}=W_{q}^{i}H_{I},W_{k}^{i}H_{I},W_{v}^{i}H_{I} \tag{2}\]  
\[Q_{S},K_{S},V_{S}=W_{q}^{s}H_{S},W_{k}^{s}H_{S},W_{v}^{s}H_{S} \tag{3}\]  
where \(W((q,k,v))^{i}\) and \(W((q,k,v))^{s}\) represent trainable model parameters. Taking the intent recognition subtask as an example, a multi-head attention mechanism is employed to capture the interdependencies within the input text from multiple perspectives. Specifically, \(H_{I}\) is linearly transformed into multiple sets of vectors \(Q_{I}\), \(K_{I}\) and \(V_{I}\). Each set of vectors enables the model to focus on different local contextual information within the input text. The multi-head attention mechanism then integrates information from these different vector groups, using the scaled dot-product formula to calculate the attention scores between \(Q_{I}\) and \(K_{S}\), thereby obtaining context-aware attention vector representations, as shown in Equations (4) and (5). \[s(Q_{I},K_{S})=\frac{(K_{S}^{T}\cdot Q_{I})}{\sqrt{d}} \tag{4}\]  
\[H_{I}^{*}=LN(H_{I}+Softmax(s(Q_{I},K_{S}))\cdot V_{I}) \tag{5}\]  
where \(S(\cdot)\) denotes the intent attention function. It captures the importance vector representation of each character in the input text based on contextual information, allowing the model to leverage key local contextual features and enhance its intent recognition capability. After applying residual connections and layer normalization, the intent vector representation with mixed semantic information, \(H_{I}^{*}\), is obtained. A similar approach is used in the slot filling subtask to obtain the slot vector representation \(H_{S}^{*}\) with mixed semantic information. To more effectively improve the performance of the joint intent recognition and slot filling model, the Bi-XTM model extends the collaborative interaction attention mechanism by introducing a subtask interaction approach. This method transforms the traditional utterance-level intent encoding into a token-level intent encoding, specifically addressing the error propagation issues present in bidirectional interactions, thereby enhancing the explicit bidirectional interaction between intent and slot information. The bidirectional propagation layer architecture is shown in Figure 2.  
The subtask interaction approach not only provides intent label information from the intent output distribution to the slot filling task but also uses the slot output distribution to supply slot label information to the intent vector representation. Specifically, the subtask interaction method applies linear transformations to the intent and slot output distributions to compute the label distributions, thereby obtaining the probability outputs for each intent label and slot label.