# Domain Knowledge Graph Question Answering Based on Semantic Analysis and Data Augmentation
## 4 Question Answering System
### Semantic Parsing
#### 4.2.1 Question Intention Classification
The BiLSTM model comprises forward and backward LSTM cells, and therefore, the output of the BiLSTM at time \(t\) is jointly determined by \(x_{t},\overset{\rightarrow}{h_{t}}\), and \(\overset{\leftarrow}{h_{t}}\). The forward output of the LSTM cell at time \(t\) is represented by \(\overset{\rightarrow}{h_{t}}\), and the backward output at time \(t\) is denoted by \(\overset{\leftarrow}{h_{t}}\). The update formula of BiLSTM layer is described below:  
\[\overset{\rightarrow}{h_{t}}=\text{LSTM}\bigg{(}x_{t},\overset{ \rightarrow}{h_{t-1}}\bigg{)} \tag{1}\]  
\[\overset{\leftarrow}{h_{t}}=\text{LSTM}\bigg{(}x_{t},\overset{ \leftarrow}{h_{t-1}}\bigg{)} \tag{2}\]  
\[h_{t}=w_{t}\cdot\overset{\rightarrow}{h_{t}}+v_{t}\cdot\overset{\leftarrow} {h_{t}}+b_{t} \tag{3}\]  
In Equations (1)-(3), \(w_{t}\) and \(v_{t}\) represent the weight matrices of the forward and backward LSTM cells, respectively. \(b_{t}\) represents the bias, and \(h_{t}\) denotes the output of the BiLSTM layer at time \(t\). The input feature vector captures the global information of the text sentence through the BiLSTM layer, and the resulting output is represented by \(T=(T_{1},T_{2},\cdots,T_{n})\). (3) Multi-head attention layer  
The attention mechanism enables selectively focusing on critical information in text. We employed the multi-head self-attention mechanism (MHA) to extract word dependencies in different semantic spaces [41]. The MHA first conducts multiple linear transformations on the input vectors and then performs attention computation on the transformed vectors to obtain a weighted vector representation. By introducing multiple heads, the MHA can learn more semantic information and improve the model performance [42; 43]. The MHA is based on the principle of scaled dot-product attention (SDA), which can be expressed mathematically as follows:  
\[\text{Attention}(Q,K,V)=\text{softmax}\left(\frac{QK}{\sqrt{d_{k}}}^{T}\right)V \tag{4}\]  
In Equation (4), \(Q\), \(K\), and \(V\) represent the query, key, and value matrices used for computing the attention, respectively. The input key and value dimension is denoted by \(d_{k}\). The attention weight of a value is obtained by calculating the dot product of the query and all keys, dividing the result by \(\sqrt{d_{k}}\), and applying the function denoted by softmax. For an input vector sequence, multiple linear transformations are first applied to obtain multiple heads (i.e., multiple vector sequences). Then, the attention weight for each head is computed separately, followed by multiplication with the corresponding vectors and summation to obtain the output vector for each head:  
\[head_{i}=\text{Attention}\Big{(}QW_{i}^{Q},KW_{i}^{K},VW_{i}^{V}\Big{)} \tag{5}\]  
Equation (5) shows that the projection matrices \(W_{i}^{Q}\in\mathbb{R}^{d_{model}\times d_{k}}\), \(W_{i}^{K}\in\mathbb{R}^{d_{model}\times d_{k}}\) and \(W_{i}^{V}\in\mathbb{R}^{d_{model}\times d_{k}}\) correspond to \(Q\), \(K\), and \(V\) respectively. The single-headed output \(head_{i}\) is concatenated to form the final vector representation, and is then dimensionally adjusted by the projection matrix \(W^{O}\in\mathbb{R}^{hd_{k}\times d_{model}}\) to obtain the multi-headed output:  
\[\text{MultiHead}(Q,K,V)=\text{Concat}(head_{1},head_{2},\cdots,head_{n})W^{O} \tag{6}\]  
(4) Linear layer  
We concatenate the CLS vector \(X_{1}\) from the RoBERTa layer, the last sequential output \(T_{1}\) from the BiLSTM layer, and the pooled output \(H\) from the multi-head attention layer, normalize the result, and apply an activation function to obtain vector \(V_{T}\):  
\[V_{T}=\text{Tanh}(\text{Norm}(\text{Concat}(X_{1},T_{1},H))) \tag{7}\]  
The feature representation, which fuses the semantic information of the textual statements, is input to a fully connected layer. The resulting representation is mapped to the instance label space to obtain the final classification result:  
\[Y=\text{Linear}(V_{T}) \tag{8}\]