# Domain Knowledge Graph Question Answering Based on Semantic Analysis and Data Augmentation
## 5 Experiment and Analysis
### Question Intention Classification Experiment  
#### 5.2.1 Comparison Model  
In order to assess the efficacy of the RBMA model in intention classification tasks, this study conducted comparative experiments by comparing the RBMA model against several \begin{table}
\begin{tabular}{c c c} \hline \hline
**Step** & **Interpretation** & **Prompt** \\ \hline Capacity and Role & What role (or roles) should [PERSON] act as? & As a user who is going to handle household registration business \\ \hline Insight & Provides the behind the scenes insight, background, and context to your request. & You have questions about the handling information of some household registration matters \\ \hline Statement & What you are asking [PERSON] to do. & Please rewrite the given question, provide a similar question \\ \hline Personality & The style, personality, or manner you want [PERSON] to respond in. & Ensure that the original meaning of the sentence is preserved. The rewriting method includes synonym substitution and random insertion of irrelevant words \\ \hline Experiment & Asking [PERSON] to provide multiple examples to you. & The sentences that need to be rewritten are: \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Prompts Creation Framework. prevailing text classification models. Furthermore, we conducted ablation experiments to examine the classification performance of RoBERTa, BiLSTM, and MultiHeadAttention. Specifically, we compared the RBMA model against seven other models:  
* DPCNN [58]: A deep network-based classification model that extracts text dependency features over long distances;
* TextRCNN [59]: A bidirectional RNN-based model that leverages context information through max-pooling to extract important features for text classification;
* BERT: A model that leverages the BERT architecture to parse the semantics of a sentence and obtain corresponding sentence embeddings for text classification tasks;
* BERT-BiLSTM-MultiHeadAttention: A model that employs BERT to obtain sentence embeddings, BiLSTM to extract contextual information, and MHA to consider multiple aspects of the sentence and perform text classification based on the combination of all information;
* RoBERTa: A model that utilizes RoBERTa pretrained models to obtain semantic embeddings of sentences for text classification tasks;
* RoBERTa-BiLSTM: A model that combines RoBERTa's semantic parsing with BiLSTM's contextual information for text classification;
* RoBERTa-MultiHeadAttention: A model that incorporates MHA with RoBERTa's semantic embeddings to consider multiple aspects of the sentence and performs text classification based on the combination of all information. For each category's prediction, we calculated TP, TN, FN, and FP, which correspond to the number of true positive, true negative, false negative, and false positive, respectively. We evaluated the performance of the question matching model using three metrics: precision, recall, and F1 score. The formulas for these metrics are shown below:  
\[\text{precision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{13}\]  
\[\text{recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{14}\]  
\[\text{f1}-\text{score}=\frac{2\times\text{precision}\times\text{recall}}{ \text{precision}+\text{recall}} \tag{15}\]  
To ensure optimal generalization abilities of the model during the training process, we saved the model parameter files when the performance of each model on the validation set was at its best, with the premise that the model was not overfitting. Then, the accuracy of the final answer when the QA system used the model for intention classification was tested on the test set. The formula for accuracy calculation is presented below:  
\[\text{accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{FN}+\text{FP}+ \text{TN}} \tag{16}\]  
The hyperparameter settings for the model experiment are shown in Table 5.