## 6 Conclusions and Future Work  
In this work, we address the critical challenge of triple-flip errors in SPARQL query building tasks, a pervasive issue affecting the performance of Knowledge Base Question Answering (KBQA) systems. We introduce a novel approach that leverages a new objective called Triple Structure Correction (TSC) to enrich the pre-training of the T5 model. Our work establishes a comprehensive framework to bridge the \"semantic gap\" between human language and machine-readable queries. The innovative Triple Structure Correction objective represents a key contribution in providing a practical solution to the challenges encountered in generating SPARQL queries. As a further testament to our methodology, we fine-tuned our model, which we named TSET, for the downstream task of SPARQL query building. Our experimental results validate the effectiveness of our approach, showing that TSET substantially outperforms state-of-the-art methods on three renowned KBQA datasets: LC-QuAD 2.0, QALD-9 plus, and QALD-10. Overall, this study paves the way for future research in KBQA, underscoring the pivotal roles of pre-training and fine-tuning in improving both the understanding and generation of complex SPARQL queries. In the future, two prominent research directions beckon our attention. Firstly, we aspire to contemplate a more diverse array of neural network models and more complex knowledge bases. While the current study predominantly focuses on the T5 model-based KBQA system, advancements in neural network technology such as ChatGPT and other LLMs may also lend valuable insights into enhancing KBQA. Additionally, with the continuous enrichment and expansion of knowledge bases, fine-tuning and optimizing our model to accommodate larger and structurally complex knowledge bases remains a challenge. Secondly, we aim to delve deeper into pre-training and fine-tuning strategies to better leverage the structural information in knowledge bases and to devise more effective objective functions for improving model performance in understanding and generating \begin{table}
\begin{tabular}{l l c c c c c} \hline \hline
**Dataset** & **Size** & **Approach** & **Total** & **Sentence** & **Triple** & **Triple Flip** \\ \hline \multirow{3}{*}{LC-QuAD 2.0} & \multirow{3}{*}{Small} & TS & 619 & 99 & 582 & 348 \\  & & TSET & 511 (\(-\)17.4) & 92 (\(-\)7.1) & 481 (\(-\)17.3) & 254 (\(-\)27.0) \\ \cline{2-7}  & & TS & 560 & 95 & 522 & 313 \\  & & TSET & 449 (\(-\)19.8) & 78 (\(-\)17.9) & 414 (\(-\)20.7) & 223 (\(-\)28.7) \\ \hline \multirow{3}{*}{QALD-9 plus} & \multirow{3}{*}{Small} & TS & 65 & 54 & 58 & 8 \\  & & TSET & 56 (\(-\)13.8) & 48 (\(-\)11.1) & 51 (\(-\)12.1) & 7 (\(-\)12.5) \\ \cline{2-7}  & & TS & 61 & 48 & 55 & 8 \\  & & TSET & 54 (\(-\)11.5) & 47 (\(-\)2.1) & 49 (\(-\)10.9) & 7 (\(-\)12.5) \\ \hline \multirow{3}{*}{QALD-10} & \multirow{3}{*}{Small} & TS & 271 & 233 & 247 & 24 \\  & & TSET & 245 (\(-\)9.6) & 220 (\(-\)5.6) & 227 (\(-\)8.1) & 14 (\(-\)41.6) \\ \cline{1-1} \cline{2-7}  & & TS & 255 & 225 & 233 & 21 \\ \cline{1-1}  & & TSET & 232 (\(-\)9.0) & 197 (\(-\)12.4) & 215 (\(-\)7.7) & 13 (\(-\)38.0) \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Error analysis conducted on the test set across three datasets, illustrating the count of instances for various error types. The values in parentheses represent the relative reduction in these errors. SPARQL queries. We believe these future efforts will propel our model toward achieving greater success in handling intricate KBQA tasks. Conceptualization, J.Q. and L.F.; Methodology, J.Q. and L.F.; Software, J.Q., C.S., and L.W. ; Formal analysis, J.Q. and L.F.; Investigation, C.S. and Z.G. ; Validation, C.S., Z.G., Z.S. and L.W. ; Writing--Original draft preparation, J.Q. ; Writing--Reviewing and Editing, L.F.; Supervision, X.W. ; Resources, C.Z. ; Funding acquisition, L.F.; Project administration, L.F. All authors have read and agreed to the published version of the manuscript. This work was supported by NSF China (No.