# A DeBERTa-Based Semantic Conversion Model for Spatiotemporal Questions in Natural Language
## 5 Experiment and Discussion
### Error Analysis  
The NL2 Cypher model, based on the DeBERTa semantic conversion of natural language spatiotemporal queries, adheres to the semantics of Cypher, removing non-executable parts of Cypher queries to achieve more accurate results. This involves handling scenarios where the generated Cypher statements contain syntactic errors, mismatches between functions and target field names, and instances where query results return null values. This mechanism can be used with any autoregressive model to improve the accuracy of results. Consequently, the final generated Cypher statements were compared with standard Cypher statements by randomly selecting 100 samples where logical forms did not match for analysis. These were divided into two categories:  
(1) \"Unanswerable\" cases (20 instances) are situations where the Cypher statements cannot generate correct query results from the given data. They primarily include the following two types: Type 1, insufficient semantic information recognition, is where complex place names, abbreviations, and spatiotemporal relationships are not accurately recognized, such as \"Xing'an League A'ershan City Hetu Ala Town\", \"Peking University\", and \"adjacent\". Type 2, limitations in question type, are described as follows: The ques \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Dataset Classify** & \multicolumn{3}{c}{**Precision of the Training Set**} & \multicolumn{3}{c}{**Precision of the Test Set**} \\ \hline \multirow{3}{*}{GeoQuestions} & \(A_{CC_{gm}}\) & \(A_{CC_{f}}\) & \(A_{CC_{a}}\) & \(A_{CC_{gm}}\) & \(A_{CC_{f}}\) & \(A_{CC_{a}}\) \\  & 0.8825 & 0.8371 & 1.054 & 0.9226 & 0.8863 & 1.041 \\ \hline \multirow{3}{*}{GeoQuery} & \(A_{CC_{gm}}\) & \(A_{CC_{f}}\) & \(A_{CC_{a}}\) & \(A_{CC_{gm}}\) & \(A_{CC_{f}}\) & \(A_{CC_{a}}\) \\  & 0.8837 & 0.8321 & 1.062 & 0.9282 & 0.8833 & 1.051 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Accuracy inspection of NL2 Cypher. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Model** & \multicolumn{3}{c}{**GeoQuestions**} & \multicolumn{6}{c}{**GeoQuery**} \\ \hline \hline  & \multicolumn{3}{c}{**Training Set**} & \multicolumn{3}{c}{**Test Set**} & \multicolumn{3}{c}{**Training Set**} & \multicolumn{3}{c}{**Test Set**} \\  & \(A_{CC_{gm}}\) & \(A_{CC_{f}}\) & \(A_{CC_{gm}}\) & \(A_{CC_{f}}\) & \(A_{CC_{gm}}\) & \(A_{CC_{f}}\) & \(A_{CC_{gm}}\) & \(A_{CC_{f}}\) \\ \hline NL2 Cypher & 88.3 & 83.7 & 92.3 & 88.6 & 88.4 & 83.2 & 92.8 & 88.3 \\ GPT2 (standard) & 84.0 & 76.0 & 83.8 & 75.5 & 81.2 & 79.3 & 82.5 & 81.7 \\ BaiChuan & 65.8 & 59.5 & 59.4 & 55.9 & 76.0 & 70.4 & 72.5 & 71.6 \\ QianWen & 57.9 & 52.4 & 55.8 & 49.5 & 52.3 & 48.2 & 61.3 & 56.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: The accuracy evaluation results obtained for these two datasets by different models, %. tion classification framework proposed in this paper includes the most commonly used basic and compound queries in the geographic spatiotemporal field but also has limitations, resulting in a few questions that cannot be classified, leading to incorrect results. For example, the question \"What is the length of the river that runs through London?\" is an attribute query related to spatial association, which is not adequately considered in this classification framework. The existing framework is limited in capturing implicit semantic relationships and complex contextual information, leading to incomplete or inaccurate conversion results. (2) For \"Answerable\" cases (80 instances), in analyzing the remaining 80 Cypher statements that were answerable, it was found that 35 of them had logical form errors. Further analysis of these 35 Cypher statements revealed that 33 had errors when generating functions, for example, overlooking \"max\" or \"min\" during comparison, and 2 contained errors in constructing the WHERE clause, such as confusing \"feature code\" when generating the clause for \"What cities in ChaoYang have the highest populations?\", which led to confusion between Chaoyang in Beijing and Chaoyang in Liaoning.