# A Knowledge and Semantic Fusion Method for Automatic Geometry Problem Understanding
## 6 Experiment and Application
### Experiment and Analysis
#### 6.1.2 Application Verification Test and Error Analysis  
Dataset (B): This dataset includes problems with complex relationship descriptions and contains entity types, relationship types, and knowledge attributes not covered during method construction (e.g., unhandled entity/relationship types, unconfigured entity descriptions, or missing RSF configurations for certain relationship types). Statistical details on problem characteristics and RMT distribution are provided in Tables 10 and 11. The higher proportion of SI and MC series RMTs indicates that Dataset B contains more problems with complex linguistic expressions. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Statistical Item** & **Total** & **Average** & **Proportion** \\ \hline Total Problems & 230 & - & - \\ Total Clauses & 1769 & 7.69 & 1 \\ No Relationship Clauses & 307 & 1.33 & 0.17 \\ Relationship Clauses & 1462 & 6.36 & 0.83 \\ \hline Unary Geo.Relationship & 182 & 0.79 & 0.10 \\ Binary Geo.Relationship & 1027 & 4.47 & 0.56 \\ Ternary Geo.Relationship & 534 & 2.32 & 0.29 \\ Numerical Relationship & 89 & 0.39 & 0.05 \\ Total Relationships & 1832 & 7.97 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Statistics on the problem information and relationship distribution of Dataset B. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Statistical Item** & **Total** & **Average** & **Proportion** \\ \hline Total Problems & 130 & - & - \\ Total Clauses & 978 & 7.52 & 1 \\ No Relationship Clauses & 226 & 1.74 & 0.23 \\ Relationship Clauses & 752 & 5.78 & 0.76 \\ \hline Unary Geo.Relationship & 125 & 0.96 & 0.12 \\ Binary Geo.Relationship & 604 & 4.65 & 0.58 \\ Ternary Geo.Relationship & 260 & 2.00 & 0.25 \\ Numerical Relationship & 52 & 0.40 & 0.05 \\ Total Relationships & 1041 & 8.01 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Statistics on problems information and relationship distribution of Dataset A. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Statistical Item** & **Total** & **Average** & **Proportion** \\ \hline Total Problems & 130 & - & - \\ Total Clauses & 978 & 7.52 & 1 \\ No Relationship Clauses & 226 & 1.74 & 0.23 \\ Relationship Clauses & 752 & 5.78 & 0.76 \\ \hline Unary Geo.Relationship & 125 & 0.96 & 0.12 \\ Binary Geo.Relationship & 604 & 4.65 & 0.58 \\ Ternary Geo.Relationship & 260 & 2.00 & 0.25 \\ Numerical Relationship & 52 & 0.40 & 0.05 \\ Total Relationships & 1041 & 8.01 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Statistics of the Relationship Mention Type distribution of Dataset A. (1) Problem Understanding Evaluation  
After testing a total of 230 problems, 209 geometry problems were successfully understood, resulting in a success rate of 91.87%. From the perspective of relationships, all processed entity and relationship types were correctly identified, even in failed problems. The effectiveness of problem understanding met expectations, thereby validating the performance of the method. (2) Text-Level ERE Evaluation and Analysis  
Baseline Methods: Four methods were compared, all built/trained on Dataset A and tested on Dataset B:  
* STM: A pure template-based method using 177 predefined sentence-level templates for entity-relation extraction (Representative method [50; 51]). * TBC: A method that uses Bi-LSTM-CRF for entity extraction and sentence template-based relationship extraction (representative method [36]). It built a total of 153 templates for both atomic and composite relationship expressions across three levels--word, sentence, and clause--to normalize the problem text. * BBC: A machine-learning-based method that uses Bi-LSTM-CRF for entity extraction and Bert-CasRel for relationship extraction (representative method [62]). * S2-Like: Integrates the POS tagging (applies ICTCLAS) and hybrid rul/template (S2 model and sentence template) approaches for relationship extraction (referring to method [52]). The sentence templates used for restoring complex relationships are identical to those used in STM for non-simple expressions, with a total of 116 templates.