## 4 Experiments
### Case Study  
To assess the reliability of the algorithm's output and analyze the causes of solving errors, our experiment sampled typical cases of varying difficulty levels from datasets. Specifically, the output from **ZhangAlg** and **LiuAlg** was compared to **PROP** algorithm. Among them, **ZhangAlg** outputs the arithmetic expression and the answer, **LiuAlg** provides a natural language description of the reasoning process and the answer, and **PROP** outputs the relation sets and the answer. The details of the case study are presented in Table 6.  
From Table 6, the following can be seen. For Case 1, all algorithms produce the correct answer. This is because it is a simple AWP that only contains a single \"general\" scene, requiring only the information stated in the problem for the solution. For Case 2, **ZhangAlg** outputs an incorrect answer because it failed to obtain external knowledge from the limited training set, leading to the erroneous inclusion of the rice quantity \"20 kg\" in the fruit quantity calculation \"(15 + 20 + 20) kg\". In contrast, both **LiuAlg** and **PROP** acquire external knowledge through LLMs, handling this problem effectively and obtaining the correct answer. For Case 3, both **ZhangAlg** and **LiuAlg** produce incorrect answers. **ZhangAlg** fails due to poor learning of long expressions, while for **LiuAlg**, errors arise from generating invalid equations such as \"(5/9) * N/N = (2/5)\" during excessive reasoning steps. **PROP**, using a relation-flow method, decomposes the problem into three \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline  & \multicolumn{6}{c}{**Algorithms Tested on Chinese Version**} & \multicolumn{6}{c}{**Algorithms Tested on English Version**} \\ \cline{2-11}  & & **YuAlg** & **ZhangAlg** & **JieAlg** & **LiuAlg** & **PROP** & **YuAlg** & **JieAlg** & **YasuAlg** & **LiuAlg** & **PROP** \\ \hline
**Dataset** & **\#** & **\%** & **\%** & **\%** & **\%** & **\%** & **\%** & **\%** & **\%** & **\%** & **\%** \\ \hline PBE & 3262 & 55.2 & 49.2 & 52.8 & 85.2 & 89.5 & 54.0 & 49.0 & 75.7 & 82.8 & 88.2 \\ Math23a & 02,160 & 60.0 & 77.8 & 87.8 & 90.1 & 91.2 & 58.2 & 84.3 & 88.0 & 90.2 & 91.0 \\ APECa & 15,245 & 58.1 & 72.6 & 81.9 & 89.4 & 90.0 & 52.3 & 80.5 & 85.5 & 89.9 & 89.6 \\ AGG & 1492 & 54.2 & 66.7 & 73.4 & 86.2 & 88.2 & 55.3 & 72.5 & 85.1 & 90.4 & 90.9 \\ MAWPS & 1987 & 69.7 & 84.3 & 91.5 & 94.6 & 94.2 & 71.2 & 92.0 & 94.4 & 95.3 & 95.7 \\ GSM8K & 1492 & 35.0 & 38.5 & 43.7 & 81.3 & 83.2 & 37.8 & 46.5 & 77.8 & 83.2 & 84.5 \\ \hline Avg & 43,465 & 58.5 & 72.6 & 81.4 & 89.3 & **90.4** & 55.7 & 79.1 & 86.1 & 89.6 & **90.3** \\ \hline \hline \end{tabular}  
* */* refers to the number of problems in dataset.  
\end{table} 
Table 4: Comprehensive experiment results on solving accuracy. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Dataset** & **LiuAlg** & **PROP** & **LiuAlg** & **PROP** \\ \hline PBE & 85.2 & 89.5 & 88.9 & 90.3 \\ AGG & 86.2 & 88.2 & 94.8 & 95.1 \\ GSM8K & 81.3 & 83.2 & 91.3 & 93.6 \\ \hline Avg & 84.6 & 87.8 & 90.9 & 92.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison results between PROP and LiuAlg. different scenes, forming three relational sets. It achieves the correct answer through precise reasoning among these relations. The first column is the title of the content in the second column. In the second column, the bold text represents the name of the algorithm being compared, the red text indicates incorrect answers output by the algorithm, and the green text indicates correct answers output by the algorithm. \begin{table}
\begin{tabular}{p{128.0 pt} p{284.5 pt}} \hline \hline
**AWP case 1** & [PERSON] has 15 stamps, and [PERSON] has 5 stamps less than [PERSON]. How many stamps does [PERSON] have? \\ \hline \multirow{7}{*}{**Output**} & **ZhangAlg:** \\  & Equation: \({}^{\text{`1}}5-5^{\prime}\), Answer: 10 Correct \\  & **LiuAlg:** \\  & 1. Let [PERSON] have xx stamps; 2. According to the question, [PERSON] has 15 stamps, that is, \(\chi=15;3\).