# Recall Mechanism and Multi-Head Attention for Numerical Reasoning
## 5 Experiment Results and Analysis  
### Overall Results  
Table 3 compares the performance of our models (RMNR, MANR, and RMMANR) with several baselines on the FINQA dataset. Notably, all three of our models outperform the existing baselines when using RoBERTa as the encoder. Specifically, RMMANR (RoBERTa-large) achieves the highest performance, surpassing all baselines by at least 2.67% in execution accuracy and 2.1% in program accuracy. Compared to FINQANet (RoBERTa-large) from the original FINQA paper, RMMANR improves execution accuracy by 6.58 points and program accuracy by 3.79 points.  
Both RMNR (RoBERTa-large) and MANR (RoBERTa-large) also demonstrate significant improvements over ELASTIC (RoBERTa-large). RMNR increases execution accuracy by 1.62 points and program accuracy by 1.4 points, while MANR improves execution accuracy by 1.25 points and program accuracy by 0.53 points. Given that RMNR, MANR, and ELASTIC use RoBERTa encoders of the same size, these results highlight the effectiveness of the recall mechanism and multi-head attention in enhancing numerical reasoning capabilities.  
Our models also outperform other baselines, such as NeRd, NumNet, and DyRRen. Unlike NeRd, which relies on external rules for operators [34], our models achieve superior  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Category** & **Models** & \multicolumn{2}{c}{**FINQA Results**} \\  & & **Exe Acc (\%)** & **Prog Acc (\%)** \\ \hline \multirow{4}{*}{Prompting-Model} & GPT-3.5-turbo [32]! & 48.56 & - \\  & GPT-4 [33]! & 68.79 & - \\  & Program-of-Thought [28]! & 68.10 & - \\ \hline \multirow{8}{*}{Fine-Tuning-Model} & NumNet [34] * & 2.32 & - \\  & NumNet+ [34] * & 10.29 & - \\  & NeRd [21] \(\ddagger\) & 52.48 & 49.90 \\  & FINQANet (RoBERTa-base) [2] \(\dagger\) & 60.10 & 58.38 \\  & FINQANet (RoBERTa-large) [2] \(\dagger\) & 65.05 & 63.52 \\  & ELASTIC (RoBERTa-base) [6] \(\dagger\) & 62.66 & 59.28 \\  & ELASTIC (RoBERTa-large) [6] \(\dagger\) & 68.96 & 65.21 \\  & DyRRen [35]! & 63.30 & 61.29 \\  & RMNR (RoBERTa-base) & 65.33 & 61.20 \\  & RMNR (RoBERTa-large) & 70.58 & 66.61 \\  & MANR (RoBERTa-base) & 64.98 & 61.12 \\  & MANR (RoBERTa-large) & 70.21 & 65.74 \\  & RMMANR (RoBERTa-large) & **71.63** & **67.31** \\ \hline \multirow{2}{*}{Human Performance} & Human Expert \(\dagger\) & 91.16 & 87.49 \\  & Human Non-Expert \(\dagger\) & 50.68 & 48.17 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Overall results of our models and the baselines on the test data. \(\dagger\) denotes that the results are sourced from the original paper. \(\ddagger\) denotes the results are sourced from [2]. * denotes that the results are sourced from [6]. \(\dagger\) denotes that the results are sourced from [39]. Bold values indicate the best performance for each metric.  
results without such dependencies. Additionally, NumNet and NumNet+ exhibit poor scalability due to limitations in their internal architectures [40], while DyRRen's retriever-reranker-generator framework is prone to cascading errors, resulting in lower performance.  
Furthermore, our models outperform all prompting models listed in Table 3, demonstrating stronger reasoning abilities compared to models like GPT-3.5-turbo and GPT-4. While RMNR, MANR, and RMMANR surpass non-expert human performance, there remains a significant gap between our models and expert human performance. This suggests that there is still considerable room for improvement on the FINQA dataset.