# Recall Mechanism and Multi-Head Attention for Numerical Reasoning
## 6 Conclusions and Future Work  
This paper introduces the recall mechanism and multi-head attention for numerical reasoning (RMMANR). RMMANR consists of two main components: an encoder and a decoder. The encoder leverages RoBERTa to encode the question and passage into contextual embeddings. The decoder, which consists of four modules (RM Module, Selector, MA Module, and Program Solver), generates inference steps and answers based on contextual information from the encoder. The recall mechanism addresses the issue of information loss in question and passage embeddings as the model depth increases, ensuring that critical details are retained throughout the reasoning process. On the other hand, multi-head attention proves particularly effective for complex numerical reasoning tasks, as it enables the extraction of multiple relevant information fragments and optimizes the solution process for multi-step problems.  
RMMNR, MANR, and RMMANR were evaluated on the FINQA dataset. Experimental results demonstrate that all three models outperform several baselines, with RMMANR achieving the highest performance by effectively combining the strengths of RMM and MANR.  
Figure 14: The accuracy of MANR with different numbers of heads on the entire FINQA. Each model employs RoBERTa-large for encoding.  
Figure 13: The accuracy of MANR with a different number of heads on multi-step reasoning problems within FINQA. Each model employs RoBERTa-large for encoding.  
For future work, we plan to focus on enhancing the model's performance on more complex multi-step reasoning tasks. This includes addressing challenges such as data scarcity for longer program steps and further optimizing the recall mechanism to reduce noise interference during operand prediction. Additionally, exploring alternative architectures or training strategies to improve generalization on complex reasoning tasks will be a key direction.  
Conceptualization, L.L.; methodology, L.L.; software, L.L. and B.Z.; validation, L.L. and B.Z.; formal analysis, L.L.; investigation, L.L.; resources, L.L.; data curation, L.L.; writing--original draft preparation, L.L.; writing--review and editing, T.-P.T.; visualization, L.L.; supervision, T.-P.T.; project administration, L.L.; funding acquisition, L.L. All authors have read and agreed to the published version of the manuscript.  
This research was supported by Puitian University (grant No. 2022048), Puitian Science and Technology Bureau (Putian Science and Technology Plan Project 2023 GJGZ003), and Puitian University (grant No. JG202388).  
Not applicable.  
The dataset is available at [[https://github.com/NeuraSearch/NeurIPS-2022-Submission-3358](https://github.com/NeuraSearch/NeurIPS-2022-Submission-3358)]([https://github.com/NeuraSearch/NeurIPS-2022-Submission-3358](https://github.com/NeuraSearch/NeurIPS-2022-Submission-3358)) (accessed on 15 July 2022) [6].  
The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.  
The following abbreviations are used in this manuscript:  
RMMANR Recall Mechanism and Multi-head Attention for Numerical Reasoning  
RMNR Recall Mechanism for Numerical Reasoning  
MANR Multi-head Attention for Numerical Reasoning