# Recall Mechanism and Multi-Head Attention for Numerical Reasoning
## 3 Methods  
Deep neural networks often suffer from the gradual loss of embedding vectors for questions and passages as the network depth increases. To address this issue, we propose a recall mechanism for numerical reasoning, designed to mitigate the loss of embedding vectors for questions and passages as network depth increases. In memory and cognitive psychology, a recall mechanism refers to the process by which the brain retrieves stored information from long-term memory. This mechanism allows individuals to access past experiences, facts, or learned skills. Inspired by the human tendency to review relevant information before making a decision, RM incorporates a recall module that revisits question and passage embeddings before generating operators and operands. Unlike most models, where the Reasoning Module directly passes the guidance vectors--obtained from embedding vectors of questions and passages--to the Generator Module for operator and operand generation, RM introduces an additional recall step. Specifically, as shown in Figure 3, the Recall Module is placed between the Reasoning Module and the Generator Module, enabling second access to question and passage embeddings. This additional retrieval step ensures that the guidance vectors contain richer contextual information, leading to more accurate operator and operand generation. By integrating this mechanism, RM prevents the degradation of initial input embeddings, enhancing numerical reasoning accuracy. On the other hand, many existing models struggle with multi-step reasoning tasks [2,6]. To address this challenge, we propose multi-head attention for numerical reasoning. As shown in Figure 4, the MA Module consists of a multi-head attention rea \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Category** & **Model** & **Math23K** & \multicolumn{2}{c}{**DROP**} & \multicolumn{2}{c}{**FINQA**} & \multicolumn{2}{c}{**TAT QA**} \\  & & **Prog Acc** & **Exu Acc** & **F1** & **Exu Acc** & **Prog Acc** & **Exu Acc** & **F1** \\ \hline  & Human Expert [2] & - & - & - & **91.16** & **87.49** & - & - \\ \hline FF & MHST [23] & - & - & - & - & - & 63.6 & 72.7 \\  & FinMath [26] & - & - & - & - & - & 58.3 & 68.2 \\ \hline RNN & DNS [10] & 64.7 & - & - & - & - & - & - \\  & NeRd [21] & **78.55** & **81.85** & 52.48 & 49.9 & - & - & - \\  & ELASTIC [6] & - & - & - & **68.96** & **65.21** & - & - \\ \hline Transformer & Ensemble [27] & 66.7 & - & - & - & - & - & - \\ \hline Graph & ODGAT [22] & - & 64.56 & **67.97** & 13.1 & - & 39.1 & 49.7 \\  & SoarGraph [7] & - & - & - & 67.2 & - & **65.4** & **75.3** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance of deep neural network approaches in different numerical reasoning datasets. Exe Acc (execution accuracy) measures whether the executed result of the generated program matches the ground truth answer. Prog Acc (program accuracy) evaluates whether the generated program is mathematically equivalent to the ground truth program. The F1 Score assesses the overlap between the predicted and gold answers using a bag-of-words representation. Bold values indicate the best performance for each metric within a given dataset. soning module and a Generator Module. Multi-head attention enables the model to focus on different parts of the input sequence simultaneously, facilitating parallel processing of multiple reasoning steps. This capability is particularly beneficial for complex numerical reasoning tasks, as it efficiently extracts relevant information fragments and optimizes the solution process for multi-step problems. RMMANR combines the strengths of the RM and MA Modules. As illustrated in Figure 5, the question and passage embeddings are first processed by the RM Module, which generates initial operators \(OP_{RM}\) and operands \(OE_{RM}\). The Selector then calculates \(stepNum\) based on \(OP_{RM}\). If \(stepNum=1\), the Selector passes \(OP_{RM}\) and \(OE_{RM}\) to the Program Solver.