## 3 Neural Network Models for Arithmetic Reasoning  
### Ad Hoc Architectures  
Several frameworks have been proposed to create \"neuro-symbolic\" systems that combine neural networks with rule-based numerical solvers or to design neural network modules specifically tailored for numerical reasoning. One of the earliest attempts to solve math word problems using a neuro-symbolic approach consisted of a vanilla sequence-to-sequence model combined with a similarity-based retrieval model, with the goal of creating a hybrid MWP solver [32]. Subsequent approaches exploited graph-based problem representations, which facilitate the solution of MWP through the use of structured descriptions reflecting a tree-like algorithmic decomposition of the problem [52; 53] or explicitly capturing the relationship between quantities and their attributes [54]. Another ad hoc architecture incorporates numerical reasoning into the system by exploiting a \"numerically aware\" graph neural network [55]. The model is composed by standard neural encoding \begin{table}
\begin{tabular}{p{14.2 pt} p{142.3 pt} p{142.3 pt}} \hline \hline
**A** & **Problem:** & [PERSON] purchased 7 kg of grapes at the rate of 70 per kg and 9 kg of mangoes at the rate of 55 per kg. How much did he pay to the shopkeeper? & \\  & **Operation stack:** & a = multiply (7, 70); b = multiply (9, 55); add (a,b) & \\  & **Answer:** & 985 & \\ \hline
**B** & **Problem:** & [PERSON] gave 50 dollars to the shopkeeper to buy a book and the shopkeeper returned some money. What could be the price of the book? Choose all plausible alternatives. & \\  & **Answers:** & (a) 55 (b) 45 (c) 37 (d) 59 (e) None & \\ \hline
**C** & **Problem:** & Write the Arabic number corresponding to the following number words:“three hundred and thirty-four”one thousand two hundred and seven”’s eighty-nine thousand and one” & \\  & **Answer:** & 334; 1207; 89001 & \\ \hline
**D** & **Problem:** & Divide 1136975704 by \(-\)142121963. & \(-\)8 \\  & **Problem:** & Let k(u) = u * 2 + u-4. Find k(0). & \\  & **Answer:** & \(-\)4 & \\ \hline
**E** & **Context:** & Before the UNPROFOR fully deployed, the HV dashed with an armed force of the RSK in the village of Nos Kalik, located in a pink zone near Sibenik, and captured the village at 4:45 p.m. on 2 March 1992. The JNA formed a battlegroup to counterattack the next day. & \\  & **Question:** & What date did the JNA form a battlegroup to counterattack after the village of Nos Kalik was captured? & \\  & **Answer:** & 3 March 1992 & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Examples of problems from representative numerical reasoning datasets. (**A**) MathQA [34]. (**B**) FeasibilityQA [38]. (**C**) Language to number translation task [16]. (**D**) Mathematics [43]. (**E**) NumGLUE [46]. and decoding modules, but also includes a \"reasoning module\" that represents the MWP quantities using a graph, where the nodes correspond to the numbers appearing in the text and the edges encode magnitude relationships (i.e., \"greater than\"). However, since the graph is pre-defined for each problem, such a model cannot deal with tasks entailing the generation of intermediate numerical results and has limited arithmetic reasoning abilities. Overall, we can argue that models incorporating ad hoc graph-based representations generally improve over standard seq2 seq architectures, but their performance is still poor when tested using more controlled (though relatively simple) datasets [35,36]. A modular design was also adopted by a recent neuro-symbolic model developed to tackle reading comprehension problems involving numerical reasoning [44]. In this architecture, a transformer-based module first predicts whether the problem answer is a count or an arithmetic expression, and then identifies the specific numbers involved in the expression. Once a proper arithmetic expression has been formed, it is given as an input to a symbolic solver to produce the final answer.