## 3 Neural Network Models for Arithmetic Reasoning
### Generic Large Language Models
#### 3.3.1 Promoting Step-by-Step Numerical Reasoning  
A promising approach to improve the numerical skills of LLMs involves the use of advanced prompting strategies (a.k.a. \"in-context learning\"), which allow the model's behavior to be shaped by providing a few input-output exemplars demonstrating the task, without actually tuning any model parameter [69]. A recent line of study has shown that prompting strategies can be improved using \"scratchpads\" [79] and \"chain-of-thought reasoning\" [80], which allow accuracy to be significantly increased on multi-step mathematical reasoning problems. The idea is that rather than having to generate a final answer immediately, the model can first generate solutions that may contain intermediate computations (see Table 2). To achieve this, the scratchpad technique introduced by [79] allows the model to produce an arbitrary sequence of intermediate tokens that are stored in a buffer and that can be further processed before producing the final answer. The authors considered the task of learning long integer addition, showing that the use of scratchpads improved the calculation accuracy in the extrapolation interval. This idea was taken further by [80]: the \"chain-of-thought\" is a series of intermediate natural language reasoning steps that are provided as an input to the model during prompting to explicitly demonstrate how to produce the final output. This technique was tested in arithmetic reasoning tasks, achieving striking performance gains compared to baseline models. Interestingly, it turns out that chain-of-thought prompting has larger performance gains for more complicated problems, mostly when it is applied to larger-scale models. At the time of publication, it established a new state-of-the-art performance on the challenging GSM8K and SVAMP benchmarks. Similarly to what happens in recurrent models with an adaptive computation time [81, 82], these advanced prompting techniques allow the neural network to process the input \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**A** & **Problem:** & 29 + 57 \\  & & 2 9 + 5 7, C: 0 \\  & & 2 + 5, 6 C: 1 \# added 9 + 7 = 6 carry 1 \\  & & 8 6 C: 0 \# added 2 + 5 + 1 = 8 carry 0 \\  & & 0 8 6 \\  & **Answer:** & 86 \\ \hline \hline
**B** & **Problem:** & If there are 3 cars in the parking lot and 2 more cars arrive, how many cars are in the parking lot? \\  & **Chain-of-thought:** & There are originally 3 cars. 2 more cars arrive. 3 + 2 = 5. \\  & **Answer:** & The answer is 5. \\  & & There were nine computers in the server room. Five more computers were installed each day, from Monday to Thursday. How many computers are now in the server room? \\  & & There were originally 9 computers. For each of 4 days, 5 more computers were added. So 5 \({}^{*}\) 4 = 20 computers were added. 9 + 20 is 29. \\
**Answer:** & The answer is 29. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Examples of numerical problems solved using advanced prompting strategies. (**A**) The scratchpad method forces the model to explicitly produce intermediate computation steps, which are iteratively refined to generate the final answer [79]. (**B**) A subset of chain-of-thought prompting examples that were given as input to the model to elicit step-by-step reasoning during problem solution [80]. information for as long as is needed, depending on the complexity of the current problem. By encouraging the model to produce an explanation along with the answer, we also steer it towards solving problems by breaking them into smaller steps that logically follow from each other. Furthermore, the buffer allows intermediate information to be stored for an arbitrary amount of processing steps, removing the need to memorize all intermediate states in the network's activations. Last, but not least, this approach provides an interpretable window into the behavior of the model, suggesting how it might have arrived at a particular answer and providing opportunities to debug where the reasoning path went wrong.