###### Abstract  
Creating learning models that can exhibit sophisticated reasoning abilities is one of the greatest challenges in deep learning research, and mathematics is rapidly becoming one of the target domains for assessing scientific progress in this direction. In the past few years there has been an explosion of neural network architectures, datasets, and benchmarks specifically designed to tackle mathematical problems, reporting impressive achievements in disparate fields such as automated theorem proving, numerical integration, and the discovery of new conjectures or matrix multiplication algorithms. However, despite this notable success it is still unclear whether deep learning models possess an elementary understanding of quantities and numbers. This survey critically examines the recent literature, concluding that even state-of-the-art architectures and large language models often fall short when probed with relatively simple tasks designed to test basic numerical and arithmetic knowledge. artificial intelligence; neuro-symbolic systems; deep learning; large language models; number embeddings; numerical cognition; mathematical reasoning; arithmetic +
Footnote †: journal:  
Despite these challenges, the recent successes of deep learning have rekindled interest in the idea that neural networks might be able to acquire high-level reasoning abilities and thus exhibit symbolic behavior [15]. Indeed, although deep networks struggle to grasp even basic concepts such as the meaning of an \"integer number\" [16], in the past few years, several models have demonstrated impressive capabilities in solving complicated mathematical tasks. For example, sequence-to-sequence architectures were shown able to learn to perform function integration and solve ordinary differential equations, sometimes with greater accuracy than popular math software packages [17]. Deep learning models have also been successfully used for automated theorem proving [18; 19; 20] or to assist expert mathematicians to formulate conjectures and establish new fundamental results in pure mathematics [21]. Very recently, deep reinforcement learning discovered a more efficient algorithm to perform matrix multiplication [22], while fine-tuning a pre-trained large language model on computer code allowed to solve university-level math questions at a human level [23]. These stunning achievements partially stem from the introduction of well-curated, large-scale datasets containing mathematical problems annotated with the corresponding solutions, but also from the design of novel (often ad hoc) architectures that can more effectively process numerical symbols and mathematical notation. Improvements in many tasks have also been enabled by the creation of large language models, which exhibit surprising \"out of the box\" numerical skills that can be further refined through fine-tuning and prompting strategies. However, as we will highlight in the present survey, these findings do not imply that such models fully master the semantics of numbers and basic arithmetic. In fact, their performance on relatively simple numerical tasks is often brittle, and their understanding of symbolic math problems may be much more shallow than it seems [24; 25]. This suggests that we might need to improve the elementary skills of these models to bootstrap more reliable mathematical abilities. This hypothesis is also supported by the extended literature on child development and education, which has shown that performance in basic numeracy tasks such as counting, quantity comparison, understanding of number ordering, and the base-ten positional numeral system is a strong predictor of later mathematics achievement [26; 27; 28].