Limitations of Language Models in Arithmetic and Symbolic Induction (2023)

Abstract: Recent work has shown that large pretrained Language Models (LMs) can not only perform remarkably well on a range of Natural Language Processing (NLP) tasks but also start improving on reasoning tasks such as arithmetic induction, symbolic manipulation, and commonsense reasoning with increasing size of models. However, it is still unclear what the underlying capabilities of these LMs are. Surprisingly, we find that these models have limitations on certain basic symbolic manipulation tasks such as copy, reverse, and addition. When the total number of symbols or repeating symbols increases, the model performance drops quickly. We investigate the potential causes behind this phenomenon and examine a set of possible methods, including explicit positional markers, fine-grained computation steps, and LMs with callable programs. Experimental results show that none of these techniques can solve the simplest addition induction problem completely. In the end, we introduce LMs with tutor, which demonstrates every single step of teaching. LMs with tutor is able to deliver 100% accuracy in situations of OOD and repeating symbols, shedding new insights on the boundary of large LMs in induction.

## introduction transformer-based large pretrained language models, such as gpt3 and t5 @xcite @xcite , have been widely used as few-shot learners in many nlp tasks. recent work even finds these models can achieve state-of-the-art performance in arithmetic and symbolic reasoning @xcite . although these models exhibit surprisingly impressive capabilities in complex arithmetic reasoning tasks, such as multiarith @xcite and gsm8k @xcite , it has also figure 1 : examples of addition: the baseline setting (top) and scratchpad @xcite with intermediate steps (bottom). a similar method with more detailed demonstration is introduced in @xcite . figure 1 shows two possible addition exemplars for lms on addition problem. the scratchpad version gives more details on how humans do basic arithmetic. @xcite show that with more fine-grained demonstrations, the accuracy of addition can be improved dramatically with fine-tuning. yet, it still can not achieve 100% on ood data, even with thousands of training data points. figure 2 shows the performance of gpt-3 and t5 on addition using the scratchpad version of training data. the problem becomes more severe when there are figure 2 : the horizontal axis is the number of digits and the vertical axis is the accuracy. the prompts for gpt3 consist of 4 examples. the t5 models are trained on 1-5 digits of up to 2,000 examples and each training example consists of random numbers in the format of 2 4 1. in-dist: in-distribution. out-of-dist.: out-of-distribution (ood). in-distribution refers to training on up to k-digit numbers and testing on up to k-digit numbers while out-ofdistribution refers to training on up to k-digit numbers and testing on numbers with more digits. α indicates the repetition level of the examples. an example x 1 • • • x n with n digits are sampled with the next digit probability p(x i+1 |x i ) = α, when x i+1 = x i ; otherwise, (1α)/9. larger α indicates a higher repetition level. as the performance drops with repeating digits, we suspect that lms might not handle the repeating symbols well. figure 2 illustrates the performance of gpt-3 and t5 on the copy task, one of the simplest symbolic manipulation operations. gpt-3 and t5 still can not perform well on ood. we further do a preliminary experiment where a t5 model is fine-tuned using the data containing repeating numbers of up to 80 digits, t5 still can not achieve 100% in-distribution accuracy on long repeating digits. the results indicate that there are two problems intervening: transformers are not good at handling repeating symbols and ood generalization. the repeating symbols can also be a problem even for in-distribution data. we believe that overcoming the aforementioned limitations is of critical importance for the future application of transformer-based lms to reasoning-intensive tasks such as data format conversion and robotic process automation. in this paper, we investigate the potential causes behind this phenomenon and examine a set of possible mitigation solutions including fine-grained computation steps, positional markers, and lms with callable programs. since incorporating computation steps improves the ood generalization in arithmetic addition @xcite , one possible direction is to provide more fine-grained computation steps in the fine-tuning data or the few-shot prompt. however, it may not be sufficient to alleviate the problem of repeating numbers. when a human does addition, the position of each digit is used to differentiate the repeating digits. however, the self-attention mechanism in the transformer may not tell which "1" is referred to in the input. this prompts us to explore using positional markers to differentiate the important tokens. using these two methods to augment the reasoning process, we find that the performance of pretrained lms still can not reach satisfying results. then we resort to a method where the copy operation is implemented as a primitive function and explore whether the lm can further boost its performance. we experiment with three symbolic manipulation tasks: copying, reversing, and addition. experimental results show that although generalization in these symbolic manipulation tasks is straightforward for humans, it is still challenging for lms, and none of these mitigation methods fully solves the problems. in the end, we introduce lms with tutor which demonstrates every single step of teaching, pinpointing where these digits come from. lms with tutor is able to deliver 100% accuracy in situations of ood and repeated symbols. in this design, lms are used to generate actions that mimic operations in multiple tape turing machines, rather than the intermediate results. these actions generate the intermediate results on tapes. we hope this could shed light on the capability of transformer-based lms in addition to providing large training datasets or scaling up the size of these models. to conclude, our main contributions are: chowdhery et al. (2022) further scale the transformer-based lms to a 540-billion parameter model, called pathways language model (palm). same as brown et al. (2020), chowdhery et al. (2022) find that scaling the lms consistently results in better arithmetic reasoning ability with few-shot prompting. however, the reasoning ability of the large lms is still limited. gpt3 struggles with 3-digit arithmetic and with direct prompting, even 540b palm can not achieve high performance on complex tasks requiring multi-step reasoning. therefore wei et al. (2022) propose the following prompting method for large pretrained lms. ## experiments in this section, we conduct experiments on three different problems including copying, addition, and another basic symbolic manipulation operation, reverse. we illustrate the limitation of lms in symbolic and arithmetic induction and the improvement that could be achieved by the mitigation methods. ## conclusion in this work, we explore the limitations of pretrained lms on arithmetic reasoning and symbolic manipulation. we experiment with three simple symbolic manipulation tasks and show that improving the locating and induction capability of lms can be important for further improving their performance. our method that combines abstraction and finest-grained step-by-step tutoring demonstrates its potential to generalize correctly, shedding light on possible directions orthogonal to scaling up lms for future work in this area.