A rithm A ttack: Evaluating Robustness of LLM s to Noisy Context in Math Problem Solving (2025)

Abstract: While Large Language Models (LLMs) have shown impressive capabilities in math problem-solving tasks, their robustness to noisy inputs is not well-studied. We propose ArithmAttack to examine how robust the LLMs are when they encounter noisy prompts that contain extra noise in the form of punctuation marks. While being easy to implement, ArithmAttack does not cause any information loss since words are not added or deleted from the context. We evaluate the robustness of eight LLMs, including LLama3, Mistral, Mathstral, and DeepSeek on noisy GSM8K and MultiArith datasets. Our experiments suggest that all the studied models show vulnerability to such noise, with more noise leading to poorer performances.

## introduction as large language models (llms) are improving in their ability to accurately process human language, their math problem-solving is also enhancing @xcite @xcite . however, these sets of questions might require reasoning capabilities to be answered. while llms have been shown to have such capabilities to some extent @xcite , their robustness to adversarial inputs remains a challenge. for instance, these models can be vulnerable to simple replacement of words with their synonyms @xcite and even typographical errors can negatively impact their ability to reason @xcite . however, such attacks can semantically alter the samples by changing the current sample features to completely different ones (e.g. amoral -> moral). in this paper, we further investigate the math problem-solving robustness of llms to a different set of changes that take the form of noisy context containing a variety of punctuation marks. given that none of the words are changed when new punctuation marks are inserted into the input text, the semantic similarity of the perturbed sentence remains unchanged. the key research question for this study is: how do llms respond to noise attacks consisting of random punctuation marks in the context of math problem-solving? figure 1 shows an example of an llm response under arithmattack, where the model behaves erratically when it sees a noisy context whereas it answers the question in the clean prompt correctly. inspired by the aeda (an easier data augmentation) method @xcite , we propose arithmattack to assess the robustness of eight llms (i.e. two llama models @xcite , two mistral models @xcite , zephyr @xcite , gemma2 @xcite , qwen2.5 @xcite ) to noisy data. similarly to aeda, we introduce this noise by randomly inserting punctuation marks into the context of math problems from two math datasets, namely gsm8k @xcite and multiarith @xcite . we then evaluate how these models perform under different noise levels, with the noise affecting 10%, 30%, and 50% of the sentence length (based on the number of words). ## related work noise insertion has been shown to be effective in deteriorating the performance of encoder models in various tasks such as toxic text classification @xcite , sentiment analysis @xcite , and natural language inference @xcite . in the context of math problem solving, large language models (llms) have been shown to be vulnerable to a variety of changes in the input context, including typographical errors @xcite , word replacement @xcite , gibberish or irrelevant context inclusion @xcite , and semantic perturbations @xcite . @xcite propose an adversarial typo attack that breaks the reasoning process of llms. instead of modifying characters, @xcite propose a dataset, called robustmath, where they replace words with their synonyms to evaluate the robustness of large language models. similarly, @xcite propose gsm-plus dataset, based on gsm8k, modified with a variety of mathematical perturbations such as distractor insertion and arithmetic variation. in the study by @xcite , the authors employ different types of textual attacks on prompts, including character, word, sentence, and semantic attacks. in contrast, @xcite propose to modify the numeric values in the questions using abstract syntax trees resulting in examples that fool the llms. while the literature mainly concentrates on modifying the lexical or semantic content of the prompts, we aim to keep the contextual information intact and instead focus on the model behavior changes in reasoning when encountering punctuation noise. in addition, an advantage of our method is that it is extremely straightforward to implement, and as we show in the results section, it is also effective in degrading the performance of llms in math problem-solving. ## experiments to carry out our experiments, we use two wellknown math datasets and eight llms. ## methodology to obtain the responses from llms, we use the zero-shot cot @xcite prompting, using the following prompt: ## results and analysis as shown in tables 1 and 2 , llama3.1 outperforms other models across both datasets in the majority of the cases. it achieves the highest accuracies in both clean and noise-affected settings (except in 30% and 50% noisy data of the multiarith dataset where deepseek in the former and both llama3 and deepseek in the latter have higher accuracies). this makes it the most reliable model for handling mathematical problems under noisy input conditions. however, in terms of asr score, llama3 has the lowest score and llama3.1 with a slightly higher one, indicating that llama models are more robust to noise than other studied models with the exception of deepseek only in multiarith dataset showing comparable robustness. in addition, the mathstral model compared to mistral exhibits more robustness which can be attributed to its higher mathematical understanding. in contrast, zephyr was the lowest-performing model, exhibiting low clean accuracy and suffering a significant decline in performance as noise was introduced. its high asr score makes it unsuitable for tasks involving noisy data, reflecting poor robustness. figure 2 shows the relationship between the model's accuracy and the noise present in the datasets. for both datasets, as the percentage of models attack success rate (%) 0 20 40 60 80 m i s t r a l m a t h s t r a l l l a m a 3 l l a m a 3 . 1 g e m m a 2 z e p h y r q w e n 2 . 5 d e e p s e e k gsm8k multiarith figure 3: comparing the attack success rates on the studied models for gsm8k and multiarith datasets (lower is better for model robustness). llama models are more robust than others under arithmattack. across all models except for zephyr, the impact of noise was more pronounced in the gsm8k dataset than in multiarith, with a larger drop in accuracy as the noise levels increased (figure 3 ). in manual inspection, we found out that the gsm8k dataset was more difficult to solve than the multi-arith dataset. this suggests that the models may struggle more with noise in math datasets with more difficulty. answer extraction accuracy to evaluate the accuracy of the models, we developed a script to extract answers from the llm responses. the extraction process underwent multiple iterations, as it needed to accurately extract the answer and compare it with the ground truth. however, even with the final prompt, we observed a couple of inconsistencies in the answer extraction. therefore, we went through outputs manually to estimate the miss rate (i.e. the rate with which the correct answer is not extracted). in manual inspection, we evaluated the entire responses for the multiarith dataset and the first 100 responses for the gsm8k dataset from all the models except for the deepseek model. for this model (due to time and labor constraints), we evaluated the first 50 samples from each dataset. ## conclusions and future work we evaluated how well different language models handle mathematical problem-solving tasks in both clean and noisy conditions. our results indicate that all studied models can be vulnerable to extra noise with varying degrees, with llama models being the highest-performing and the most robust model in the majority of the experiments. in addition, comparing the two models of mathstral and mistral from the same family, the one with mathematical knowledge exhibited more robustness to noise. lastly, the findings revealed that more complex datasets such as gsm8k can become more difficult to understand as they become noisier. future research can include datasets beyond gsm8k and multiarith as well as other reasoning tasks such as logical and causal reasoning, which could provide deeper insights into the models' robustness in different scenarios. further experimentation with different types of noise could also help enhance our understanding of the latent vulnerabilities in llms. finally, explaining why arithmattack can break the reasoning flow of llms would be another valuable follow-up to this work.