Adversarial Math Word Problem Generation (2024)

Abstract: Large language models (LLMs) have significantly transformed the educational landscape. As current plagiarism detection tools struggle to keep pace with LLMs’ rapid advancements, the educational community faces the challenge of assessing students’ true problem-solving abilities in the presence of LLMs. In this work, we explore a new paradigm for ensuring fair evaluation—generating adversarial examples which preserve the structure and difficulty of the original questions aimed for assessment, but are unsolvable by LLMs. Focusing on the domain of math word problems, we leverage abstract syntax trees to structurally generate adversarial examples that cause LLMs to produce incorrect answers by simply editing the numeric values in the problems. We conduct experiments on various open- and closed-source LLMs, quantitatively and qualitatively demonstrating that our method significantly degrades their math problem-solving ability. We identify shared vulnerabilities among LLMs and propose a cost-effective approach to attack high-cost models. Additionally, we conduct automatic analysis to investigate the cause of failure, providing further insights into the limitations of LLMs.

## introduction recent advances in large language models (llms) have revolutionized the world of education, primarily due to the great improvements in their natural language generation and problem-solving capabilities. this has transformed how students access information and complete assignments, raising significant concerns among educators in accurately evaluating students' true problem-solving abilities with the presence of such powerful tools @xcite @xcite . while efforts like plagiarism detection exist @xcite , their effectiveness in identifying llm-generated content is limited @xcite , underscoring the need for more advanced anti-plagiarism methods to match llm advancements. at the same time, adversarial attacks on llms have gained more attention due to increased awareness of the potential risks associated with llms. most work on adversarial attacks focuses on developing prompts to elicit specific outputs from llms @xcite @xcite recent work suggests that even the most powerful aligned llms are vulnerable to such attacks @xcite . hence, we might expect that as llms become stronger, detecting their outputs will become more difficult, but adversarial examples may still persist @xcite . to this end, we introduce a new paradigm for generating homework assignments that llms cannot solve by utilizing adversarial attacks. we focus on math word problems (mwps), which present a unique and challenging intersection of language and mathematics. in this work, we ask the question: "can llms still solve an mwp after changing its numeric values?" we aim to create mwps that llms are unable to solve while maintaining the original difficulty and coherence of the problems. our aim is not just to challenge llms but to do so in a way that reflects real-world educational standards to ensure the mwps remain educationally valuable and relevant. while conceptually simple, doing this automatically is challenging as directly modifying numbers in the problem could lead to nonsensical problems. for example, consider the problem "a class has 6 male students and half as many female students. how many students in total?" directly change the number of male students from 6 to 5 (or any odd numbers) without checking its intermediate computational steps might result in "2.5 female students," figure 1 : method overview: given a mwp that an llm can correctly solve, our method first transforms it into python code. the python code then is converted into an ast representation, which is used to generate adversarial problems by modifying the numeric values in a controllable manner. we place constraints on the nodes of the ast to ensure that the modified problem maintains the same difficulty level as the original problem. despite this, we find that the resulting adversarial examples cause llms to predict incorrect answers. to effectively generate modifications in large scale to assess the robustness of llms also requires: (i) the answer to the altered problem changes must be recomputed automatically; and (ii) preserving the difficulty and validity of the modified problem requires us to ensure that all the intermediate calculations in the new problem are consistent with the original problem. to tackle these challenges, we first convert mwps to a code representation and leverage abstract syntax trees (asts) to map each calculation step into a node. we then define educational constraints for each node to ensure all the desired properties are preserved for the generated new problem. in this work, we evaluate several llms and demonstrate the effectiveness of our method by achieving a significant attack success rate (asr). our approach outperforms the previous rephrasing attack by an average of 62% asr. we investigate universal attacks and attack transferability, proposing a cost-effective approach to attack high-cost models (e.g., gpt-4) by reducing api request calls by 90% while achieving high performance. we conduct human evaluations to verify that our generated problems indeed preserve the original coherence and difficulty. furthermore, we perform a regression analysis and find that our adversarial examples exploit different weaknesses of each model, offering valuable insights into llm's limitation. ## methodology mwps are presented in natural language, which creates challenges for systematic structural and syntactic modifications. in this section, we describe our approach to generating adversarial mwps. an overview of our method can be found in figure 1 . we denote an original problem-answer pair as @xmath0, y), where x is a sequence of tokens x 1 , . . . , x n in the problem and y is its ground-truth answer. we define g as a ground-truth function that computes the correct answer for a math problem and f as a function which maps the elements of a sequence as: f (x i ) ∼ r (i.e., a random real number) if x i is numeric and f ( xi ) = x i otherwise. 3 we denote the set of all possible adversarial modifications to it as: a(x, y) will also consist of many unnatural and difficult problems, hence later in §3.2 we will introduce filtering constraints for selecting the adversarial examples that preserve difficulty. ## experiments and results in this section, we present multiple experiments and demonstrate the effectiveness of our method on attacking various llms. ## analysis and discussion in this section, we conduct analysis to better understand our generation methods and the mathematical capabilities of llms. ## conclusion we introduce a novel method to generate adversarial mwps using asts. our approach effectively challenges the mathematical problem-solving abilities of llms while maintaining the original difficulty and coherence of the problems. the generated adversarial examples significantly degrade the performance of both open-and closed-source llms, surpassing previous attacks by an average of 62% asr. we validate our adversarial examples through human evaluation and investigate universal attacks and transferability, proposing a costeffective method to attack high-cost api-based models with up to a 90% reduction in requests. automatic regression analyses reveal distinct weaknesses of different models when solving mwps. our work contributes to the development of fair and robust educational tools, ensuring ethically sound evaluations and promoting the responsible use of llms in education.