Models Can and Should Embrace the Communicative Nature of Human-Generated Math

Authors: sasha  boguraev
ben  lipkin
leonie  weissweiler
kyle  mahowald
Venue: NeurIPS_cc_2024_Workshop_MATH-AI
Published: 2024-09-21 05:17:59

Abstract: Math is constructed by people for people:
just as natural language corpora reflect not just propositions but the communicative goals of language users, the math data that models are trained on reflects not just idealized mathematical entities but rich communicative intentions.
While there are important advantages to treating math in a purely symbolic manner, we here hypothesize that there are benefits to treating math as situated linguistic communication and that language models are well suited for this goal, in ways that are not fully appreciated.
We illustrate these points with two case studies. First, we ran an experiment in which we found that language models interpret the equals sign in a humanlike way---generating systematically different word problems for the same underlying equation arranged in different ways. 
Second, we found that language models prefer proofs to be ordered in naturalistic ways, even though other orders would be logically equivalent.
We advocate for AI systems that learn from and represent the communicative intentions latent in human-generated math.

Main paper: 
 introduction: 
 Language Models sometimes rely on heuristics and statistics rather than being perfectly compositional idealized reasoners, especially in domains like math and logic [27, 30, 33, 34, 36, 42, 45, 47] . Whereas language production and comprehension involve some idealized composition using abstract rules [8, 20] , in tandem with memorization and pragmatic inference [9, 16] , math and logic reflect domains where one might expect an idealized compositional system to be required for obtaining precise solutions. Indeed, whether an expression is written 5 + x = 7 or 7 -5 = x or "What is 5 less than 7?" or "Seven frogs were sitting on a log. Five left. How many are there now?", there is an underlying computation that can be extracted and performed (namely, the expression 7 -5). To properly solve these problems, the thinking goes, systems should abstract away from their situated format into symbolic space.There is an intuitive, and well-justified, idea that competent human mathematical reasoners employ exactly this kind of abstraction. By contrast, less competent mathematical reasoners (e.g., children struggling to learn math) are often shown to rely on heuristics, schemas, and keywords [6, 10, 24, 38, 48] . For instance, kids might learn that every time they see the phrase "in total" in a word problem, they should add up all the numbers [39] . While the "heuristic" keyword-based direct translation approach may be less cognitively taxing, it is also prone to translation errors [49] . Students who 38th Conference on Neural Information Processing Systems (NeurIPS 2024). report adopting the more involved strategy of first parsing a math word problem into a structured mental model, then planning computation and finally evaluating the solution in that space, are more successful problem solvers [19] .Taken together, these ideas might make it seem like the goal of AI math models should be to leave the messy domain of language behind and translate expressions into symbolic representations. And, indeed, combining language models with symbolic solvers has proven successful in a variety of math and reasoning domains [4, 14, 18, 32, 46, 53] .Here, we argue that something is lost when disregarding the original context. We introduce the Communicative Math Hypothesis: Math is constructed by people, for people. As such, there are conventions and pragmatics that people bring to the production and comprehension of mathematical expressions-communicative interpretations that go beyond the purely symbolic. Such traces of information are particularly well suited for study via the tools of linguistics and cognitive science. The choice to write 3x + 9 instead of 3(x + 3) conveys something to the reader, even though they are equivalent. Similarly, the proof of a theorem is not only a formalization that could be computationally verified, but is a communicative act, with intention of being internalized and understood by others.Drawing on research in math education that we believe is underappreciated in machine learning, we make the case for AI researchers to take the Communicative Math Hypothesis seriously. We present some initial proof-of-concept experiments showing that LLMs pick up on these communicative regularities. We argue that this information should not always be ignored or explained away, but is a crucial component of human mathematics. 
case study one: equations are asymmetric: 
 Asymmetry in human mathematical interpretation has long been studied in math education. In particular, there is a wealth of literature on the perils of grade-school-aged children's asymmetrical understanding of math -that is, a difficulty in reasoning with a problem such as = 2 + 4, despite relative comfort with the complementary equation of 2 + 4 = [2, 37] . But, such sensitivity to asymmetry is not isolated to students. Even expert mathematicians understand math asymmetrically [29] , offering different interpretations of equivalent expressions based on what is on the left or right of the equals sign. Here, we present results from a case study demonstrating that LLMs are sensitive to such asymmetries in equations as well and, like humans, do not learn a purely symmetrical interpretation of the equals sign.Methods To test LLMs' sensitivity to symmetry, we conduct an experiment assessing their ability to reconstruct the equations they used to create a specific word problem, as shown in Figure 1 . Formally, we perform a three-step experiment. We first generate a set of n paired forward and reverse equations, denoted as E = {e 1 , e 2 , . . . , e n }, where each paired equation e i consists of the forward equation e i f and the reverse equation e i r . Thus, we can express each e i as e i = {e i f , e i r }. Next, for each of our n pairs, we pass both equations in e i to GPT-4o, and prompt it to generate a corresponding pair of word problems, w i = {w i f , w i r }, that could be solved by e i , with W = {w 1 . . . w n }. We finally ask the LLM to extract the equations e i = {e i f , e i r } for each w i ∈ W , with E = {e 1 . . . e n }. Our hypothesis is that across all n equations, the LLMs will more often recover e i f from e i f and e i r from e i r . For details on the equations used, their generation, and model prompting, see Appendix A.We measure the recovery rate of an equation's original order and the respective reverse order using GPT-4o across 5 different sets of 200 pairs of randomly generated starting equations. We found that the original equation was recovered on average 52% of the time with a 95% CI of [51%, 54%] across 5 runs. The reverse equation was nearly never recovered: 0.2% of the time, with a 95% CI of [0.0%, 0.4%] -a mere 3 times over the 1000 samples.These results suggest a difference between the word problems generated from a "forward" equation and word problems generated from a logically equivalent "reverse" equation, and that this difference is itself recoverable by GPT-4o. We posit that this information, which a purely symbolic solver would be agnostic to, is crucial information for systems that aim to use math in collaboration with humans or in human-like ways. These findings are consistent with work showing that premise order matters in LLMs' ability to reason [3, 7, 51] , although they frame this order sensitivity as primarily revealing LLMs' brittleness. We interpret these findings (and theirs) as revealing sensitivity to important communicative factors inherent in the data. 
case study two: mathematical rules and proofs have orders: 
 Our second case study focuses on mathematical communication of the sort more likely to take place among professional mathematicians: mathematical rules and proofs. Proofs, in particular, are widely used in academic math, as well as related fields, and are duly an area of major focus for AI for math.Proofs are written to communicate truths that are, in some sense, tautological. Nonetheless, mathematicians have strong expectations and interpretations about the directionality of equation. For instance, there are generalized principles associated with equal signs, like that the right side of the equation expounds upon or explains the left side [29] . Thus, while a = b and b = a are equivalent statements by our agreed-upon set of axioms and inference rules, the choice of one or the other might communicate a different message when used in a proof.To explore the preferred orderings used by mathematicians in proofs and rules, Mirin and Dawkins [29] utilize a set of breaching experiments. Breaching experiments are a class of experiments which try to break rules in an attempt to confirm their existence [41] . In particular, the authors first provided expert mathematicians with a host of formal mathematical equations, such as the distributive rule or an inductive proof. However, these equations were ordered in an unnatural manner -that is, in the case of rules, orders which are not commonly encountered in formal mathematical texts, or in the case of proofs, orders in which steps do not sequentially build from one to the next. The authors measured whether these mathematicians reported any perceived breaches, with any such breaches providing evidence for the existence of the mathematicians' ordering preferences. Our case study into LLM ordering preferences in formal mathematics follows in this vein, measuring LLM surprisals for various natural (extant) and unnatural (unobserved) equation orderings.Methods Our set of mathematical equations consists of all examples used in the breaching experiments of Mirin and Dawkins [29] . This totals ten different examples, six of which are one line equivalences, expressing common mathematical rules, and the other four of which are a series of equivalences comprising a longer proofs. Each example further contains a brief textual introduction before the series of equivalences. All examples are reported in Appendix B.We first split each equation into its individual expressions. We then generate every possible ordering of a given equation by permuting the order of these individual expressions. Finally, for each model we calculate the average per-token surprisal for every ordering of expressions in a given equation, conditioned on that equation's textual introductions. Our calculations are performed using the minicons package [31] , a wrapper around Huggingface's transformers package [50] .In this case study, we use the instruction-tuned variants of four models: LLaMa 3.1 8B [12] , Mistral 7B v0.3 [23] , Mathstral 7B [1] , and Qwen2-Math 7B [52] . Two of these models were trained on general corpora (LLaMa and Mistral), the other two fine-tuned on math (Mathstral and Qwen2-Math).To control for our ten equations potentially being within the evaluated model's training data, we also performed evaluations with three sets of modified, but equivalent, variants of each equation. Our first variants consist of all proofs reworded in a logically equivalent, but expressively different, manner. Our second variants systematically replace all equation variable names, and some rule names, with emojis -maintaining the correctness of equations, but presenting . We find that the original order ( ) has lower per-token surprisals on average (more probable) than equivalent counterfactual orders.them in a unique, unseen, manner. Our last set of variants combines the two previous variants, substituting emojis into the reworded variants. All variants are reported in Appendix C.As seen in Figure 2 , the evaluated models display clear and consistent preferences for the natural ordering in nine of ten equations. In seven of these, all models display uniform preference for each equation's natural ordering. Of the remaining two equations (DIFFERENCE QUOTIENT and PROOF), only a few nearby orders had a lower surprisal than the natural orders (99.6 th and 98.3 rd percentiles, respectively). The only equation for which there is no clear model preference for the natural form is PRODUCT RULE, but this was also a rule noted as unusual by participants in Mirin and Dawkins [29] : mathematicians expressed surprise at seeing f and g instead of f (x) and g(x). When we instead use the latter notation, we see consistent preferences for the natural order. We do not find significant differences between the performances of math-fine-tuned models and more generalized language models across all equations (paired t = 0.606, p = 0.548).These results are further consistent across our equation variants (Figure 2 ). While there is minor variability here (e.g., after rewording proofs, models no longer display clear preferences in EXPONENT PROD RULE but do display clear preferences for natural orderings in PRODUCT RULE) the evaluated LLMs maintain clear and consistent preferences for natural equation orderings even when modified.These results suggest that LLMs agree with expert mathematicians in their preferences for ordering of proofs and rules, that is in a manner which expresses clear communicative intent. Further, our work with equation variants suggest that they are aligned due to more than just memorizing training data. This alignment leads to AI systems able to produce math interpretable by those using them, which in comparison to much of the uninterpretable math produced by symbolic solvers and logic programming systems, is a highly desirable quality. As such, while the proofs LLMs produce in their current iteration may not always be correct, any remedies attempting to improve on that correctness should not do so to the detriment of this alignment, if the goal is human use. 
practical applications: 
 We focused our experiments on equation asymmetry and proof ordering, showing that LLMs learn extra-symbolic communicative information in both domains. But these principles encompass a much broader class of phenomena. For instance, several patterns identified as reflecting LLMs' brittleness may instead be fruitfully seen as contributing to the communicative interpretation of math.• Even though they don't matter logically, variable names matter for communicating math (e.g., functions are often f and g). This pattern extends to programming as well [21, 28] . • Logically extraneous or pragmatically anomalous information can matter for inferences about how expressions are interpreted [35, 45] . • Notation choice and instruction/prompt phrasing can matter for how problems are solved [17, 22] .Seeing these aspects of LLMs as possible features, and not bugs, could be an important step in developing AI systems that can work with humans. For instance, working mathematicians were long limited to purely symbolic theorem provers. Such systems in isolation neglect the more human aspects of math, ignoring differences in style and comprehensibility. We recommend developing proof assistants that are sensitive to these regularities in human proof-writing and other communicative cues. LLM-based proof systems offer the promise of mathematical assistants that can work with people [11] , alongside them and not just for them as blackbox tools. Below we discuss this idea in two particularly relevant domains.Math Education Math educators leverage their explicit and implicit understanding of mathematical communicative signaling to enhance teaching. They carefully choose problems presented in manners that probe the intended concepts [26, 44] . They can identify subtle misunderstandings in their students' reasoning just by observing how they discuss mathematical concepts and use them in practice [5, 25, 40, 43] . These are key skills for educators, allowing for more efficient and effective teaching. As we move towards building AI assistants for math education, it is pertinent to develop systems that, like math educators, can both produce and identify these rich communicative signals.The furthering of knowledge in any field depends upon the ability to communicate new ideas. If AI math systems are unable to communicate with those using them, we risk merely developing powerful systems that remain limited in their benefits. Instead, we advocate for building systems that produce math in a manner which is communicative and human-like by design, offering promise of furthering our collective mathematical knowledge base. While there is some benefit to systems that can solve problems and prove theorems that humans cannot, what we gain is limited if little information from their methods can be communicated and thusly understood. Of course, AI systems augmented with symbolic solvers do possess the necessary qualities of correctness and robustness which current, non-augmented, LLMs do not. We are not arguing that future AI math systems should lose these qualities, merely that they should also possess communicative sensitivity, something that many current approaches lack. Developing a new generation of hybrid systems, that work through problems via human-interpretable traces, while in parallel formalizing and verifying via symbolic means, appears to be a fruitful path forward. 
conclusion: 
 While necessarily fuzzier than purely symbolic representations, the communicative principles in human-generated math are not lawless or illogical but can be studied, systematized, and modeled as rational behavior-as they are in linguistics and cognitive science [9, 13, 15] . We join Zhang et al. [54] in their call for a cognitive science perspective on AI and mathematics, centering the role of math as a group activity and communicative endeavor. The math of the people, by the people, for the people, shall not perish from our models. 
 
 Appendix: For step one of this experiment, we create our equation sets as follows. We first create two independent expressions, each of which consists of two operands, either added or subtracted to each other. One of these operands is a single digit number, with the other being a variable quantity in x with a single digit coefficient. All operands, operations, and choice of which operands are the variable quantity are selected at random. We then form our pair of complimentary equations by placing an equals sign between these two expressions, in both orders. That is, given the two expressions a and b, our pair of complimentary equations would be a = b and b = a. To illustrate further, a generated set of expressions may include, for example, 2x + 3 = 4 -5x or 8 -5x = 2 + 3x, but not 2x = 3, 4x + 5y = 8x -2, or 9x * 2 = x -4.Our experimental methodology necessitates prompting GPT-4o twice for each equation in our evaluation set: once to create a word problem from a given equation, and once to try and recover an equation given a math word problem. Below, we describe the prompts used for each of these steps.For a given equation, EQUATION, we first prime GPT-4o with the following command: "You are a helpful middle school math teacher."We then prompt the model to generate a word problem using the following prompt: "Create a grade-school math problem representing the following equation: {EQUATION}. Make sure your problem is clear, concise, represents every term of the equation, and ends in a question mark. Generate just the problem and nothing else."For a given math word problem, PROBLEM, we first prime GPT-4o with the following command: You are a helpful assistant.We then prompt the model to recover the equation that is represented by PROBLEM with the following prompt:"What is the underlying math equation represented by the following situation: {PROBLEM}. Use the letter 'x' for the unknown quantity. Please do not explain, or write any accompanying text, give just a single equation and nothing else."We use the following set of equations from Mirin and Dawkins [29] for evaluating model order preferences in formal mathematics. We name each following subsection as they are labeled in Figure 2 . Each equation is presented below in its "natural" form. All T E Xformatting used to render the following sections, up-to the end of the equations, is included in our experiment.The difference quotient of a function g is defined to beg(x + h) -g(x) (x + h) -xwhere h is nonzero. Let f : R → R be the function defined by f (x) = x 2 . The following shows the difference quotient:f (x + h) -f (x) (x + h) -x = f (x + h) -f (x) h = (x + h) 2 -x 2 h = x 2 + 2xh + h 2 -x 2 h = 2xh h 2 = 2x + h B.2 DISTRIBUTIVEThe distributive law tells us that for all numbers x, y, and z, Let S, and S , be binary algebraic structures. A homomorphism from S, to S , is a function φ : S → S such that for all x, y ∈ S, φ(x y) = φ(x) φ(y)The following is a portion of a proof by induction that for all natural numbers k, k 3 -k is divisible by 6. At this point in the proof, it has been assumed that n 3 -n is divisible by 6, and it is being shown that (n + 1) 3 -(n + 1) is therefore also divisible by 6.(n + 1) 3 -(n + 1) = (n 3 + 3n 2 + 3n + 1) -(n + 1) = (n 3 + 3n 2 + 3n + 1) -(n + 1) = (n 3 -n) + (3n 2 + 3n) = (n 3 -n) + 3n(n + 1)The product rule for derivatives says that if f and g are differentiable functions, then f g + f g = (f g) B.9 PROOF Theorem 1. Suppose S, and S , be binary algebraic structures, and φ is an isomorphism from S, onto S , . Further suppose that e is a left identity element in S, . Then φ(e) is a left identity element in S , . Proof. Let s be an element of S . Since φ is onto, there exists some s ∈ S such that φ(s) = s . Hence s = φ(s) = φ(e s) = φ(e) φ(s) = φ(e) sThe following is a proof in a set theory textbook that if a is a transitive set, then (a + ) = a. Note that a transitive set is defined to be a set a such that all members of a are subsets of a, and a + is defined to be a ∪ {a} Proof. Inductively prove that ∀k ∈ N, k 3 -k is divisible by 6. We will show that (n + 1) 3 -(n + 1) is divisible by 6, with the prior assumption that n 3 -n is divisible by 6. A transitive set is defined to be a set a such that all members of a are subsets of a, and a + is defined to be a ∪ {a}. We show a proof that if a is a transitive set, then (a + ) = a.f (x + h) -f (x) (x + h) -x = f (x + h) -f (x) h = (x + h) 2 -x 2 h = x 2 + 2xh + h 2 -x 2 h = 2xh h 2 = 2x + h C.1.2 DISTRIBUTIVEProof.  


Decision: This paper presents an interesting observation that LLMs can capture more than mathematical reasoning, but also communicative aspects of how humans write mathematics. It should be read more as a position paper, suggesting a space to be explored and motivating it via simple case studies that show the point that LLMs are sensitive to communicative features of language that are formally irrelevant (yet meaningful for humans). I believe it will spark interesting discussions to the workshop. I suggest the authors also discuss potential implications of how the field might concretely respond to the "Communicative Math Hypothesis", and address several of the other points raised by reviewers.

Reviews: review: title: title: This is not a real research paper. The design of experiments is also wrong.
main_review: main_review: This paper explores (1) if large language models can understand asymmetry in equations and (2) if they can detect the ordering of equations. 

Even though the authors did some simple experiments, this paper lacks any novelty in methods or results. All they did is putting prompts into large language models and getting the output. 

Moreover, experiment 2 is even wrong in its design. The ten examples of equations tested in this paper are all well-known equations and large language models have already seen them millions of times if not billions. If you reorder the equations, no wonder that these models will be surprised.

I also doubt the result of experiment 1. The authors may put the construction step and reconstruction step in one single chat. ChatGPT-4o will then give the same forward/backward equation just based on contexts.

If it was 1998 when Google first launched its search engines, this paper is just like "I found that we can get result A but not B if we search the keyword xxx". This is not a research paper. It can be a blog post though.
title: title: Great theoretical idea, further practical usages are highly recommended for the key findings
main_review: main_review: This paper explores a novel perspective on how AI models, particularly large language models (LLMs), should handle mathematical reasoning. The authors propose the "Communicative Math Hypothesis," which emphasizes that mathematics is not just a symbolic manipulation but also a communicative act constructed by humans for humans. The paper demonstrates this hypothesis through two case studies: the asymmetry in equations and the ordering of mathematical proofs. The results suggest that LLMs, like humans, show preferences for certain equation structures and proof orders, which the authors argue should not be disregarded in future AI system designs. The paper skillfully bridges the gap between machine learning, cognitive science, and mathematics. By drawing parallels between linguistic communication and mathematical reasoning, it offers a fresh perspective that could inform the future development of AI systems, especially in fields like education and theorem proving. The interdisciplinary nature of this research enhances its contribution to multiple fields.

However, there are still improvements the authors could make:
While the findings demonstrate that the mathematical expressions generated by the models follow patterns consistent with human conventions, this result is somewhat expected. Transformer-based models, such as those used in this study, are typically trained on large datasets that reflect human-generated language, including mathematical language. Naturally, the sequences generated by these models will align with the common structures found in human-written mathematical expressions. This is largely because the training data itself encodes these conventions, and the models learn to predict the most probable next token based on the patterns they have observed during training. Transformers, particularly when trained with teacher forcing techniques, learn to replicate the input-output mappings found in the data, and thus the generated sequences closely mirror the input format. Although the paper introduces an interesting theoretical perspective, it does not sufficiently address how these findings could be applied in real-world scenarios. For example, it remains unclear how embracing the communicative nature of mathematics in AI models would improve current AI-based mathematical tools, such as those used in education or research. A more detailed discussion on the practical implications and potential use cases of this approach would greatly enhance the relevance and impact of the paper. Without clear pathways to application, the findings risk being seen as more theoretical than actionable.
Besides, in figure 2, it would be great if the authors could explain more about the counterfactual possible orders, inlcuding whether those counterfactual orders are equivalent to the original order or not. 
In summary, while the theoretical observations are sound, the findings themselves may not come across as particularly novel or surprising, especially given the underlying mechanics of how LLMs are trained. Furthermore, the paper lacks a clear indication of how these results could be leveraged for future research or practical advancements. Providing more insight into how these findings could influence future studies or be integrated into real-world systems would improve the significance of the contribution.
title: title: Review for "Models Can and Should Embrace the Communicative Nature of Human-Generated Math"
main_review: main_review: This paper argues that treating mathematical reasoning solely in terms of abstract symbolic manipulation overlooks important communicative and pragmatic aspects inherent in human-generated mathematics. For this central point, as a mathematician myself, I conditionally agree, where the conditions depend on the level of reasoning. In high-level theorem proving or scientific calculations, precision and symbolic rigor may be more important than communicative flexibility. While in more elementary mathematics (especially in education), the communicative nature is crucial.  The authors propose the "Communicative Math Hypothesis," which emphasizes that mathematical language is not purely symbolic but deeply situated within the context of human communication. The paper presents two case studies that explore how large language models (LLMs) like GPT-4 exhibit sensitivity to mathematical expression ordering, suggesting that LLMs can learn and reflect these communicative nuances. Although I believe that this paper is of wide interest to the readership of this workshop, there are some concerns need to be addressed first:

(1)	I think the authors didn’t survey the literature extensively, there are already some literatures on the reversal curse (in similar spirit of LLM’s sensitivity to symmetry discussed in this paper) of LLMs, like The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" and “Exploring the reversal curse and other deductive logical reasoning in BERT and GPT-based large language models.”
(2)	While the two case studies presented (on equation asymmetry and proof ordering) are interesting, they are somewhat limited in scope. The sample size in the asymmetry experiment is relatively small, and while the results show statistical significance, they lack the robustness that a larger and more diverse dataset could provide. I suggest authors also include different types of equations with multiplications, bracket, division etc. involved. Although I would guess confidently that the same conclusion would still hold, but still I would include them for a more comprehensive set of empirical analysis.
(3)	Following up the previous point, I think authors also need to include more detailed error analysis, especially for the reverse equation. More specifically, I believe that LLMs tend to bias towards certain types of errors and author and this part would be more informative.
(4)	Back to the main point, the paper’s central claim is that models should embrace the communicative nature of math. However, it does not fully address potential counterarguments. As I said, theorem proving or scientific calculations, precision and symbolic rigor may be more important than linguistic communicative flexibility. I suggest authors include a more thorough discussion of when communicative considerations should take precedence would be useful.
(5)	While the authors suggest that LLMs could improve collaboration with humans by better aligning with communicative norms, they do not provide concrete examples of how this would work in practice. Including scenarios where LLMs assist human mathematicians or students in understanding or solving problems could illustrate the real-world impact of their approach. Moreover, I think a user study evaluating how humans interact with communicative versus purely symbolic math AI systems would have strengthened the paper’s claims.