How Often Are Errors in Natural Language Reasoning Due to Paraphrastic Variability? (2024)

Abstract: Large language models have been shown to behave inconsistently in response to meaning-preserving paraphrastic inputs. At the same time, researchers evaluate the knowledge and reasoning abilities of these models with test evaluations that do not disaggregate the effect of paraphrastic variability on performance. We propose a metric, PC, for evaluating the paraphrastic consistency of natural language reasoning models based on the probability of a model achieving the same correctness on two paraphrases of the same problem. We mathematically connect this metric to the proportion of a model’s variance in correctness attributable to paraphrasing. To estimate PC, we collect ParaNlu, a dataset of 7,782 human-written and validated paraphrased reasoning problems constructed on top of existing benchmark datasets for defeasible and abductive natural language inference.1 Using ParaNlu, we measure the paraphrastic consistency of several model classes and show that consistency dramatically increases with pretraining but not fine-tuning. All models tested exhibited room for improvement in paraphrastic consistency.

## introduction the nlp community has transitioned away from ''deeper'' abstract semantic representations (e.g., framenet @xcite ) towards ''shallower'' representations (e.g., universal dependencies @xcite ) which retain attributes of their original surface form. the culmination of this trend is to use natural language as a semantic representation itself for evaluating a model's reasoning ability. this has enabled rapid advancement across a host of tasks including nli @xcite and qa @xcite , with the latest generation of large language models saturating many benchmark natural language understanding datasets. however, natural language as a meaning representation is highly ambiguous @xcite . while versatile and compact, it leaves open the possibility that systems are not robust to different ways of expressing the same meaning in natural language. benchmark evaluation datasets such as snli @xcite consist of a collection of reasoning problems designed to probe particular aspects of commonsense knowledge, with each example represented by a singular linguistic expression. when a system gets a particular example correct, it is only evidence that it was able to correctly reason for the particular phrasing used in the example, allowing for the possibility of systems that can correctly solve one form of a reasoning problem, but not others. conversely, if a model gets a question wrong, how can we tell if the error was due to a failure in language understanding or a failure in reasoning? consider the defeasible reasoning example in figure 1 . a language model finetuned on the δ-nli dataset @xcite ) may correctly predict that the original update sentence strengthens a human's belief in the hypothesis sentence. however, different linguistic expressions of that same update sentence may yield high variance in a model's predictions. if models stay consistent in the face of paraphrastic variability, we may conclude that correctly reasoning about one expression is indicative of an understanding of that reasoning problem, a desirable property of teaching machines to reason entirely in natural language. we explore the sensitivity of natural language reasoning models to paraphrasing as a way to better characterize their knowledge and reasoning ability, contextualize their performance on evaluation sets, and evaluate room for improvement on the basis of consistency. under the assumption figure 1 : δ-nli instance with a set of paraphrased update sentences. we study paraphrastic consistency, or the probability that a model's prediction for two phrasings of the same problem match. to study this, we build on top of two nlu datasets-abductive nli @xcite and defeasible nli @xcite -by collecting paraphrases of reasoning problems using label-preserving paraphrasing, a functional change of traditional paraphrasing that preserves the semantics of the core reasoning problem. our dataset, paranlu, contains 7,782 human-elicited and 7,295 model-elicited paraphrases across 1,000 reasoning problems spanning both datasets. we select diverse examples to paraphrase ranging in difficulty @xcite and model confidence. our dataset is entirely manually validated, ensuring semantic equivalence while maximizing paraphrase diversity. we measure paraphrastic consistency (p c ), or the likelihood of model's prediction remaining consistent under different phrasings, in order to understand the types of surface-form changes that models are sensitive to. we study the relationship between consistency and various data conditions and modeling paradigms, exploring factors such as data source, example difficulty, model complexity, and training dynamics. for their given accuracy level, we find that models still have room to improve on paraphrastic consistency. since no model demonstrates high accuracy and high paraphrastic consistency, we conclude that attempts to measure their reasoning abilities will be confounded by inconsistencies in their linguistic abilities. ## paraphrastic consistency in principle, natural language reasoning tasks like abductive nli and defeasible nli require the ability to linguistically decode the meaning of the text that expresses an underlying problem, as well as the knowledge and reasoning capabilities to solve the underlying problem. by analogy, consider evaluating a child's understanding of the concept of addition. instead of simply presenting them with a mathematical expression (say, 7 + 7), we write a word problem that can be answered by (1) understanding the situation in natural language, (2) recognizing that the answer corresponds to the mathematical reasoning problem 7 + 7, and finally, (3) solving 7 + 7. if the child answers incorrectly, we must figure out whether they did not understand the goal of the word problem or were not able to perform the arithmetic in order to evaluate their mathematical reasoning ability. for models tasked with natural language reasoning problems, teasing apart these two failure modes (namely, deficiencies in language understanding versus deficiencies in knowledge or reasoning) requires more than reporting test set accuracy. the design of natural language reasoning test sets does not facilitate this type of analysis: if a test set contains 100 different natural language reasoning problems, and a model correctly answers 80% of them, which failure mode should we attribute the 20% of errors to? for practitioners, it is useful to characterize performance by measuring paraphrastic consistency alongside accuracy: how likely is it that a model's prediction for a natural language reasoning problem will remain the same given a different phrasing of the problem? we collect a dataset that changes the language of nli examples while maintaining the underlying logic of the problem to tease the two apart. for a test example, x, we collect a set of paraphrases {x 1 , x 2 , ...}, which we call a bucket ( ). after collecting many such buckets, we can directly estimate the probability that a model's prediction for any two paraphrases belonging to the same bucket will be the same. ## constructing paranlu we study paraphrastic consistency by paraphrasing the target portions (t ) of examples from α-nli and all three data sources in δ-nli (δ-snli, δ-atomic, and δ-social). for an original example x, we collect a bucket of paraphrased examples x i such that the context portions c and gold label l remain identical, while the target portions t are rewritten as label-preserving paraphrases. this allows us to modify the surface form of the example while retaining the underlying commonsense reasoning problem. label-preserving paraphrases. we construct quasi-paraphrases @xcite of target sentences in reasoning problems by loosening the requirement of semantic equivalence. given a natural language reasoning task l, a textual instance x of task l with label ℓ l (x), we say that x is a label-preserving paraphrase of x if: ## consistency on human paraphrases we first examine several different models' behavior on paranlu to measure robustness to different linguistic expressions. while language models such as roberta are trained on vast amounts of text that may instill some paraphrastic consistency, especially given label-preserving paraphrases that not be semantically equivalent, other non-pretrained models without access to such knowledge may falter. we characterize the progress of models with respect to p c to understand whether factors such as training setups (training from scratch, supervised finetuning, prompting) and model complexity (ranging from bag-of-words representations to gpt-3) affect consistency. ## paraphrase source human-written paraphrases in paranlu span all of the transformations delineated by @xcite , sometimes involving more complex reasoning that falls between linguistic and world knowledge. such paraphrases were elicited by providing humans the entire example, encouraging them to engage with both the reasoning problem itself and the wide scope of possible meaning-preserving transformations. to understand the utility of label-preserving paraphrases, we compare a roberta model's behavior on our human-written paraphrases with paraphrases generated automatically, as previous studies have explored @xcite . using our roberta models ( §5.1), we probe the relationship between paraphrastic consistency ( p c ) and the source of paraphrased examples. are models more robust to the paraphrastic transformations produced by automatic paraphrase generation models, or do they only struggle with the more complex, example-aware transformations made by humans? since human paraphrases and model paraphrases are generated by different processes, and are thus drawn from different distributions, they may exhibit different properties. paraphrase generation models are predisposed to biases arising from n-gram frequency effects. 4 however, reasoning models should exhibit consistency regardless of whether correct answers are phrased as high-probability sentences under a language model. we use two models ( §6.2) to automatically paraphrase target sentences in original examples and compare model p c on automatic and human paraphrases. ## do artifacts explain inconsistency? many nlp datasets are constructed by crowdworkers writing most or all parts of a natural language reasoning problem. while efficient and scalable, this paradigm can give rise to annotation artifacts @xcite , or statistical biases in parts of examples that correlate with the correct label @xcite . for example, @xcite found that negation words @xcite in nli examples are strong indicators of the contradiction label. one way to detect artifacts in datasets is through a partial-input baseline @xcite , a setting in which only the target portion of an nli instance (i.e., the hypothesis in a traditional nli setup) is used to train a model to predict entailment. when partial-input models achieve high accuracy, it is often indicative of annotation artifacts. full-input models that are trained on datasets with annotation artifacts may learn to rely on such shallow signals instead of performing true inferential reasoning. this may lead to lower paraphrastic consistency in the face of alternative phrasings of examples, since they may no longer contain the annotation artifact the model leveraged. can we attribute a model's issues with paraphrastic consistency on paranlu entirely to the presence of annotation artifacts? that is, are models only inconsistent on examples with artifacts, since they are relying on spurious correlations? or, are they inconsistent even when no artifacts are present? 7.1 experimental setup @xcite find that partial-input baselines trained on δ-nli perform at least 10% better than random chance, indicating the presence of annotation artifacts in their dataset. as such, we focus on δ-nli for our experiments. 5 for each split of δ-nli, we train a roberta-large model using only the update sentence as input, keeping the training hyperparameters identical to the full-input roberta models from §5.1. then, we use these partial-input models to partition buckets in paranlu into two subsets: those on which the partial-input model correctly predicted the label from the update sentence of the original example, indicating that an artifact is likely present in the partial-input full-input artifacts. . . a o a a a o a a p c p c δ-snli likely 100 81.8 54.6 54.3 56.6 85.8 74.7 91.4 unlikely 0 20.7 6.9 45.5 48.8 79.6 75.1 84.0 δ-atomic likely 100 77.6 53.9 55.6 58.3 78.0 76.4 87.4 unlikely 0 21.3 8.3 50.9 49.9 79.2 75.9 86.5 δ-social likely 100 77.2 58.9 52.9 55.5 85.8 73.9 90.9 unlikely 0 28.8 8.4 50 58.7 90.7 74.7 93. 5 ## training dynamics and paraphrastic consistency lastly, we explore the relationship between different parts of the model training pipeline (e.g., pretraining and finetuning) and paraphrastic consistency. how does consistency change as these training processes progress, and does it change in a similar manner as accuracy? is it the case that simply increasing the volume of pretraining or finetuning data linearly impacts paraphrastic consistency? we train a series of roberta models and adjust the number of pretraining tokens ( §8.1) and finetuning examples ( §8.2) to explore how they impact a model's consistency. ## conclusion as more studies investigate the capabilities of llms, the ability to disentangle the effects of paraphrastic variability from other target attributes will be an important analytical tool. this work introduces a new methodology and dataset for measuring paraphrastic consistency, or p c , of models on natural language reasoning tasks. p c captures the probability that a model will remain consistent in its prediction given different phrasings of the same underlying reasoning problem. we design p c as a metric complementary to accuracy, and propose practitioners use it alongside accuracy when diagnosing modeling errors, summarizing a model's performance, or deciding when a model is ready for deployment to users for a particular application. our results confirm that paraphrastic sensitivity is present in all models, but decreases with pretraining volume. because p c only requires model predictions to be labeled as correct or incorrect, our approach can generalize to any task with binary scoring (and where answers must be invariant to paraphrases). future work may consider adapting this approach for tasks with more complex or open-ended evaluations.