Shortcutted Commonsense: Data Spuriousness in Deep Learning of Commonsense Reasoning (2021)

Abstract: Commonsense is a quintessential human capacity that has been a core challenge to Artificial Intelligence since its inception. Impressive results in Natural Language Processing tasks, including in commonsense reasoning, have consistently been achieved with Transformer neural language models, even matching or surpassing human performance in some benchmarks. Recently, some of these advances have been called into question: so called data artifacts in the training data have been made evident as spurious correlations and shallow shortcuts that in some cases are leveraging these outstanding results. In this paper we seek to further pursue this analysis into the realm of commonsense related language processing tasks. We undertake a study on different prominent benchmarks that involve commonsense reasoning, along a number of key stress experiments, thus seeking to gain insight on whether the models are learning transferable generalizations intrinsic to the problem at stake or just taking advantage of incidental shortcuts in the data items. The results obtained indicate that most datasets experimented with are problematic, with models resorting to non-robust features and appearing not to be learning and generalizing towards the overall tasks intended to be conveyed or exemplified by the datasets.

## introduction reasoning helps humans to cope with their experience, whether in a complex situation such as devising a plan to solve a pandemic or in a simple, intuitive inference like what will happen to a coffee mug when dropped to the ground. it helps to foresee new events as well as to form new beliefs and justify and defend them before others @xcite . reasoning with knowledge widely shared by humans, usually termed as commonsense, makes up a significant portion of our higher level cognitive skills: commonsense knowledge encompasses human values and needs, and by reasoning with it we can organize sensible arguments and decide on effective actions. endowing computing devices with these knowledge and reasoning capabilities should allow them to better get access to world view of humans, their needs, capabilities, beliefs, and thus allow them to act more appropriately. to a large extent, acting on the basis of these capabilities can be verbalized, knowledge can be written down, and the inference chain of a reasoning process can be expressed in some human idiom. in natural language processing (nlp), commonsense processing tasks usually involve answering questions that, when addressed by humans, require different types of commonsense knowledge and reasoning to be answered. accordingly, for a model coping with this kind of tasks to achieve good performance, it should have acquired such knowledge and reasoning skills. whether current state-of-the-art nlp deep learning models genuinely grasp the underlying tasks they are handling is a debated topic. recently, an increasing number of published experiments indicate that models may be latching at spurious cues present in the data @xcite @xcite , implying that they will severely lack generalization capacity when presented with out-of-distribution data. as they become more refined, pre-trained language models are continuously closing the gap to humans in commonsense reasoning tasks @xcite @xcite . in light of the skepticism about the true capabilities of deep learning models, this question cannot be avoided: to what extent are they actually learning commonsense reasoning? in the present paper, we seek to contribute to a better understanding about if and how the models are learning commonsense reasoning skills. we se-lect four prominent commonsense reasoning tasks, namely argument reasoning comprehension task (arct) @xcite , ai2 reasoning challenge (arc) @xcite , physical iqa (piqa) @xcite , and commonsenseqa (csqa) @xcite ). we adopt the successful transformer architecture and resort to prominent pre-trained language models: the encoder-only roberta @xcite , the decoder-only gpt-2 @xcite , the encoder-decoder t5 @xcite and bart @xcite , and the neurosymbolic comet(bart) @xcite ). and we set on to gain insight with stress experiments that seek to find evidence that hopefully permits to advance in answering questions like the following: are the models taking into account the actual input as a whole? or could the models be only looking at certain parts of the input, therefore not performing the underlying task but some derivative. how robust are the models? can they withstand adversarial attacks on the basis of powerful generalization they gained during training, or are they brittle and crumble under attack? how well can models perform zero-shot evaluation on each other? to what extent can they transfer the knowledge between each other and how much could that be hindered by their being stuck in learning non-transferring cues based on spurious data artifacts? the answers obtained in this paper indicate that most datasets experimented with are problematic, with models resorting to non-robust features and appearing not to be generalizing towards the overall tasks intended to be conveyed by the datasets. in the next section 2, we cover related work. the tasks experimented with are presented in section 3, and the experiments undertaken are described in section 4. section 5 presents and discusses the results obtained. the paper closes with concluding remarks in section 6. 1 2 related work nlp has found an overarching yet flexible approach to a wide range of its processing tasks in the transformer model @xcite . an extensive research path has since then been pursued in 1 our code is publicly available at: https://github.com/nlx-group/ shortcutted-commonsense-reasoning . the literature, seeking to refine its architecture and training methodology. it has become commonplace to first pre-train a language model on large corpora and then refine it with respect to a specific language processing task @xcite @xcite . subsequent works would refine this methodology by introducing other pre-training tasks that benefit downstream performance @xcite @xcite , and with that approach steadily increasing the state of the art on benchmarks such as glue @xcite and superglue @xcite . shortcuts deep neural networks, such as transformers, as well as loss functions (crossentropy) and optimizers, tend to favor simple functions @xcite @xcite @xcite , which can be susceptible to wrong generalizations as the eventual models "hook" onto spurious artifacts in the data. this has been shown to happen in nlp, as illustrated with these examples, among others: even when using novel pre-trained language models, such as bert, which have more knowledgeable priors due to their pre-training regime, such phenomena persists. @xcite studied the results from the semeval 2018 task 12 -argument reasoning comprehension task (arct) @xcite . bert shined, obtaining 77% test accuracy, slightly below untrained humans, who reach 80%. these authors found that the presence of "not" and other high-frequency words such as "is", "do" and "are" was highly correlated with the output, obtaining above random performance with just "not". adding adversarial examples that counteracted this correlation, the strong spurious signal disappeared, and the performance dropped to random chance. data contamination a different type of "cheating" has also attracted some attention recently. pre-trained language models appear to be able to use (factual) knowledge encoded in its parameters during fine-tuning.in one study, the authors were able to show that in open-book q&a challenges, large pre-trained language models can be competitive with systems that access external knowledge sources by just accessing their internal "memory" @xcite . this memorization power is a problem if the task dataset and the pre-training corpus have been constructed from the same sources, and thus have some text in common. the authors of gpt-2 @xcite creates bloom filters from webtext, its pre-training corpus, and calculatse an upper bound for text collisions between downstream tasks and webtext, labelled as data contamination. it was found that due to text overlap, the model gains small but consistent benefits, arguably due to memorization. in the next iteration, gpt-3 @xcite gave rise to larger-scale data contamination experiments with exact n-gram matching (instead of bloom filters). the results varied, with some datasets being completely clean and others worryingly contaminated. popular machine reading datasets such as quac @xcite , squad 2 @xcite or drop @xcite are flagged @xmath0. piqa @xcite was flagged with 29% contamination, however, removing the contaminated text only decreases the performance by 3%, regardless of model size. the authors see this as a sign that memorization may not be at play but rather statistical cues in the data, though they did not offer empirically based support to that conjecture. ## tasks we adopt four commonsense related tasks, covering different domains and demands in terms of reasoning, setting a challenging environment to probe the capacity of models addressing them. figure 1 provides a dataset example for each of the tasks described in this section. appendix b describes the dataset size for each task. ## methodology in the present paper, the tasks described above are assessed under the experiments described below. ## conclusion commonsense is a quintessential challenge to artificial intelligence and deep learning techniques have been delivering important performance scores and progress in nlp tasks and benchmarks that require commonsense reasoning. in this paper, we set to assess how much of these gains are due to the capacity of generalization beyond the training datasets supporting these tasks and whether these advances may be suffering from the problem that recently has been uncovered in some other areas of nlp, namely that the outstanding models may owe much of their prowess to spurious shortcuts in the data rather than to robust generalizations that permit them to sustain or even approximate their level of performance beyond their benchmark datasets. to pursue this goal, we focused on a number of tasks involving commonsense reasoning and their datasets that have been widely used in the literature and submit them to stress tests with the potential of helping to uncover data spuriousness problems. in a first sign that such problems are most probably present, we found that most models do much better than expected without being supplied with all the segments in a given input, even approximating their performance when they are provided with the full input, namely in the case of arct, arc and piqa tasks, while csqa shows the expected degraded performance. further experiments were consistent with this pattern. again, the three tasks, arct, arc and piqa, are also clearly more sensitive to adversarial attacks than the latter. also csqa stands out in its generalization capacity in comparison to the lower capacity of the other three tasks to generalize as it provides much better approximation to other tasks in a zero shot setting. additionally, with the help of data contamination tests, these contrasts and problems were shown not to result from possible data contamination, leaving one with the most plausible justification that they are rather due to problems of data spuriousness that support shortcutted learning of commonsense reasoning, with reduced generalization capacity. these results call for future research on a careful review and comparison between the methods and procedures used in the development of these datasets in order to learn how to avoid the pitfalls of producing datasets affected by data spuriousness. but the present results are already very useful in as much as they let one knows that the performance obtained with the datasets studied in the present paper, for which there appear a consistent indication of data spuriousness, should be taken with an extra grain of salt.