Supervised and Unsupervised Probing of Shortcut Learning: Case Study on the Emergence and Evolution of Syntactic Heuristics in BERT (2025)

Abstract: Contemporary language models (LMs) such as BERT (Devlin et al., 2019, T5 (Raffel et al., 2023), GPT-4 (OpenAI, 2023), have exhibited remarkable capabilities, effectively addressing long-standing challenges in the field. However, these models rely on shortcut learning, using a decision rule that relies on superficial cues that are spuriously correlated with the labels (Geirhos et al., 2020). In this research, we focus on the reliance on a specific type of shortcuts, namely syntactic heuristics, in BERT when performing Natural Language Inference (NLI), a representative task in Natural Language Understanding (Jeretic et al., 2020). By making use of two probing methods, one supervised, one unsupervised, we investigate where these shortcuts emerge, how they evolve and how they impact the latent knowledge of the LM. Our findings reveal that syntactic heuristics are absent in pretrained models but emerge and evolve as the model is finetuned with datasets of increasing size. The adoption of these shortcuts varies across different hidden layers, with specific layers closer to the output contributing more to this phenomenon. Despite the model’s reliance on shortcuts during inference, it retains information relevant to the task, and our supervised and unsupervised probes process this information differently.

## introduction probing the dynamics of shortcut learning. contemporary neural network-based language models (lms) such as bert @xcite , t5 @xcite , @xcite , have exhibited remarkable capabilities, effectively addressing long-standing challenges in the field. despite these achievements, and the development of increasingly larger models that excel on various benchmarks, certain limitations and risks have recently demanded increased attention. high scores on standard benchmarks do not necessarily indicate that a model actually possesses the underlying generalization capabilities and knowledge essential for such performance @xcite @xcite . for instance, browning and lecun demonstrate that the outstanding performance of lms on the winograd schema challenge is not driven by a genuine understanding of language, world knowledge and common sense reasoning, capabilities originally deemed necessary to solve the challenge. across various fields and applications, researchers have observed that while a model may achieve impressive results on standard benchmarks, this performance does not generalize well across different datasets. specifically, there is a substantial drop in performance on out-of-distribution (o.o.d.) datasets. geirhos et al. identify this as a manifestation of shortcut learning, where models rely on superficial strategies that work well on standard benchmarks but fail when faced with more complex testing scenarios. essentially, shortcut learning occurs when models rely on surface-level cues to complete tasks, instead of learning to reason about the task. the problem of shortcut learning in lms can result from many different factors, including the limitations of the training dataset, the model architecture and size, pretraining objectives and finetuning procedures @xcite . in this paper, we investigate shortcuts used in natural language inference (nli) tasks, where the aim is to determine if one sentence (hypothesis) is entailed by another sentence @xcite . by combining two probing methods, one supervised, one unsupervised, we investigate where these shortcuts emerge, how they evolve and how they impact the latent knowledge of the lm. research questions and contributions. to investigate the adoption and development of these syntactic heuristics and examine their influence on the knowledge embedded within the hidden repre-sentations, we address the following research questions: ## related work syntactic heuristics solving nli tasks typically requires syntactic and semantic understanding, common sense reasoning and world knowledge. however, @xcite , @xcite , @xcite and @xcite demonstrated that lms finetuned on nli datasets fail to build on these requirements, and instead rely on the use of shortcuts. consequently, they have learned to solve the dataset instead of solving the task @xcite . among these shortcuts are the syntactic heuristics identified by mccoy et al.: the lexical overlap heuristic, subsequence heuristic and constituent heuristic. the lexical overlap heuristic assumes that a premise entails all hypotheses constructed from words in the premise. 1 the subsequence heuristic suggests that a premise entails all its contiguous 1 for example, the lexical overlap heuristic would correctly assume that the premise 'the pupil was asked by the teacher' entails the hypothesis 'the teacher asked'. however, this hypothesis would incorrectly assume that the premise entails the hypothesis 'the pupil asked the teacher'. probing the dynamics of shortcut learning our approach focuses on the layerwise evolution of shortcuts and uses a combined methodology of supervised and unsupervised probing. it builds upon the foundational studies on the reliance on shortcuts in nli, as demonstrated in @xcite , as well as broader research on shortcut learning in lms @xcite @xcite . more recently, @xcite have probed causality manipulation hierarchically by introducing different shortcuts to models and observing their behaviors, while @xcite have investigated lms' reliance on shortcuts or spurious correlations within prompts. @xcite demonstrate how more recent, larger generative models exploit spurious correlations for predictions. unsupervised probing for latent knowledge for our unsupervised probe, we build on the work of @xcite and use contrast consistent search (ccs). ccs is built around a key characteristic of the notion of truth: logical consistency, a sentence and its negation cannot both be true. assuming that a model internally evaluates the truth value of an input, this can in principle be recovered from the hidden representations. ccs learns a linear transformation of the hidden representations such that the transformed space embeds this logical consistency @xcite . the input sentences are converted into contrast pairs, a sentence and its negation, foot_2 and passed through the model. the learned linear projection is applied to the generated hidden representations, enabling the model to predict opposite labels for the two sentences of the pair. this classification reflects whether the representations correspond to a true or false statement. ccs scores quantify the number of statements that are correctly identified as 'true' or 'false'. ## methodology this case study focuses on a single model, a specific finetuning dataset, and one challenge set, enabling a controlled analysis of both the method's effectiveness and a concrete instance of shortcut learning. we conduct the study on bert, an open-source lm widely used and studied, particularly in interpretability research, where the field of "bertology" examines its behavior and intermediate representations in relation to linguistic properties @xcite . given its prominence, bert is a natural starting point for testing this new methodological paradigm. concretely, in a first step we finetune bert @xcite with a binarized version ('entailment' and 'contradiction' examples only) of multinli @xcite . following finetuning, we convert mnli and hans examples into contrast pairs. foot_3 these contrast pairs are fed into the finetuned models, and the accuracy scores of our supervised probe, logistic regression (lr), and usupervised probe, ccs, are measured. to obtain lr scores, we use the representations of the contrast pairs, along with the true labels. the hidden states are split into train and test and lr accuracy is measured on the test set. having access to the labels makes this approach similar to a regular supervised probe. by comparing the -supervised-lr accuracies for mnli and hans examples, we can determine whether the model relies on the syntactic heuristics. since these heuristics fail on hans examples, consistently lower scores on the hans examples than on mnli examples provide evidence for the reliance on those heuristics. the gap between both scores (lr mnli versus lr hans) can be taken as a measure of the reliance on syntactic heuristics. we examine this gap in models fine-tuned with datasets of different sizes and measure the progression of this difference across the hidden layers. we identify which layers have the most significant impact on the adoption of syntactic heuristics and whether the presence of these heuristics increases towards the layers closer to the output layer. to obtain ccs scores, we use the same contrast pair representations without the labels, and apply the learned linear projection, enabling the model to predict opposite labels for the two sentences of the pair. ccs scores quantify the number of statements that are correctly identified as 'true' or 'false'. as such, the gap between the hans and mnli based ccs scores indicate whether the syntactic heuristics are inherently encoded within the hidden representations. thereby we demonstrate that ccs can resolve nli questions in an unsupervised manner, supporting its effectiveness as a probing method. additionally, we demonstrate the complementary nature of ccs and lr. with a supervised probe like lr, the performance could be attributed to the probe's learning capacity. by contrast, ccs is unsupervised, which allows us to access information inherently present in the model's representations. ccs shows which patterns are genuinely latent versus those that only appear through probe training. the final research question explores the extent to which the learned heuristics have altered the model's internal knowledge and its representation of the truth foot_4 of the input sentences. to address this question, we examine the knowledge in the hidden representations of the finetuned models, reflected in the gap between mnli and hans lr and ccs scores, and compare this to inference results. ccs was initially conceived as a prototype to explore unsupervised methods for identifying the beliefs foot_5 of a model @xcite . comparing ccs to lr and inference results, enables us to investigate whether a significant gap exists between the knowledge the model possesses and what it outputs. ## experiment and results to examine whether the degree of reliance on syntactic heuristics changes with the size of the dataset used for finetuning, we finetuned bert 8 @xcite progressively. ## conclusions in this research, we traced the evolution of the adoption of syntactic heuristics as a decision rule for nli classification, adopted by bert, finetuned on mnli datasets. to identify the learned heuristics, we used a challenge dataset, hans, and combined supervised and unsupervised probing methods. our findings demonstrate the progressive adoption of these shortcuts as a result of finetuning dataset size, observable across the model's hidden layers, with specific layers closer to the output contributing more to this phenomenon. our study illustrates the need for finetuning with diverse datasets, including examples where shallow heuristics fail. a deeper understanding of the inductive biases of lms and ongoing research into o.o.d. challenge datasets can aid in ensuring the diversity of training data. despite the model's reliance on shortcuts during inference, it retains information relevant to the task, and our supervised and unsupervised probes process this information differently. a key challenge remains in effectively leveraging this latent knowledge and encouraging the model to reason over it. further research is needed to uncover the underlying mechanisms and explore strategies to harness this knowledge. one of the key contributions of this paper is therefore methodological: integrating results obtained with supervised and unsupervised probing with inference results mitigates individual limitations of these approaches.