Dive into the Chasm: Probing the Gap between In- and Cross-Topic Generalization (2024)

Abstract: Pre-trained language models (PLMs) perform well in In-Topic setups, where training and testing data come from the same topics. However, they face challenges in Cross-Topic scenarios where testing data is derived from distinct topics. This paper analyzes various PLMs with three probing-based experiments to better understand the reasons behind such generalization gaps. For the first time, we demonstrate that the extent of these generalization gaps and the sensitivity to token-level interventions vary significantly across PLMs. By evaluating large language models (LLMs), we show the usefulness of our analysis for these recent models. Overall, we observe diverse pre-training objectives and architectural regularization contribute to more robust PLMs and mitigate generalization gaps. Our research contributes to a deeper understanding and comparison of language models across different generalization scenarios.

## introduction probing @xcite is widely used to analyze pre-trained language models (lms) @xcite @xcite . it enables a better understanding of how lms encode information and how it evolves in the architecture by studying linguistic properties such as part-ofspeech or dependency-tree parsing @xcite . however, probing methods @xcite @xcite mainly rely on the general in-distribution (id) scenario, where we distribute train and test instances independent and identically. as a result, other more realistic out-of-distribution (ood) scenarios @xcite , like generalizations regarding forthcoming figure 1 : generalization gap of fine-tuning lms on argumentative stance detection @xcite in the in-or cross-topic evaluation setup. the dashed line marks the ideal case of equal performance. addressing this research gap, we propose -for the first time -a probing-based approach to comprehensively analyze lms in a challenging ood setup. more precisely, we rely on cross-topic foot_0 evaluation where we deliberately withhold instances from specific topics for testing. following @xcite , we define topic as the query used to compose a specific datasetsuch as arguments covering gun control or marijuana legalization. this evaluation setup is highly relevant for challenging argument mining (am) downstream tasks @xcite . it allows for simulating, in a controlled setup, how well lms handle topic-shifts when unseen semantic features (such as topic-specific vocabulary) arise in future and new topics. previous studies found that cross-topic argument mining is challenging compared to the in-topic setup @xcite . the major reason lies in the apparent generalization gaps between randomly composing training and testing data (in-topic) and using distinct groups of topics for training and testing @xcite . figure 1 shows such performance gap when fine-tuning on the ukp argmin dataset @xcite -labeling arguments as in favor, against, or neutral to one of eight topics. notably, we observe gaps between in-and cross-topic varying considerably across lms -with bart outperforming the others in the cross-topic setup. such inconsistencies underline the need to investigate such crucial generalization capabilities. thus, we propose extensive probing-based experiments to examine the gap between in-and cross-topic generalization and show that embedding spaces of lms vary considerably regarding their generalizability and robustness. in detail, we propose three probing-based experiments to answer the following research questions, considering three linguistic probes (dependency-tree parsing, partof-speech tagging, and named-entity recognition) based on ukp argmin dataset: how do generalization gaps of lms differ after pre-training? ( § 4) we find generalization gaps substantially differ across lms while becoming more prominent for tasks with more semantically difficulties, such as ner. in addition, we crucially observe that probing generally underperforms on lexical unseen instances (like highly rare entities), and deduplicating pre-training data provides more robust embedding space when evaluating larger and more recent lms. how do lms depend on topic-specific vocabulary? ( § 5) next, we assess the influence of topicspecific tokens by removing them using amnesic probing and lms significantly differing in their reliance on and robustness concerning such semantic features. interestingly, pre-training objectives or architectural regularization influence robustness, suggesting their potential importance in building robust and competitive lms. how do generalization gaps evolve during finetuning? ( § 6) finally, we re-probe tuned lms on the ukp argmin dataset and find that in-topic fine-tuning erases more linguistic properties than cross-topic fine-tuning. to sum up, we expand the probing scope to cross-topic generalization and highlight probing as a universal tool complementing the study of language models beyond general evaluation setups. while we focus on an in-depth analysis of in-vs. cross-topic generalization gaps, our experimental setup generalizes to other types of ood scenarios where one verifies generalization regarding other text genres (like the social media domain), languages, or temporal changes in the languages @xcite @xcite . ## in-and cross-topic probing the following section formally outlines the probing setup and tasks before elaborating on the generalization gap and comparing the evaluation of in-and cross-topic probing. ## experimental setup we propose three experiments to analyze the varying generalization gap between lms after pretraining ( § 4), their dependence on topic-specific vocabulary ( § 5), and the evolution of these gaps during fine-tuning ( § 6). we outline general details about these experiments, while details and results are provided in the subsequent sections. ## the generalization gap of lms the first experiment shows that the generalization gap already exists after pre-training and varies regarding specific lms and probing tasks. we analyze general ( design we probe eight lms on the probing tasks dep, pos, ner, and stance and verify them by observing significant performance drains using random initialized lms (appendix § b.2). for a holistic evaluation, we provide general results and group instances into two categories: seen and unseen. we define seen instances as already processed during training but in another context. for example, the pronoun he might appear in both training and test data, but in distinct sentences. by evaluating the lms on seen instances, we gain insights into the influence of token-level lexical information versus context information from surrounding tokens. in contrast, unseen instances were not encountered during the training of a probe. they allow assessing whether lms generalize to tokens that are similar to some extent (such as berlin and washington) but not seen during training. results for evaluation setups upon analyzing dep pos ner all ∆ seen ∆ unseen all ∆ seen ∆ unseen all ∆ seen ∆ unseen instance ratio -85% 15% -86% 14% -65% 35% in-topic albert 43.8 +0.21 -3.2 80.2 +0.41 -17.7 48.6 +1.1 -5.8 bart 36.5 +0.13 -3.0 75.4 +0.20 -16.5 48.7 +1.3 -7.0 bert 25.4 -0.02 -0.8 68.5 +0.20 -16.5 45.4 +1.0 -5.8 deberta 32.8 +0.07 -1.5 73.7 +0.09 -12.7 48.8 +1.0 -5.6 roberta 25.1 -0.01 -0.9 64.0 -0.04 -15.5 48.4 +1.0 -5.7 average --0.08 -1.9 -+0.17 -15.8 -+1.1 -6.0 instance ratio -78% 22% -81% 19% -51% 49% cross-topic albert 39.5 +0.03 -2.3 78.0 +0.51 -12.9 45.8 +2.2 -5.3 bart 36.9 +0.01 -4.0 74.1 +0.24 -16.5 45.3 +2.4 -5.8 bert 25.6 -0.09 -0.7 67.5 +0.20 -14.0 41.6 +1.9 -5.1 deberta 29.9 -0.07 -1.3 74.6 +0.14 -11.7 42.4 +2.0 -5.2 roberta 23.6 -0.22 -0.3 65.5 +0.00 -14.7 42.1 +1.9 -5.2 average --0.08 -1.7 -+0.22 -14.0 -+2.1 -5.3 results for probing tasks considering next, we separately compare tasks for seen and unseen instances. dep shows the slightest performance difference compared to all. we assume that the pairwise nature of the task leads to a larger shared vocabulary between unseen and training instances -since a pair can be unseen, but it may contain a frequent word like of. in contrast, apparent differences between ner and pos are visible -with less performance drain on unseen instances for ner than pos. therefore, we assume for ner a higher semantic overlap with training instances since they could include -as being an n-gramwords from the training vocabulary. in contrast, tokens of unseen pos instances are always single words; thus, we assume a smaller semantic overlap with the training. results for encoding models we now compare lms amongst themselves. the four bestperforming lms of in-topic differ up to 7.6 (al-bert -bert), while for cross-topic, this difference narrows to 4.1 (albert -electra). these results confirm the varying generalization gap between them and, again, that we can not transfer conclusions from one evaluation setup to another. for example, the probing performance of bart for in-topic stance is the best and the third best for cross-topic. generally, we do not see a clear correlation between better average performance and a smaller generalization gap. lms like deberta perform better for in-and cross-topic but show a bigger gap (-5.1) compared to lower performing lms like electra (-1.0), but there are also worse lms with a bigger gap (gpt-2, -5.6) or better ones with a smaller gap (albert, -4.6). overall, we see the generalization gap being more pronounced for better-performing lms. considering absolute performance, al-bert and bart performs the best for both evaluation setups, while electra excels pos and dep, and deberta performs for ner and stance. in contrast, bert, roberta, gpt-2, and gloveunderperform the others. thus, lms with architectural regularization, such as layer-wise parameter sharing (albert), encoder-decoder layers (bart), disentangled attention (deberta), or discriminator (electra), tend to provide higher cross-topic performance. similarly, albertor debertagenerally achieve more performance gains for seen instances and fewer performance drops for unseen ones than models without regularization such as bert or roberta. we hypothesize that architectural and regularization aspects give lms a more generalizable and robust encoding space. ## the dependence on topic-specific vocabulary to this point, we saw that the generalization gap varies between different lms and probing tasks. since topic-specific vocabulary crucially affects generalization gaps, we analyze the varying dependence on the topic-specific vocabulary of lms using amnesic probing @xcite . we observe apparent differences among lms and assume their embedding space clearly differs beyond single evaluation metrics. therefore, we emphasize considering various lms when using amnesic probing. additional insights of comparing seen and unseen instance and distinct ner classes are provided in the appendix ( § b.4, § b.6). design to measure how lms depend on topicspecific vocabulary, we employ amnesic probing @xcite to remove the latently encoded topic-specificity z i from the embeddings h i of a token w i . more precisely, we compare how the performance of a probing task (like ner) changes when we remove z i . a more negative effect indicates a higher dependence on topic-specific vocabulary, while this property is a hurdle when performance improves. we first train a linear model on token-level topic-specificity r ( § 2.3). to shape it as a classification task, we categorize r into three classes (low, medium, high). 5 next, we find a projection matrix p that projects all embeddings h i -gathered as h -using the learned weights w l of l to the null space as w l p @xmath0. using p we update h i by neutralizing topic-specificity from the input as h ′ @xmath1. following @xcite , we verified our results by measuring less effect of removing random information from h i (see appendix § b.3). ## the evolution of the generalization gap during fine-tuning finally, we re-evaluate fine-tuned lms using our proposed probing setups and show that fine-tuning leads to a drain in probing performance. we use these results to retrace apparent differences between evaluation setups and the varying generalization gap between lms. this is relevant for a broader understanding of how fine-tuning affects lms @xcite , and what they learn during fine-tuning @xcite . design we fine-tune the lms on an argumentative stance detection task and re-evaluate them on dep, pos, and ner probing tasks. to be consistent with our probing setup, we used the same folds for fine-tuning. further details are in the appendix ( § a.5). we compare these results with the probing performance of their pre-trained counterparts ( § 4 and § 5) and correlate this change with the generalization gap observed on the downstream task. we limit our analysis to albert, bert, bart, de-berta, and roberta. ## related work the rise of lms @xcite @xcite enabled big success on a wide range of tasks @xcite . nevertheless, they still fall behind on more realistic cross-topic, like generalizing towards unseen topics @xcite @xcite . one primary reason is that lms often rely on unwanted spurious correlations. despite lms seeing such vocabulary during pre-training, they failed to consider test vocabulary in the required fine-grained way @xcite . further, kumar et al. (2022b) found linear models can outperform finetuning lms when considering out-of-distribution data. thus, a broader understanding of lms in challenging evaluation setups is crucial. probing @xcite @xcite helps to analyze inners of lms. this includes to examine how linguistic @xcite , numeric @xcite , reasoning @xcite , or discourse @xcite properties are encoded. other works focus on specific properties used for other tasks @xcite , or finetuning dynamics @xcite @xcite . however, these works target the commonly used in-topic setup and less work considering cross-topic setups. @xcite analyzed metaphors across domains and language, or @xcite crossdistribution probing for visual tasks. they found that models generalize to some extent across distribution shifts in probing-based evaluation. nevertheless, these works focus on specialized tasks and consider the generalizations across distributions in isolation. in contrast, we propose with our experiments a more holistic probing-based evaluation of lms, covering different generalization aspects after pre-training and fine-tuning. ## conclusion discussion we analyzed and compared in-and cross-topic evaluation setups and found generalization gaps significantly differing regarding specific lms and probing tasks. 6 further, we make various crucial observations contributing to a better understanding of the generalizability of lms: to conclude, we highlight the practical utility of probing to analyze and compare the capacities of various lms from a different perspective -considering different generalization scenarios. thereby, our work points out the importance of probing as a universally applicable method, regardless of size or being static or contextualized, to complement existing work on analyzing language models @xcite . outlook with our findings in mind, we regularly see probing lms and large lms and consider forthcoming learning paradigms as indispensable for a holistic evaluation of their verity and multiplicity. therefore, we will continue to analyze language models, including a broader set of tasks and focus-ing on general and rare vocabulary to increase our understanding of how, why, and where they differ.