Measuring the Robustness of NLP Models to Domain Shifts (2024)

Abstract: Existing research on Domain Robustness (DR) suffers from disparate setups, limited task variety, and scarce research on recent capabilities such as in-context learning. Furthermore, the common practice of measuring DR might not be fully accurate. Current research focuses on challenge sets and relies solely on the Source Drop (SD): Using the source in-domain performance as a reference point for degradation. However, we argue that the Target Drop (TD), which measures degradation from the target in-domain performance, should be used as a complementary point of view. To address these issues, we first curated a DR benchmark comprised of 7 diverse NLP tasks, which enabled us to measure both the SD and the TD. We then conducted a comprehensive large-scale DR study involving over 14,000 domain shifts across 21 fine-tuned models and few-shot LLMs. We found that both model types suffer from drops upon domain shifts. While fine-tuned models excel in-domain, few-shot LLMs often surpass them cross-domain, showing better robustness. In addition, we found that a large SD can often be explained by shifting to a harder domain rather than by a genuine DR challenge, and this highlights the importance of TD as a complementary metric. We hope our study will shed light on the current DR state of NLP models and promote improved evaluation practices toward more robust models.

## introduction modern transformer-based nlp models, and particularly large language models (llms) have proven effective on various tasks and evaluation setups, including fine-tuning @xcite and in-context learning @xcite . following that, there has been an improvement in the models' ability to perform tasks while transferring to domains in the classic and observed scenarios, we observe a 15point drop between the source in-domain performance (ss) and the cross-domain performance (st). conversely, in the unobserved and no challenge scenarios, @xmath0, meaning the model gains 15 points upon domain shift. we would typically conclude that there is a dr challenge only in the first two scenarios. however, we argue that this commonly adopted perspective is inaccurate since it overlooks the target indomain performance (tt). our work provides a fresh perspective by considering both degradation metrics: the source drop (sd) and the target drop (td). with no labeled data available @xcite @xcite . despite these improvements, the performance upon domain shift can still be inferior to the model's performance on the source domains, a problem we refer to as the domain robustness (dr) challenge @xcite @xcite . research of dr is quite disparate: a wide variety of setups, models, training procedures, and different dataset sizes are used. there is also a severe lack of variety in evaluation tasks for dr: most papers use classification tasks, omitting important tasks such as sequence tagging, question answering, and text generation @xcite . moreover, many past works use challenge sets to measure the dr challenge. these are highly curated datasets that select synthetic @xcite or particularly hard samples for models to process under domain shifts @xcite . all this makes it hard to compare different works and map out the extent of the dr challenge in a natural domain shift setting. moreover, prior works focused solely on finetuned models, disregarding few-shot setups that have become prominent. 1 in those setups, the dr challenge manifests itself more moderately: no training data can potentially anchor the model to the source distribution, but only a few demonstrations from the source domain are used in the prompt @xcite . adding to the above, we observe a fundamental problem with how we examine the dr challenge. let us conduct a thought experiment, illustrated in figure 1 : a model is trained and tested on data from domain a (@xmath1, achieving a score of 90, but when tested on domains b and c, it scores 75. the observed 15-point drop typically leads to the conclusion the model lacks robustness, a common assertion in dr papers. but what if we were told that "had the model been trained and tested on data from b, it would have achieved a score of 80", would we still consider it as facing a severe dr challenge, given only a 5-point drop from b's indomain performance (@xmath2, rather than 15? furthermore, if the model attains a score of 70 when trained and tested on domain c (@xmath3, can we still assert a dr challenge exists even when it performs better cross-domain (@xmath4 building on the insights from the thought experiment, our paper introduces a novel perspective on the dr challenge. traditional approaches typically focus on the source drop (sd), assessing how model performance degrades compared to its source in-domain performance. however, this view overlooks the degradation compared to the setup where the model had been trained and tested on the target domain, which we define as the target drop (td). we study these variables and build various metrics upon them in §3. importantly, most works focus solely on the sd and overlook the td, resulting in a partial depiction of the dr challenge. for instance, in studies involving challenge sets that report a large sd, the drop may be primarily attributed to shifting to a harder domain (@xmath5, see §3.1), and not by a genuine dr challenge, e.g., the classic and observed scenarios in figure 1 . by incorporating both metrics, we aim to provide a more holistic and accurate understanding of the dr challenge. to overcome deficiencies in the current body of research, in §4 we introduce a novel dr benchmark. unlike existing benchmarks, which largely rely on synthetic, adversarial, or challenge sets that may not adequately represent natural settings, our benchmark is unique and possesses four key properties: (i) it focuses on shifts (such as topical shifts) that naturally occur in real-life scenarios; (ii) it covers a wide variety of nlp tasks, more than other studies, including sequence and token level classification, qa, and generation tasks; (iii) each task consists of several domains; and (iv) each domain has a sufficient amount of labeled data, enabling its use as a source and as a target domain. following that, we conduct an extensive study by benchmarking many fine-tuned models and fewshot llms, detailed in §5. we examine factors such as the model size, dataset size, number of few-shot demonstrations, and more. our findings, reported in §6, incorporate results of more than 14,000 domain shifts of 21 models and various training and testing setups. our main findings are: 1. fine-tuned models suffer from drops upon domain shifts. while the extent of the drop varies, challenging shifts are prevalent in every task; 2. increasing the size of fine-tuned models enhances both in-domain and cross-domain performance while reducing performance drops, particularly in classification tasks; 3. few-shot models also face a dr challenge as the domain of the demonstrations impacts their performance. however, the domain shift effect for few-shot models is weaker and more nuanced; 4. increasing the fine-tuning dataset size as well as the number of few-shot demonstrations enhances in-domain and cross-domain performance but can also mildly increase the drop due to stronger "source domain anchoring"; 5. while fine-tuned models excel in-domain, fewshot llms often surpass them cross-domain, showing better robustness and smaller drops; 6. considering only one metric can lead to wrong conclusions since many domain shifts are not classic, and only one drop metric (sd or td) is positive while the other is negative; 7. we found that a large sd can often be explained by shifting to a harder domain, and not by a genuine dr challenge; 8. our focus on many natural domain shifts reveals that while challenge sets are helpful diagnostic tools, they tend to overestimate the severity of dr, which is generally milder; in conclusion, we show that thoroughly assessing dr in nlp models requires evaluating multiple domain shifts and incorporating both drop metrics (sd and td). we manifest that while nuanced, the dr challenge is still prevalent. in §7, we delve into the implications of our findings for the nlp community. in appendix §a, we present a theorem that elucidates some of our findings regarding the relationship between the dr metrics. we hope this work will provide a fresh perspective on model robustness and facilitate further research. ## related work the term dr generally refers to the extent to which the performance of a model does not degrade when applied to newly collected samples from other domains. in some cases, robustness refers to consistency (low variance) @xcite . literature on robustness in nlp can be categorized by the type of distribution shift examined: synthetic and natural @xcite . synthetic shift works include adversarial attacks @xcite , input perturbations @xcite , counterfactual @xcite , diagnostic @xcite and challenge (or contrast) sets @xcite . these works assess robustness using datasets designed to challenge nlp models rather than represent a natural language distribution. while the synthetic shifts are helpful diagnostic tools @xcite , they do not accurately depict the actual state of dr "in the wild". hence, we focus on natural domain shifts. natural shift study focuses on organic scenarios where a discrepancy exists between the training and deployment data. these studies encompass various setups, including medium shift @xcite , temporal shift @xcite , and domain shift (e.g., to medical @xcite and legal @xcite . researchers proposed various benchmarks to evaluate the robustness of nlp models and the quality of solutions, including domain shifts in a single nlp task @xcite @xcite @xcite @xcite , with challenge sets @xcite @xcite or only with fine-tuned models @xcite @xcite . our study addresses a broad range of domain shifts in many more nlp tasks than previous work, including sequence and token-level classification, qa, and generation. in addition, we examine both small fine-tuned models and few-shot llms. importantly, unlike other works, which focused on the source drop, we also consider the target drop, providing a more holistic perspective on dr. to the best of our knowledge, this is the most comprehensive dr study in nlp. ## domain robustness domain is a widely used term in nlp that typically refers to a cohesive corpus or dataset, which may be characterized by factors such as topic, style, genre, syntax, linguistic register, and medium. although 'domain' lacks a clear and consistent definition @xcite , we formally describe a domain d by a joint distribution p d (x, y ) over x (the input space) and y (the outcome space). in a domain shift, the source domain s, and the target domain t differ in their underlying joint distribution p s (x, y ) ̸ = p t (x, y ). given a training set of examples from the source domain s ∼ s, the goal of the nlp model is to learn p s (x, y ) (or p s (y |x)), and to the generalize to the (potentially unknown) target domain distribution(s) in which it will be deployed, p t (x, y ). to evaluate the performance on the target domain, we use a test set t ∼ t , which is unobserved during training. we use the term domain robustness (dr) to describe the inherent (in)ability of an nlp model to generalize from the source domain to the target domains. for fine-tuned models, the dr challenge arises when the test data comes from a domain that is different from the labeled training data. meanwhile, few-shot models face the dr challenge when the domain of the demonstrations used in the prompt differs from that of the target data. ## the domain robustness benchmark in sections 1 and 2, we identified shortcomings in existing dr benchmarks. these include an overemphasis on challenge sets and synthetic datasets, coupled with neglecting key nlp tasks such as token-level classification, qa, and particularly generation tasks. to our knowledge, this is the first dr study focusing on various generation tasks, which have gained prominence with the widespread use of llms and genai. moreover, most benchmarks consider only a single or very few domains and often use target domains with only test splits, preventing measuring target drops. these limitations restrict a complete understanding of the state of the dr challenge in "natural settings". to bridge these gaps, we curated a novel dr benchmark that focuses on natural shifts and covers seven downstream tasks. each task consists of several domains with the same amount of labeled data, enabling using any domain as a source or a target and computing the metrics from §3. ## experimental setup fine-tuning models for classification tasks (sa, nli, ab, qa) we employ encoder-only models. specifically, we use roberta @xcite and deberta-v3 @xcite , as well as the smaller distilbert @xcite . for conditional generation tasks (qg, as, tg), we utilize two common encoder-decoder models: t5 @xcite and bart @xcite . we chose these open-source models because they offer a variety of sizes (see zero-shot and few-shot llms we examine llms with an api, including gpt3.5 (turbo) and gpt4 @xcite , as well as the open-sourced llms llama v2 @xcite , orca v2 @xcite (which is based on llama v2 and fine-tuned using signals from gpt4), mistral-7b @xcite and neuralchat @xcite (which is based on mistral and fine-tuned using the orca dataset @xcite ). for each test example from a target domain, the llm receives an input comprising a task instruction and the example. in few-shot setups, the input is augmented with additional demonstrations from the source domain. task instructions and prompt examples are provided in appendix §e.1. due to the high costs of api calls and the quadratic increase in the number of experiments with the number of domains, we limit our presentation of few-shot results to three domains and 600 examples for each task (see appendix §d.3). metrics for classification tasks (sa, nli, ab, qa) we report the f1 score. for generation tasks (qg, as, tg) we report the bertscore @xcite with a pre-trained sbert model @xcite . please see our note in §8.l1. ## discussion in this work, we study the dr challenge in modern nlp models. to this end, we constructed a new dr benchmark comprising various nlp tasks and domain shifts. we proposed shift-level and task-level metrics for precise evaluation and benchmarked numerous fine-tuned models and few-shot llms while examining the effect of multiple factors, such as model size, dataset size, number of demonstrations, and more. our extended discussion in appendix §b delves into the key implications of our findings. specifically, our comprehensive study highlights the need for a nuanced approach to assessing robustness, and that current research can paint a skewed picture. we advocate for researchers also to use the td to study domain robustness since it complements the sd and can reduce biases in their conclusions. our study shows that not everything that appears as a robustness problem is truly such. sometimes, it is just a transition to a more challenging domain. finally, our work underscores the ongoing relevance of domain adaptation research in nlp and the importance of developing robust, adaptable models capable of handling the diverse nature of real-world data. ## limitations l1. prompt engineering noteworthy, we experimented with various prompts and task instruction revisions but saw no significant change. following @xcite , we also tried selecting demonstrations from the source domain most similar to the target test example using a pre-trained sbert model @xcite . however, this approach did not enhance performance and introduced biases, such as demonstrations from only one class, leading the llm to classify the test example with this class. l2. larger models although we examined a broad range of models of various sizes, we did not fine-tune models with more than one billion parameters. this decision stems from two reasons. the first is our belief that fine-tuned models should be relatively fast and compact. otherwise, fewshot llms like those we examined in the study can be used. second, the volume of experiments (including hyperparameter tuning) imposed practical limitations, and examining larger fine-tuned models was not feasible due to their computational resource requirements. nonetheless, we believe that the trends observed in the smaller fine-tuned models will likely persist, and we leave the examination of larger models to future research.