Revisiting Out-of-distribution Robustness in NLP: Benchmarks, Analysis, and LLMs Evaluations
Authors: Lifan Yuan
Yangyi Chen
Ganqu Cui
Hongcheng Gao
FangYuan Zou
Xingyi Cheng
Heng Ji
Zhiyuan Liu
Maosong Sun
Venue: NeurIPS 2023 Datasets and Benchmarks Poster (2023)
Keywords: NLP
OOD Robustness
Large Language Models
TL;DR: Establish a standard OOD robustness benchmark for NLP, analyze the ID-OOD performance correlation of fine-tuned PLMs, and evaluate existing robustness-enhanced methods and LLMs.

Abstract: This paper reexamines the research on out-of-distribution (OOD) robustness in the field of NLP. We find that the distribution shift settings in previous studies commonly lack adequate challenges, hindering the accurate evaluation of OOD robustness. To address these issues, we propose a benchmark construction protocol that ensures clear differentiation and challenging distribution shifts. Then we introduce
BOSS, a Benchmark suite for Out-of-distribution robustneSS evaluation covering 5 tasks and 20 datasets. Based on BOSS, we conduct a series of experiments on pretrained language models for analysis and evaluation of OOD robustness. First, for vanilla fine-tuning, we examine the relationship between in-distribution (ID) and OOD performance. We identify three typical types that unveil the inner learning
mechanism, which could potentially facilitate the forecasting of OOD robustness, correlating with the advancements on ID datasets. Then, we evaluate 5 classic methods on BOSS and find that, despite exhibiting some effectiveness in specific cases, they do not offer significant improvement compared to vanilla fine-tuning. Further, we evaluate 5 LLMs with various adaptation paradigms and find that when sufficient ID data is available, fine-tuning domain-specific models outperform LLMs on ID examples significantly. However, in the case of OOD instances, prioritizing LLMs with in-context learning yields better results. We identify that both fine-tuned small models and LLMs face challenges in effectively addressing downstream tasks. The code is public at https://github.com/lifan-yuan/OOD_NLP.

Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations
Lifan Yuan', Yangyi Chen', Ganqu Cui', Hongcheng Gao Fangyuan Zou', Xingyi Cheng', Heng Ji Zhiyuan Liu Maosong Sun NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing
University of Illinois Urbana- Chhaaa--h
3 University of Chinese Academy of Sciences Tencent 1 1ievanyuan173e gma com
Abstract
This paper reexamines the research on nut-of-distribution (OOD) robustness in the field of NLP. We find that the distribution shift settings in previous studies com- monly lack adequate challenges, hindering the accurate evaluation of OOD robust- ness. To address these issues, we propose a benchmark construction protocol that ensures clear differentiation and challenging distribution shifts. Then we introduce BOSS, Benchmark suite for fuitr--distribution robustneSS evaluation covering 5 tasks and 20 datasets. Based on BOSS, we conduct a series of experiments on pre- trained language models for analysis and evaluation of OOD robustness. First, for vanilla fine-tuning, we examine the relationship between in-distribution (ID) and OOD performance. We identify three typical types that unveil the inner learning mechanism, which could potentially facilitate the forecasting of OOD robustness, correlating with the advancements on ID datasets. Then, we evaluate classic meth- ods on BOSS and find that, despite exhibiting some effectiveness in specific cases, they do not offer significant improvement compared to vanilla fine-tuning. Further, evaluate 5 LLMs with various adaptation paradigms and find that when sufficient ID data available, fine-tuning domain-speciff models outperform LLMs on ID examples significantly. However, in the case of OOD instances, prioritizing LLMs with in- in-context learning yields better results. We identify that both fine-tuned small models and LLMss face challenges in effectively addressing downstream tasks. The code is public at https:/ / / / thub. com/ fan yuan _OP_.NP.
Introduction
Pretrained language models (PLMs) have excelled downstream tasks and gained widespread adoption [24, 60]. However, existing evaluation often assumes independent and identically distributed (i.i.d) condition [94, 92], which is often violated in real-world scenarios, highlighting the crucial problem of out-of- nut-of-distribution (OOD) robustness in NLP models. In this paper, first revisit the evaluation of PLMs through an examination of evaluation benchmarks. Thereafter, we delve into the ID-0OD performance correlation of fine-tuned models by adopting various model scales, training steps, available training samples, and tunable parameters. Finally, we conduct extensive evaluations of current robustness-enhanced methods and large language models (LLMs)..
Definition. There exist multiple definitions of OOD in literature [2, 115], and we define distribution shifts considered in this paper from two perspectives. Firstly, [2] classifies distribution shifts into "semantic shift" and "background shift", Our use of "outtof-disttriution" aligns with the concept of
"Corresponding Author.
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

A
Previous Paradigm
1/ Search Datasets 2) Select popular datasets
3) Randomly choose ID-00D pairs
Conduct experiments
amazon Google
IMob
Evaluation Task
3.1) Analyzing and computing semantic similarity
Our Paradigm
Collect candidate dataset
2) Select ID dataset (large diverse) 3.2) Measuring performance drop
Figure 1: The comparison of previous work and our protocol on dataset selection of OOD benchmarks.
"background shift", which involves changes in the domain or style of the text while preserving the semantic content. Secondly, [115] formally defines three types of distribution shifts: covariate shift, label shift, and concept shift. In our work, we mainly focus on the combination of covariate shift and concept shift. This indicates that the model needs generalize well to different input features (a.k.a, covariate shift) and adapt to variations in the underlying concepts within the data (a.k.a, concept shift).
Benchmark. Our study begins by surveying existing literature on OOD robustness in NLP (Table 8). We observe a lack of standardized OOD benchmark suites tailored for NLP since there is no single uniform set of datasets for evaluation, resulting in the adoption of heuristic and popularity-based dataset selection strategies previous work [39, 109, 107]. This approach suffers from two main drawbacks: (1) The selected OOD datasets may come from similar distributions the ID dataset, reducing the OOD evaluation to ID settings and thus hindering rigorous OOD robustness evaluation; (2) The challenge stemming from distribution shifts is limited, deviating from the expectation for difficulty posed by benchmark construction principles [8] and potentially leading to an overestimation of the OOD robustness of language models. As result, current OOD robustness benchmarks may inadequately assess NLP models.
To address the aforementioned concerns, we establish a protocol as shown in Figure 1, consisting of three fundamental principles, for selecting both ID and OOD datasets: (1) ID datasets should be large and diverse for comprehensive knowledge; (2) Selection for OOD datasets should prioritize distinct distributions and dissimilarity, regarding text sources and semantics; (3) Challenging distribution shifts should be prioritized based performance degradation. to ensure that the benchmark stands the test of time [8]. Based on the protocol, we compile BOSS, a more holistic and challenging NLP Benchmark suite for OOD robustneSS evaluation. Unlike existing benchmarks which only consider single task types such as classification [39, 107] or reading comprehension [109], BOSS covers a wider range of task formats, including sentiment analysis, toxic detection, and natural language inference for classification, name entity recognition for structured prediction, and extractive question answering for reading comprehension. We establish one ID and three corresponding OOD datasets for each task.
Analysis. We recognize the lack of analysis of models' learning behaviors regarding ID performance and OOD generalization in the field of NLP, hindering the development and understanding of OOD robustness. Thus, investigate the correlation between the performance ID and OOD datasets using the BOSS benchmark. To regulate the ID performance, we manipulate four related factors, i.e. model scale, training steps, available training samples, and tunable parameters. Three typical categories of ID-OOD correlation are observed, namely, monotonic linear positive correlation, monotonic piecewise linear positive correlation, and non-monotonic V-shaped correlation (see Figure 2). We discuss the potential reasons for the causes these identified correlations in section 3.
After examining the learning mechanism of PLMS in vanilla fine-tuning, we scrutinize their performance with existing robustness-enhanced methods and then proceed to prevalent L.LMs. Due to the absence of standard benchmark, previous evaluations of existing methods can be
Evaluations.

---

imprecise and thus misleading the estimations of progress in this field. Moreover, given the increasing focus on LLMs [9, 89] in NLP research, it is essential to evaluate their effectiveness in handling OOD challenges and explore the efficacy of different adaptation paradigms.
For robustness-enhanced methods, we evaluate five representative methods [99] BOSS. Our main observation is that vanilla fine-tuning (a.k.a, empirical risk minimization) remains a strong baseline, while certain methods may slightly improve OOD performance some cases. We further evaluate various LLMs and adaptation paradigms. We consider three recent prevailing LLMs, namely LLaMA [89]. OpenAI text-davinci-000 [9]. and OpenAI gpt-3.5-turbo. We include two relatively smaller models TO-3B [81] and T5-3B [77] for comparison. We apply zero-shot inference, in-context learning, few-shot fine-tuning, and full-data fine-tuning to one or multiple models. Through our experiments, we find that when provided with enough training data, fine-tuning domain-specific models remain the preferable choices for handling ID examples, while leveraging L.M with in-context learning superior for tackling OOD instances. In addition, we observe that the impact of in-context learning on generalization ability varies across models. We provide more detailed discussions in section 4.2.
2 BOSS Benchmark
2.1 Motivation
NLP models should exhibit robustness across diverse distributions to ensure reliable applications. To achieve this, a standardized and recognized evaluation benchmark for OOD robustness is imperative. However, previous efforts in constructing benchmarks have predominantly relied on random selections and dataset popularity, lacking a systematic design [39, 109, 107]. Two deficiencies are thus identified: (1) Dataset similarity, as exemplified by the SST and IMDb datasets for sentiment analysis [83, 64]. which share movie reviews and exhibit high semantic similarity (see Table 2). This blurs the line between II) and OOD evaluation, hindering rigorous assessment of OOD robustness; (2) Limited distribution shift challenges, exemplified by the high accuracy of a model trained on Amazon [65] when tested on IMDb (see Table 3). However, the significant performance drop on our considered Dynasent [75] suggests that OOD robustness still remains a critical problem in the sentiment analysis task. Thus, there is need for universally applicable challenges across all dataset selections [8].
2.2 Protocol to Construct OOD benchmark.
We aim to establish standard benchmark for rigorous evaluation of OOD robustness in NI.P. To address the above issues, we first survey and gather existing candidate datasets from Paperswi thcode?, Kaggle², and ACL Anthol ogy websites. We consider the release date and public availability of datasets. Then we carefully examine three criteria to determine the ID and corresponding OOD datasets. The first criterion focuses the ID dataset selection, and the other two criteria are proposed for OOD datasets, targeting the two issues in previous work, respectively.
The ID dataset should provide sufficient knowledge for models to handle the task. ID dataset should encompass comprehensive task-level knowledge [44], enabling models to grasp the underlying rationale necessary for task completion. Alternatively, if the model exclusively learns biased features, may struggle to adapt to other features during distribution shifts. To this end, is necessary for the ID datasets to possess the following characteristics:
(1) Sufficiently large size; (2) Diversity, which is achieved through collection from multiple sources or the inclusion of several subtypes (i.e., styles, topics, levels of formality, et al). Our intuition is in line with [87], which demonstrates that training on large and diverse datasets improves the robustness of vision models.
i
Datasets within a given task should originate from diverse distributions for a holistic evaluation. We guarantee this through qualitative analysis of data source diversity and quantitative measurement of semantic similarity using SimCSE [31]. To avoid overlap. we select at most one dataset per text source. Additionally, we ensure OOD datasets in the benchmark exhibit relatively low semantic similarity, and thus enhancing distinctiveness.
hnttps://papersw/ thoode com datasets hittps:/wwwkkkggt com hhttpe://aclanthology
3

---

OOD shifts should be challenging to provide an accurate assessment of progress in OOD ro- bustness [8]. To quantify the challenge, we train model on the ID dataset and test it on all candidate datasets. Specifically, we tune TS-large [77] with manual templates on four tasks, except for NER, on which we adopt DeBeRTa-large [38] with conventional fine-tuning due to the lack of a standard prompt-bane hased tuning schema for this task. For this reason, all experiments in this paper follow this choice of model selection. For each text source, we first discard candidates similar to the ID dataset in semantics. Then, to construct challenging distribution shifts, we prioritize the dataset provoking the most severe performance drop of the ID model and adopt it as the OOD dataset in our benchmark.
2.3 Dataset Selection
We take sentiment analysis as a case to demonstrate how we select ID and OOD datasets for each task according to our protocol. The selection process for other tasks can be found in Appendix D.
Candidate Datasets. We first collect all sentiment analysis datasets on Paperswi wanrrssithcor thoode,
Kaggle, and ACL anthology as aforementioned. We filter out datasets released before the 2010s, as they are largely resolved with the advent of pre-trained language models [25]. As result, seven datasets remain as candidates, i.e., Amazon [65], DSC [48], Dynasent [75], IMDb [64], SemEval [70], SST [83], and Yelp [116]. Considering the inconsistency in the number of categories across the datasets, we align them by converting them into a three-class classification setting. See Appendix C.2 for detailed explanation of the dataset processing procedure.
According to our protocol, dataset size and text sources are assessed for ID dataset selec- tion. Subsequently, semantic similarity and ID model performance degradation guide OOD dataset selection. To this end, two probing experiments are conducted: (1) Comparing semantic similarity using SimCSE for candidate dataset pairs, and (2) Evaluating the performance of the selected ID model. In the first experiment, for better semantic representation, resort to the best SimCSE model provided by [31], supervised RoBERTa- large [60]. We load the model checkpoint from Huggingface". For each dataset, we first encode each sample into a continuous embedding and then average the embeddings across the dataset obtain centroid representation of the dataset. Finally, we calculate the cosine similarity between a pair of centroids as the semantic similarity between two datasets. In the second experiment, train T5- T5-large model on the selected ID dataset and evaluate its performance on all the candidate datasets.
Probing Experiments.
Table 1: Statistics of sentiment analysis candidate datasets.
Samples
Avg.
Length
Dataset
Source
Classes
Train
Test
Train
Test
Amazon
Product
3
30,000
38,905
71.69
54.84
DSC
Product
2
92,244
11,531
132.29
130.14
Dynasent
Adversarial
93,553
13.19
IMDb
Movie
2
25,000
25,000
233.79
228.53
SemEval SST
Twitter
3
6,000
20.622
19.44
19.62
Movic
4,004
1,067
18.78
18.75
Yelp
Product
30,000
30,000
132,49
131.62
Dataset Selection. The dataset information and semantic simi- larities are provided in Table and Table 2, respectively. The text sources of the datasets vary from product reviews, movie reviews, Twitter, and adversarial texts. We observe that datasets originating from the same source tend to exhibit higher SimCSE scores, indicating higher seman- tie similarity. It is worth noting that for IMDb and SST, the widely used ID-OOD dataset pair in sentiment analysis [39, 107], the SimCSE score demonstrates one of the highest levels among dataset pairs. This reinforces the first deficiency of previous benchmarks, where dataset pairs have similar semantics and unclear distribution shifts. Hence, in contrast to existing practices, our benchmark construction considers only one dataset from each source.
Table 2: SimCSE scores between each pair of datasets regarding the sentiment analysis task.
Train
Test
Amazon
DSC
Dynasent
IMDB
SemEval
SST
Yelp
Amazon
100
86.02
$7.30
36.67
24.74
33.70
49.22
DSC
86.02
100
59.15
54.55
31.70
44.40
55.45
Dynusent
57.30
59.15
100
32.69
28.17
19.68
88.99
IMDb
36.67
54.55
32.69
100
46.95
84.62
39.88
SemBval
24.74
31.70
28.17
46.95
100
40.45
24.03
SST
33.70
44.40
19.68
84.62
40.45
100
19.43
Yelp
49.22
55.45
88.99
39.88
24.03
19.43
100)
For the ID dataset selection, we first exclude DSC and IMDb since they are binary classification datasets, on which the trained model cannot tackle the unseen class neutral. For dataset size,

---

SemEval and SST are disregarded due to their limited number of samples per class (less than 10k). Among the remaining datasets, Amazon chosen as the ID dataset for sentiment analysis as encompasses reviews from 29 distinct product categories, offering greater diversity than Yelp.
For OOD datasets selection, we train a T5-large model on the ID dataset (i.e., Amazon) and eval- uate it on all candidate datasets, as illustrated in Table 3. We in- clude Dynasent and SemEval in the benchmark suite due the following reasons: (1) They are the sole adversarial and Twitter datasets available, (2) They demonstrate low semantic similarity. and (3) They exhibit a notable performance degradation, making them crucial for evaluation. For movie reviews, SST is prioritized due to lower SimCSE scores compared to IMDb and larger performance drop of the ID model. Eventually, this yields three distinct and challenging distribution shifts in the sentiment analysis task: Amazon + (DynaSent, SemEval, SST).
Table 3: The OOD performance of the T5-large when trained on the Amazon dataset.
Train
Test
Amazon
DSC
Dynasent
IMDb
SemEval
SST
Yelp
Amazon
90.94
95.63
47.38
92.69
49.90
75.16
89.25
2.4 BOSS
Based on the aforementioned protocol, we introduce BOSS, an
Table 4: The datasets included in the BOSS benchmark. Corre- sponding abbreviations are shown in brackets.
NLP benchmark suite for OOD
robustness evaluation. BOSS comprises five essential NLP tasks: sentiment analysis (SA),
Task
ID Dataset
0OD Datasets
SA
Amazonn (AZ)
Dynasent (DS)
SemEval (SE)
SST (SST)
TD
Civil Comments (CC)
AdvCivil (AC)
Implicit Hate (IH)
ToxiGon (TG)
NLI
MNLI (MN)
ANLII (AN)
ContractNLJ (CN)
WANLI (WN)
toxic detection (TD), natural lan- guage inference (NLI), name en-
NER
FewNerd (FN)
CoNLL (CoNLL)
E-NER (ENER)
WNUT (WNUT)
FQA
SQuAD (SQuAD)
AdvQA (AQA)
NewsQA (NQA)
SearchQA (QA)
tity recognition (NER), and ex-
tractive question answering (EQA). These tasks represent diverse practical applications and provide comprehensive coverage for evaluating models' capabilities, from aspects of classification, structured prediction, and extraction. Each task in the benchmark includes one ID dataset and three associated OOD datasets (see Table 4).
Sentiment Analysis. Amazon [65] contains reviews of 29 different categories of products from the Amazon website. DynaSent [75] first identifies naturally challenging sentences from several existing datasets, and then creates adversarial sentences with a human-and and-model--nn -in-tthe-loop annotation approach. SemEval [70] is a three-class sentiment analysis dataset focusing on tweets. SST [83] consists of sentence-level movie reviews from the Rotten Tomatoes website.
Toxic Detection. Civil Comments [6] contains public comments the Civil Comments platform, with users from diverse groups and various subtypes of toxic texts. AdvCivil, a new toxic dataset introduced in this paper. is generated from Civil Comments by textual adversarial attacks in an automated model-in-the-loop adversarial pipeline. Please refer to Appendix C. for details. Implicit Hate [29] contains toxic tweets in both explicit and implicit forms. The latter can circumvent keyword- -bsed toxic detection systems. ToxiGen [36] is synthesized by GPT-3 [9], covering several types of subtly and implicitly toxic texts on 13 minority groups.
Natural Language Inference. MNLI [102] provides ten different categories of written and verbal sentence pairs, with diverse styles, topics, and levels of formality. ANLI [73] is an adversarial approach, where each premise mainly comes from Wikipedia and the hypothesis is generated by human adversaries. ContractNLI [49] considers each contract as a premise and holds a fixed set of hypotheses throughout the dataset. WANLI [59] is synthesized by GPT-3 [9], each example containing challenging patterns identified in MNLI.
dataset collected in a human-and moadd-ooel-----heee
Name Entity Recognition. Few-NERD [26], the arguably largest dataset for NER, labels about 188k Wikipedia sentences into eight coarse-graines entity types. CoNLL [88] takes stories from Reuters news, containing four basic entity types. E-NER [3] is based on legal text. We use the four-category version in this paper, which treats all legal entities as miscellaneous ones. WNUT [23] collects training data from Twitter and mines test data from Reddit, StackExchange, Twitter, and YouTube, containing six coarse- graniedd entity types in Few-NERD.
5

---

-WNNUT
SQUAD A4
Miero F1, on VINUT (D00)
50
Micro FI AdvGA (000)
50
Accuracy on ANU (OOO)
38
66-
%
90
92
10
23
10 25
10
40
50
80
10
20
43
60
80
30
40
50
00
70
80
50
Micro on Few NLRD (ID)
Micro 12 on SQUAD (I))
Accuracy on MNLI IID)
(a) Type I
(b) Type
(c) Type III
Small-sized
Base Training Stops
Large-sized
Linear Fit
Green Available Samples
Tunable Parameters
Three representative correlations between ID-OOD performance: (a) Type I (monotonic linear positive correlation) indicates consistent linear improvement of OOD performance with increasing ID performance. (b) Type II (monotonic piecewise linear positive correlation) exhibits accelerated OOD performance growth after turning point. (c) Type III (non- non-monotonic V-shaped correlation) shows an initial negative correlation, followed by positive correlation after turning point. The r² value in Figure (a) is 0.9677, and the values of the left and right fits in Figure (b) are 0.9553 and 0.9396 whereas in Figure (c) are 0.7690 and 0.8 24 respectively.
Figure 2:
Extractive Question Answering. SQuAD [78] constructs question-answer pairs based on Wikipedia passages. AdversarialQA [4] composes adversarial questions for contexts in SQuAD in a human-and- model-in-the-loop procedure, similar to ANLI NewsQA [90] writes questions for CNN news articles, each of which requires reasoning to answer, rather than relying solely on word overlap and textual entailment. SearchQA [28] adopts reverse construction pipeline, employing the Google search engine to retrieve relevant contexts for each question-ansswend pair from the J!Archive website.
Analysis of OOD Robustness
Despite OOD robustness in NLP has been extensively studied [43], potential concern pertains the usage of nonstandard benchmarks, as discussed in Section 2, resulting in inaccurate conclusions. To address this issue, we conduct a series of empirical analyses and evaluations to gain in-depth insights into OOD robustness in NLP. Previous research primarily concentrates on method comparisons without delving into models' learning behaviors. Therefore, we first analyze the models' learning mechanism by assessing the correlation between ID and OOD performance.
Setting. We assess the correlation between ID and OOD performance across various conditions. We manipulate the ID performance of models by varying their scale, training steps, available training samples, and tunable parameters. Further implementation details can be found in Appendix 1.1.1
Results. We observe that the correlation between ID and OOD performance on datasets of the five tasks is inconsistent, but be broadly categorized into three types (see Figure 2): monotonic linear positive correlation (Type I), monotonic piecewise linear positive correlation (Type II), and non-monotonic V-shaped correlation (Type III.. We also identify exceptional case in Figure which does not fall into any of the three categories. The full results are shown in Figure 4.
Type I. This is the most prevalent type of correlation observed across all ID-OOD pairs for sentiment analysis, name entity recognition, and the majority for toxie detection. As shown in Figure 2a, in this category, OOD performance is positively and linearly correlated with ID performance, indicating that the task knowledge learned on source distribution can be effectively generalized to other distributions. This observation is consistent with results in the computer vision domain [68], which shows that OOD performance is lincarly correlated with ID performance across various model architectures, hyperparameters, training dataset size, and training duration. However, the slope of the line fitted by the least square method is less steep than the y = x diagram, and it eventually lies below the diagonal, implying that the performance degradation of models under distribution shift will be escalated with the increase of the ID performance.
Type II. This category is observed on ID-OOD pairs for extractive question answering. As presented in Figure 2b, the results can be fitted into polyline, indicating a piecewise linear correlation. The correlation between OOD performance and ID performance is positive and linear, with notable differ-

---

ence in the slope before and after the turning point. Specifically, OOD performance demonstrates slow growth until the turning point, after which a minor increase in ID performance yields a substantial im- provement in OOD performance. The observed trend may be attributed to the findings of [91], which indicates that models initially capture spurious correlations in ID datasets before acquiring comprehen- sive task knowledge. Consequently, models prioritize learning these spurious correlations to address the ID task, resulting in minimal improvements on OOD datasets. However, in the later stages of train- ing, models progressively acquire greater task knowledge, leading to improved OOD performance.
Type III. The V-shaped fitted lines shown in Figure 2c mainly occurs on ID-OOD pairs of NLI tasks. This pattern is divided into two stages by a turning point in ID performance. In the first stage, OOD performance experiences worsening performance degradation during the distribution shift. However, in the second stage, the ID-OOD correlation becomes positive. This trend resembles the U-shaped scaling law of L.LMs observed by [100], thus suggesting shared explanation. [100] attributes this phenomenon to the "distractor task" in the dataset, apart from the "true task". Medium- capacity models may perform better than low-capacity models on the distractor task, which may harm performance on the "true task". As the model capability increases, it can ignore the distractor task and focus on improving performance on the true task. Here, we identify the distractor task in NLI datasets as detecting the word overlap other spurious correlations between the premise and hypothesis.
Outlier. There is an exceptional case regarding the distri- bution shift from Civil Comments to AdvCivil (see Figure 3). The figure depicts two distinct lines, both exhibiting monotonic linear negative correlation. This may stem from the model's increased reliance on spurious correla- tions and the adversarial nature of AdvCivil. Prior research suggests that models can learn non-robust features, such as spurious correlations, to enhance ID accuracy [44]. How- ever, adversarial samples preserve the semantics of the original texts while introducing perturbations that elim- inate spurious correlations. Hence, when the ID model becomes more dependent spurious correlations dur- ing training, its performance degradation on adversarial samples intensifies.
Civil Comments AdvCivil
Arcuracy Advchvi (OOD)
80
70-
60
550
40
30
20
10
20 Accuracy Civil Comments
30
40
50
60 70
80 90
Figure 3: The OOD performance ex- hibits a negative correlation with ID per- formance. Refer to Figure 2 for legends.
4 Evaluation of OOD Robustness
4.1 Robustness-enhanced Methods
After analyzing the learning behavior of PLMS under vanilla fine-tuning, examine their perfor- mance when trained with other methods. Although massive methods have been proposed to improve the robustness of PLMs, their evaluations rely on non-standard benchmarks, which may result in inaccurate evaluations and impede progress clarity. Therefore, in this section, we first conduct extensive experiments to re-evaluate the effectiveness of diverse abbustess-enhhance methods
Setting. We consider the categories of robustness-enhancee methods summarized by [99]: data- driven, model and training-based, inductive-prios peaetive-prior-baseeeee brior- and causal intervention methods. We select the most representative one from each category for evaluation. Specifically, we choose EDA [101] and FreeI.B [1 18] for data-driven methods, label smoothing
[84] and focal loss [58] for model and thining-based methods, and model ensemble [16] for inductive-prion based methods. We do not consider causal intervention methods since they are typically applied low-resource scenarios. As explained in section 2.3, we apply the above methods to DeBERTa- base models for the NER task and to T5-base models for the other tasks.
Results. The results are shown in Table 5, where the mark indicates that a certain method is not applicable to the task. We summarize the findings in the following takeaways:
The vanilla fine-tuning (empirical risk minimization) remains a strong buseline. Despite existing methods outperforming vanilla fine-tuning on certain datasets like E-NER, WNUT, and NewsQA, they show limited superiority or can potentially harm model performance. Specifically, only FreeLB demonstrates beneficial effects over half of the datasets, standing out as the most
Takeaway 1:
7

---

Table 5: The evaluation of robustness- methods. The results that surpass the vanilla baseline are underlined, We use abbreviations representative of the datasets to conserve space. Please refer to Table 4 for their corresponding full dataset names.
Bak
SA
TD
NLI
NER
EQA
Datser
A2
DS
CC
11
TG
MN
WN
RN
CUNILL
ENER
WNUT
SQ:AD
AQA
NQA
9Q4
57.47
RDA
$1.06
46. 10
48.00
15.81
87.15
57.43
61:77
68.83
66.57
$4.50
46.25
ReeLB
$1.29
47.94
47.58
T0.05
85.63
63.55
62.22
67.98
89.3
36.13
3010
53.55
90.08
65.06
50.54
47.77
93.51
51,07
65.00
30,57
Fl.
51.02
66.30
50.11
16.35
87:10
37.75
60.20
61.56
19.17
36.55
10.26
51.32
70.00
61.M4
50:49
41:31
50.64
61.6
16.01
LS
47.31
36.38
16.1
8665
3783
02.66
07.98
80.03
3630
19:36
62.85
79.66
08.1
2570E
27.21
93.32
50,95
85.7
34.51
ES
50.72
41.83
$4.96
4702
61055
65.32
7787
35.76
Table 6: Evaluations of L.MSs on BOSS. Small Ref represents the results of supervised fine-tuned small models in Table 5 (Vanilla). We observe that given enough ID data, fine-tuning domain-specific models is predominant when testing ID examples. In contrast, LLMs with In-context learning should be given priority on OOD instances.
Tulk
NLI
Datsor
A2
DX5
SR.
SST
EX:
AF
AN
CN
WX
FX
CONEL
WNEFT
SQ/AD
AQAA
NODA
5gg5
T0:38
0.ck
14.35
07.90
1116
54 16
11.50
16
11.15
50
15.00
15.79
74.07
0
$0.90
$1.59
$1.35
00
84.55
33.63
34.27
17.08
75.94
40.72
44.04
25.44
33.55
46.20
46.64
38.08
21.37
15.42
K01
$4.55
53.83
34.23
7166
21.39
75-64
40.22
44.14
35.44
$3.35
45.35
97.15
0
15.30
11166
0.15
7.75
TVE-4r3
63:63
51.11
5005
34.13
96.13
66.75
63.12
30.04
14.51
15.97
64.57
16
15.00
40.24
5663
34.97
94.30
$7.46
05.55 80
$4.75
LLAMA-TB
KL.
$4.30
20.19
13.66
31.16
91.30
20417
$3.60
$1.58
30.51
33.90
15.30
38.17
DAS
6.70
0.11
1656
67.57
11.35
44.13
45.76
0.21
D-dee
81.25
56-05
17:75
54.59
50.07
15.00
63.05
50.80 83
13.17
24.45
17.06
78.14
3.16
1.00
0.47
0/00
66.90
37.31
05.15
55.0
02.12
54:71
37.59
$0.01
35.55
05.55
38.35
20.42
0.35
69.71
$8,67
ICL
35.46
35.00
37.31
0.50
0.00
43.99
40.30
60:50
Dovbett
KS.
81.00
76.85
23.23
76.18
66:45
23.50
71.21
$0.00
55.35
52.32
18.56
57.04
46.27
62 10
$2.55
$2.56
51.10
09.55
75.24
66.14
75.34
14:91
$1.06
05.93
46.71
$1.41
56.
85.05
74.40
08.35
33.04
76.30
MA57
$2,00
22.10
66.75
46.56
$0.82
48.55
36.35
51.35
3623
33.35
$3.21
5549
47.08
06.35
Tubo
KTT
97.75
77.18
75.00
16
$9.56
$1.07
08.
46-47
45.55
57.07
58.77
15
$2.00
58.78
91
effective approach. Conversely, the inductive- inductive-prior- based ensemble is the worst, consistently leading to performance degradation except for the SemEval dataset.
Takeaway Method effectiveness is consistent, yet limited to specific datasets. Across multiple datasets, various methods consistently demonstrate (in)effectiveness, as indicated in Table 5. However, no method consistently performs well on all datasets within the same task.
To summarize, cuurrrttiieeeeettteeeeee approaches fall short of meeting the expectations in enhancing the 0OD robustness of models, highlighting the urgent demand for more advanced improvement techniques.
4.2 Large Language Models
LLMs are receiving increasing attention from NLP researchers. Considering the impressive zero/few- shot ability of LLMs,, and the large difference in their fine-tuning and in-context learning paradigma it is also intriguing to benchmark their generalizability on various downstream tasks and explore the best paradigm to leverage their power.
Setting. We consider three prominent state-of-the-art LLMs, LLaMA-7B and LLaMA-13B (i.e., LLaMA-series) [89] .OpenAl text-davinci-003 [9] and gpt-3.5-turbo (denoted as Davinei3 and Turbo respectively). For comparison, we include two relatively smaller (yet still large) models, TO-3B [81] and T5-3B. We perform zero-shot inference based on task instructions this paradigm is the most general. Then, we adopt other paradigms in a model-specific way. For T5-3B, we include 5-shot and full-data fine-tuning, and also select examplars from the ID dataset for in-context learning. For the other three LLMMs, we apply in-context learning with two kinds of contexts, one from the ID dataset and another from the original training split of the evaluated OOD dataset, denoted as ICL and ICL" respectively. The implementation details are in Appendix E.2.
all these models since
Results. We present the results in Table 6, where the mark '.' means that ICL" paradigm is not applicable for those datasets due to the absence of training split or the limit of context window size. Our findings can be summarized follows:
Takeaway I: Tine-tuning small domain- domain-specifee specific models is superior when enough training data is available, while LLMs may be favored in low-resource scenarios. To be specific, supervised fine- tuned small models and T5-3B with the entire dataset consistently achieves the best performance on the ID dataset, especially for the structure prediction task (c.g., NER), In contrast, LLMs exhibit
8

---

better performance on most OOD datasets. This observation reinforces the view that large pre- pre-trained models possess strong generalization capabilities, whereas, with sufficient training data, an accurate estimate of data distribution can be achieved even without large number of parameters [76].
Takeaway 2: In-context learning always brings no gains the generalization ability of small models, while it generally helps Turbo and significantly improves LLaMA- and Davinci3. For small models like T5-3B, the performance of in-context learning is the same with or even worse than the zero-shot inference. For Turbo, providing ID examples for in-context learning presents advantages on nearly two-thirds of the datasets, with the NER task benefiting the most. For LLaMA--eriee and Davinci3, the superiority of in-context learning is prominent as it enhances performances on most of the datasets.
Takeaway 3: Examples from ID datasets are generally more effective for in-contexi learning than those from the original training split of the lesting OOD dataset. Specifically, when considering samples from our OOD) datasets as contexts, the performance of Turbo is comparable to using II) samples, whereas the LLaMA-series and Davinci3 models consistently exhibit inferior performance compared to using ID examples as contexts. However, all models show improved performance on the EQA task when contexts from our OOD datasets are utilized. This may be attributed to the variations in sample length or question styles across EQA datasets, hence models acquire more precise instructions from the original training samples. The overall ineffectiveness of ICL" can be explained by the findings of [69]. According to [69], in-context demonstrations aim to guide the models learn the target label space, instead of the feature-label mapping. The ID examples contain more diverse information due to the construction process of our benchmark. Thus, ID examples can better prompt the language models to locate the target label space, compared to the OOD examples that may target specific domain.
Discussion. Two paradigms are prevalent in developing downstream NLP systems: leveraging general-purpose LLMs or gathering domain-specific data for fine-tuning smaller models. For the first paradigm, the overarching objective of general-purpose LLM development is to employ single model for solving various downstream tasks [9]. Consequently, LLMs are anticipated to exhibit high performance on both ID and OOD datasets. However, our study exposes the shortcomings of LLMs on ID datasets when compared to fine-tuned domain-specifer models. Considering the higher inference and deployment costs associated with I.LMs, substantial progress still needed to effectively improve LLMS in developing downstream applications, particularly for challenging tasks like EQA. For the second paradigm, our study reveals the limitations of fine-tuning models on ID datasets for OOD performance in comparison to LLMs. Thus, further research is required to develop advanced techniques that enhance the robustness of fine-tuned domain-specific models. Overall, the existing two prevalent paradigms still fall short in addressing the OOD problem in NLP, necessitating further advancements and effective approaches.
However, we also note that there can exist confounders in our evaluations. It is still ambiguous which datasets are indeed OOD to LLMs, given that LLMs have been pre-trained on massive public corpora. The potential data contamination issue can result in overinflated performance on our OOD datasets, tangling the credit of the memory and generalizability of LLMs. The only confirmed distribution shift for LLMs is the temporal shift, necessitating the evaluation based on data released subsequent to their pre-training data collection cut-off. Therefore, the NLP community demands new downstream datasets independent of the pre-training corpus to meet the evaluation requirements for LLMs.
5 Related Work
Distribution shifts in NLP has been widely studied in various forms. We examine several represen- tative cases as outlined below. Domain shift refers to the challenge of testing data originating from diverse domains, often due data collection from various sources [63, 40, 53, 79]. Temporal shift examines the degradation of models' performance over time [42, 1]. Spurious correlation examines the issue of models acquiring sptaset- knowledge on ID data, which may not generalize effectively to OOD data [66, 73, 91, 32, 37, 16, 17]. Additionally, a requirement is for models to exhibit robustness when confronted with artificially constructed OOD samples. One typical type is malicious adversarial attacks, which involve assessing the resilience of models against inputs crafted by malevolent adversaries [56, 51, 112]. These inputs, distinet from ID samples, have the potential to induce model failures [12]. Adversarial attacks can also be effectively utilized to simulate

---

diverse user imputs to examine models' robustness in the real world [98, 13, 33]. Another category is backdoor attacks, characterized by intentionally introduced spurious correlations that can be exploited by attackers for their advantage [18, 55].
OOD Evaluation in NLP can be broadly classified into automatic and static evaluation approaches. Automatic evaluation utilizes diverse textual transformation techniques, such as introducing typos, conduct rigorous evaluation of OOD robustness. Three essential elements the automatic OOD evaluation encompass the establishment of suitable transformation methods, evaluation metrics, and ef- fective techniques to ensure sample validity [34, 98]. Static evaluation, contrast automated meth- ods, offers the advantage of constructing benchmarks with higher quality, resulting in an improved estimation of OOD robustness. Numerous OOD benchmarks have been introduced, focusing on adversarial attacks [96] or spurious correlations [1 [117, 66]. A relevant study to ours is GLUE-X [107], which establishes an OOD benchmark derived from the GLUE benchmark [93]. Nevertheless, they do not establish coherent benchmark construction protocol and primarily rely on dataset selection driven by popularity, incorporating datasets into the benchmark without comprehensive explanation and seemingly opting for somewhat arbitrary selection, thus lacking systematic approach.
6 Conclusion
We revisit OOD robustness research in NLP, identifying deficiencies in benchmarks and evaluation. Correspondingly, benchmark construction protocol and an OOD robustness evaluation suite proposed to facilitate future research. The correlation between OOD and ID performance, the effectiveness of existing methods, and the challenges faced by LLMs are investigated.
Limitation
We identify two limitations in this work. First, as discussed in section 4.2, due to the lack of new datasets in the community, there is a possibility that some datasets have been included the pre- training corpus of LLMs, so they may not be suitable to test the generalizability of recent LLMs. However, we note that with our benchmark construction protocol, we can easily update the benchmark as new datasets come out. Second, we only consider five tasks in this benchmark, which is not comprehensive collection of current NLP literature. We explain the reason for the current task selection in Appendix A.I.
Acknowledgement
This work sponsored by the Seinghua-Toyonn Joint Research Fund.
Lifan Yuan and Yangyi Chen initiated the project. Lifan Yuan, Yangyi Chen, and Ganqu Cui designed the experiments. Lifan Yuan, Yangyi Chen, and Hongcheng Gao constructed the AdvCivil dataset. Lifan Yuan conducted experiments and wrote the paper. Yangyi Chen and Ganqu Cui revised the paper. Everyone participated in the discussion. Heng Ji, Zhiyuan Liu, and Maosong Sun advised the project.
References
[1] Oshin Agarwal and Ani Nenkova. Temporal effects on pre-trained models for language processing tasks. arXiv preprint arXiv:2111.12790. 2021.
[2] Udit Arora, William Huang, and He He. Types of out-of-distribule texts and how detect them. In Proceedings of EMNLP, 2021.
[3] Ting Wai Terence Au, Vasileios Lampos, and Ingemar Cox. E-NER an annotated named en- tity recognition corpus of legal text. Proceedings of the Natural Legal Language Processing Workshop, 2022.
[4] Max Bartolo, Alastair Roberts, Johannes Welbl, Sebastian Riedel, and Pontus Stenetorp. Beat the AI: Investigating adversarial human annotation for reading comprehension. Transactions of ACL, 2020.
10

---

[5] Mohaddeseh Bastan, Mihai Surdeanu, and Niranjan Balasubramanian. Bionli: Generating biomedical nli dataset using lexico-semantic constraints for adversarial examples. In Findings of EMNLP, 2022.
[6] Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. CoRR, 2019.
[7] Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of EMNLP, 2015.
[8] Samuel R. Bowman and George Dahl. What will it take to fix benchmarking in natural language understanding? In Proceedings of NAACI, 2021.
[9] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Preceedings of NeurIPS, 2020.
[10] Mohit Chandra, Ashwin Pathak, Eesha Dutta, Paryul Jain, Manish Gupta, Manish Shrivas- tava, and Ponnurangam Kumaraguru. AbuseAnalyzer: Abuse detection, severity and target prediction for gab posts. In Proceedings of COLING, 2020.
[11] Jiaao Chen, Dinghan Shen, Weizhu Chen, and Diyi Yang. HiddenCut: Simple data aug- mentation for natural language understanding with better generalizability. In Proceedings of ACL-IJCNLP, 2021.
[12] Yangyi Chen, Hongcheng Gao, Ganqu Cui, Fanchao Qi, Longtao Huang, Zhiyuan Liu, and Maosong Sun. Why should adversarial perturbations be imperceptible? rethink the research paradigm in adversarial nip. In Proceedings of EMNLP, 2022.
[13] Yangyi Chen, Hongcheng Gao, Ganqu Cui, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, al. From adversarial arms race model-centric evaluation: Motivating a unified automatic robustness evaluation framework. In Findings of ACL, 2023.
[14] Yangyi Chen, Lifan Yuan, Ganqu Cui. Zhiyuan Liu, and Heng Ji. A close look into the calibration of pre-trained language models. In Proceedings of ACL, 2023.
[15] Hao Cheng, Xiaodong Liu, Lis Pereira, Yaoliang Yu, and Jianfeng Gao. Posterior differ- ential regularization with f-divergence for improving model robustness. In Proceedings of NAACL:HLT, 2021.
[16] Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. Don't take the easy way out: Ensemble based methods for avoiding known dataset biases. In Proceedings of EMNLP. IJCNLP, 2019.
[17] Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. Learning to model and ignore dataset bias with mixed capacity ensembles. arXiv preprint urXiv:2011.03855 2020.
[18] Ganqu Cui, Lifan Yuan, Bingxiang He, Yangyi Chen, Zhiyuan Liu, and Maosong Sun. A unified evaluation of textual backdoor learning: Frameworks and benchmarks. In Proceedings of NeurlPS, 2022.
[19] Sarkar Snigdha Sarathi Das, Arzoo Katiyar, Rebecca Passonneau, and Rui Zhang. CON- TaiNER: Few-shot named entity recognition via contrastive learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), 2022.
[20] Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. Automated hate speech detection and the problem of offensive language. Proceedings of AAAI on Web and Social Media, 2017.
11

---

[21] Ona de Gibert, Naiara Perez, Aitor Garcia-Pablos and Montse Cuadros. Hate speech dataset from a white supremacy forum. In Proceedings of Workshop on Abusive Language Online (AI.22), 2018.
[22]
De Marneffe, Mandy Simons, and Judith Tonhauser. The commitmentbank: Investigating projection naturally occurring discourse. In Proceedings of Sinn und Bedeutung. 2019.
[23] Leon Derezynski, Eric Nichols, Maricke van Erp, and Nut Limsopatham. Results of the WNUT2017 shared task on novel and emerging entity recognition. In Proceedings of Workshop on Noisy User- Aser-generated Text, 2017.
[24] Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL. 2019.
[25] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Pruc. of NAACL, 2019.
[26] Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang. Xu Han, Pengjun Xie, Haitao Zheng, and Zhiyuan Liu. Few-NERD: A few-shot named entity recognition dataset. In Proceedings ACL-IJCNLP 2021.
[27] Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernnncourt, Jiuxiang Gu, Tong Sun, and Xia Hu. Towards interpreting and mitigating shortcut learning behavior of NLU models. In Proceedings of NAACL:HLT, 2021.
[28] Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Güney, Volkan Cirik, and Kyunghyun Cho. Searchqa: A new q&a dataset augmented with context from search engine. CoRR, 2017.
[29] Mai EISherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choudhury, and Diyi Yang. Latent hatred: A benchmark for understanding implicit hate speech. In Proceedings of EMNLP. 2021.
[30] Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, and Danqi Chen. MRQA 2019 shared task: Evaluating generalization in reading comprehension. In Proceedings of 2nd Muchine Reading for Reading Comprehension (MRQA) Workshop at EMNLP, 2019.
[31] Tianyu Gao. Xingcheng Yao, and Danqi Chen. SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of EMNLP, 2021.
[32] Matt Gardner, William Merrill, Jesse Dodge, Matthew E Peters, Alexis Ross, Sameer Singh, and Noah Smith. Competency problems: On finding and removing artifacts in language data. arXiv preprint arXiv:2104.08646. 2021.
[33] Karan Goel, Nazneen Rajani, Jesse Vig, Samson Tan, Jason Wu, Stephan Zheng, Caiming Xiong, Mohit Bansal, and Christopher Ré. Robustness gym: Unifying the nlp evaluation landscape, arXiv preprini arXiv:2101.04844 2021.
[34] Karan Goel, Nazneen Fatema Rajani, Jesse Vig, Zachary Taschdjian, Mohit Bansal, and Christopher Ré. Robustness gym: Unifying the NLP evaluation landscape. In Avi Sil and Xi Victoria Lin, editors, Proceedings of NAACL-HLT, 2021.
[35] Tanya Goyal, Junyi Jessy Li, and Greg Durrett. News summarization and evaluation in the era of gpt-3. CORR, 2022.
[36] Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. ToxiGen: A large-scale machine -ee.erttee dataset for adversarial and implicit hate speech detection. In Proceedings of ACL, 2022.
[37] He He, Sheng Zha, and Haohan Wang. Unlearn dataset bias in natural language inference by fitting the residual. arXiv preprint arXiv: 1908. 10763, 2019.
[38] Pengeheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: bert with disentangled attention. In Proceedings of ICLR, 2021.
12

---

[39] Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. Pretrained transformers improve -uttt--distriuutionn robustness. In Proceedings of ACL, 2020.
[40] Dan Hendryeks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. Pretrained transformers improve out-of-distribution robustness. arXiv preprini urXiv:2004.061000 2020.
[41] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. 2019.
[42] Xiaolei Huang and Michael J Paul. Examining temporality in document classification, In Proceedings of ACL, 2018.
[43] Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulls, Karim Lasri, Naomi Saphra, Arabella Sinclair, Den- nis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Koustuv Sinha, Leila Khalatbari, Maria Ryskina, Rita Frieske, Ryan Cotterell, and Zhijing Jin. State-of-the-art generalisation research in NLP: a taxonomy and review. CoRR, 2022.
[44] Andrew llyas. Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In Proceedings of NeurIPS, 2019.
[45] Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. TriviaQA: large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of ACL, 2017.
[46] Divyansh Kaushik, Eduard Hovy, and Zachary C. Lipton. Learning the difference that makes difference with aounterfactually-aagmeeee data. In Proceedings of ICLR, 2020.
[47] Pride Kavumba, Ryo Takahashi, and Yusuke Oda. Are prompt-based models clueless? In Proceedings of ACL, 2022.
[48] Zixuan Ke, Bing Liu, Hao Wang, and Lei Shu. Continual learning with knowledge transfer for sentiment classification. In Proceedings of ECML-PKDD, 2020.
[49] Yuta Koreeda and Christopher Manning. ContractNLI: A dataset for document-level natural language inference for contracts. In Findings of EMNLP, 2021.
[50] Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: A benchmark for question answering research. Transactions of ACL. 2019.
[51] Thai Le, Jooyoung Lee, Kevin Yen, Yifan Hu, and Dongwon Lee. Perturbations in the wild: Leveraging human-written text perturbations for realistic adversarial attack and defense. arXiv preprint arXiv:2203.10346, 2022.
[52] Seanie Lee, Minki Kang, Juho Lee, and Sung Ju Hwang. Learning to perturb word embeddings for out-of-distribution QA. In Proceedings of ACL-IJCNLP, 2021.
[53] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for earfmeter-efffcient prompt tuning. arXiv preprint arXiv: 104.0099, 2021.
[54] Brian Lester, Rami AI-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Proceedings of EMNLP, 2021.
[55] Jiazhao Li, Yijin Yang, Zhuofeng Wu, V. G. Vinod Vydiswaran, and Chaowei Xiao. Chatgpt as attack tool: Stealthy textual backdoor attack via blackbox generative model trigger. CoRR, 2023.
13

---

[56] Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. Textbugger: Generating adversarial text against real-world applications. arXiv preprint arXiv:1812.05271, 2018.
[57] Chen Liang, Simiao Zuo, Minshuo Chen. Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, and Weizhu Chen. Super tickets pre-trained language models: From model compression to improving generalization. In Proceedings of ACI-IJCNIPP, 2021.
[58] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. 2017.
[59] Alísa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. Wanli: Worker and ai collaboration for natural language inference dataset creation. In Findings of EMNLP, 2022.
[60] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. CoRR, 2019.
[61] Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Ziwei Ji, Samuel Cahyawijaya, Andrea Madotto, and Pascale Fung. Crossner: Evaluating cross-doman named entity recognition. In Proceedings of AAAI, 2021.
[62] Florian Ludwig, Klara Dolos, Torsten Zesch, and Elcanor Hobley. Improving generalization of hate speech detection systems to novel target groups via domain adaptation. In Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH), 2022.
[63] Xiaofei Ma. Peng Xu, Zhiguo Wang, Ramesh Nallapati, and Bing Xiang, Domain adaptation with bert-based domain classification and data selection. In Proceedings of the 2nd Workshop Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), 2019.
[64] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng. and Christo- pher Potts. Learning word vectors for sentiment analysis. In Proceedings of NAACL, 2011.
[65] Julian McAuley and Jure Leskovec. Hidden factors and hidden topics: Understanding rating dimensions with review text. In Proceedings of ACM Conference on Recommender Systems, 2013.
[66] R Thomas McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. arXiv preprint arXiv:1902.01007, 2019.
[67] John Miller, Karl Krauth, Benjamin Recht, and Ludwig Schmidt. The effect of natural distribution shift on question answering models. In Proceedings of ICML, 2020.
[68] John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization. In Proceedings of ICML, 2021.
[69] Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? Proceedings of EMNLP, 2022.
[70] Preslav Nakov, Alan Ritter, Sara Rosenthal, Fabrizio Sebastiani, and Veselin Stoyanov. SemEval-2016 task Sentiment analysis in Twitter. In Proceedings of International Workshop Semantic Evaluation (SemEvall), 2016.
[71] Isar Nejadgholi and Svetlana Kiritchenko. On cross-dataset generalization in automatic detection of online abuse. In Proceedings of the Fourth Workshop on Online Abuse and Harms, 2020.
[72] Nathan Ng. Kyunghyun Cho, and Marzyeh Ghassemi. SSMBA: Self-superviser manifold based data augmentation for improving out-of-domain robustness. In Proceedings of EMNLP, 2020.
14

---

[73] Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adversarial nli: A new benchmark for natural language understanding. In Proceedings of ACL, 2020.
[74] Bhargavi Paranjape, Pradeep Dasigi, Vivek Srikumar, Luke Zettlemoyer, and Hannaneh Hajishirzi. Agro: Adversarial discovery error-prone groups for robust optimization. 2022.
[75] Christopher Potts, Zhengxuan Wu, Atticus Geiger, and Douwe Kiela. DynaSent: A dynamic benchmark for sentiment analysis. In Proceedings of ACL-IJCNLP, 2021.
[76] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of ICML, 2021.
[77] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 2020.
[78] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of EMNLP, 2016.
[79] Alan Ramponi and Barbara Plank. Neural unsupervised domain adaptation in nlp-a survey. arXiv preprint arXiv:2006.00633 2020.
[80] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. In Proceedings of EMNLP, 2019.
[81] Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang. Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong. Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Stella Biderman, Leo Gao, Tali Bers, Thomas Wolf, and Alexander M. Rush. Multitask prompted training enables zero-shot task generalization, 2022.
[82] Zhengxiang Shi, Qiang Zhang, and Aldo Lipani. Stepgame: new benchmark for robust multi- spatial reasoning in texts. In Proceedings of AAAI, 2022.
[83] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP, 2013.
[84] Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zhigniew Wojna. Re- thinking the inception architecture for computer vision. In Proceedings of CVPR, 2016.
[85] Aarn Talman and Stergios Chatzikyriakidis. Testing the generalization power of neural network models across NLI benchmarks. In Proceedings of ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 2019.
[86] Alon Talmor and Jonathan Berant. MultiQA: An empirical investigation of generalization and transfer in reading comprehension. In Proceedings of ACL, 2019.
[87] Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Measuring robustness to natural distribution shifts in image classification. In Proceedings of NeurIPS, 2020.
[88] Erik F. Tjong Kim Sang and Fien De Meulder. Introduction to the con1l-2003 shared task: Language-inden innuunge-nndependent named entity recognition. In Proceedings of NAACL, 2003.
[89] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models, ArXiv, 2023.
15

---

[90] Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. NewsQA: A machine comprehension dataset. In Proceedings of Workshop on Representation Learning for NIP,, 2017.
[91] Lifu Tu, Garima Lalwani, Spandana Gella, and He He. An empirical study on robustness to spurious correlations using pre-trained language models. Transactions of the Association for Computational Linguisties, 2020.
[92] Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. Superglue: A stickier benchmark for general-purpose language understanding systems. In Proceedings of NewrlPS, 2019.
[93] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.074
2018.
[94] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of ICLR, 2019.
[95] Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, and Jingjing Liu. Infobert: Improving robustness of language models from an information theoretic perspective. 2021.
[96] Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. arXiv preprint arXiv:2 (eX:v:22//..00 /.028400 2021.
[97] Tianlu Wang, Rohit Sridhar, Diyi Yang, and Xuezhi Wang. Identifying and mitigating spurious correlations for improving robustness in nlp models. In Findings of NAACL: 2022.
[98] Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang. Rui Zheng. Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang. Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing. Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, and Xuanjing Huang. TextFlint: Unified multilingual robustness evaluation toolkit for natural language processing. In Proc. of ACL, 2021.
[99] Xuezhi Wang, Haohan Wang. and Diyi Yang. Measure and improve robustness in NLP models: A survey. In Proceedings of NAACL, 2022.
[100] Jason Wei, Yi Tay, and Quoe V. Le. Inverse scaling can become u-shaped. ArXiv, 2022.
[101] Jason Wei and Kai Zou. EDA: Easy data augmentation techniques for boosting performance text classification tasks. In Proceedings of EMNLP-IJCNLP, 2019.
[102] Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of NAACI, 2018.
[103] Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel Weld. Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models. In Proceedings of ACL- IJCNLP, 2021.
[104] Xing Wu, Chaochen Gao, Liangjun Zang, Jizhong Han, Zhongyuan Wang, and Songlin Hu. Esimese: Enhanced sample building method for contrastive learning of unsupervised sentence embedding. In Proceedings of COLING, 2022.
[105] Hongwei Xue, Yuchong Sun, Bei Liu, Jianlong Fu, Rui Song. Houqiang Li, and Jiebo Luo. Clip-vip: Adapting pre-trained image-text model to video-language representation alignment. In Proceedings of ICLR, 2022.
[106] Linyi Yang, Lifan Yuan, Leyang Cui, Wenyang Gao, and Yue Zhang. Factmix: Using few labeled in-domain examples to generalize to cross-domain named entity recognition. In Proceedings of COLING, 2022.
16

---

[107] Linyi Yang. Shuibai Zhang, Libo Qin, Yafu Li, Yidong Wang, Hanmeng Liu, Jindong Wang. Xing Xie, and Yue Zhang. Glue-x: Evaluating natural language understanding models from an out-of-distribution generalization perspective. In Findings of ACI., 2023.
[108] Zhilin Yang, Peng Qi. Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of EMNLP, 2018.
[109] Hai Ye, Yuyang Ding, Juntao Li, and Hwee Tou Ng. Robust question answering against distribution shifts with test-time adaption: An empirical study. In Findings of EMNLP, 2022.
[110] Wenpeng Yin, Dragomir Radev, and Caiming Xiong. Docnli: A large-scale dataset for document-level natural language inference. In Findings of ACL, 2021.
[111] Dani Yogatama, Cyprien de Masson d' Autume, Jerome Connor, Tomas Koeisky, Mike Chrzanowski, Lingpeng Kong, Angeliki Lazaridou, Wang Ling, Lei Yu, Chris Dyer, and Phil Blunsom. Learning and evaluating general linguistic intelligence, 2019.
[112] Lifan Yuan, Yichi Zhang, Yangyi Chen, and Wei Wei. Bridge the gap between CV and nip! A bradient-based textual adversarial attack framework. In Findings of ACL, 2023.
[113] Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. Predicting the type and target of offensive posts in social media. In Proceedings of NAACL, 2019.
[114] Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can machine really finish your sentence? In Proceedings of ACL, 2019.
[115] Aston Zhang, Zachary Chase Lipton, Mu Li, and Alex Smola. Dive into deep learning. CoRR, 2021.
[116] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In Proceedings of NeurlPS, 2015.
[117] Yuan Zhang, Jason Baldridge, and Luheng He. Paws: Paraphrase adversaries from word scrambling. arXiv preprint arXiv: 0904.014411111 1300, 2019.
[118] Chen Zhu, Yu Cheng. Zhe Gan, S. Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced adversarial training for natural language understanding. In Proceedings of ICLR, 2019.
17

---

Table 7: The comparison among different semantic representation methods, from which we can observe that despite different methods varying to some extent, the trend of similarity almost remains
unchanged.
Methed
SunCSR
ESinCSE
Sentencx- Sentezax-Trasseerm
Sec
SST
Yelp
DSC
5ST
Yelp
DSC
D5
Yelp
A7
110)
$6.10
67.30
24.54
15,30
40.20
05.
76.07
76.50
76.
81.75
100)
00:25
06.57
05.00
30.65
48.92
05.35
DSc
86-02
103.00
99.15
$4.55
51 1/00
48.00
15.05
09:37
118
64:23
89:06
76.08
10.90
79:36
00.15
100)
17.37
01.53
25.28
$4.79
-5.49
DS
57.30
59.15
100
32,50
28.13
10.08
$8.99
88.46
84.22
1.00
73.64
79.25
74.41
90.52
46:57
57.27
100
24.76
36.94
32.65
78.37
IMDS
66.62
19.88
99:06
118
100
60.97
02.44
12.65
24.55
Yolp
-9.22
55.45
30.99
39.58
24.03
19.43
100
91.75
79.26
90.52
72.62
08.64
62.44
100
49.23
49.49
78.37
34.05
19.36
24.58
100
Appendix
A
Frequently Asked Questions
A.1 What is the rationale for current task selection and why not include more difficult tasks?
The chosen tasks within BOSS evaluate models across natural language understanding, structured data prediction, and question answering, which are core language model competencies. In constructing our benchmark, we did consider extending other tasks such as NLG and commonsense reasoning, but in practice, we found a lot of difficulties.
For NLG tasks, the primary concern lies in evaluation. Since different datasets exhibit distinct text styles, an in-domain (ID) model might generate responses with styles diverging from the reference answers when tested on out-of-domain (OOD) datasets. However, current NLG metrics evaluate predictions based on their resemblance reference answers, which might accurately reflect text quality in scenarios with a vast output space [35]. An emerging alternative involves employing Language Models like GPT-4 for scoring predictions. However, this method is cost-intensive and lacks reproducibility due to unpredictable updates. Considering these evaluation challenges, including generation tasks within this OOD benchmark might not be appropriate.
For commonsense reasoning, multiple datasets exist from various sources, each demanding distinct knowledge. These differences substantial enough that knowledge gained from ID dataset might not be applicable to OOD datasets. For instance, HellaSwag [1 14] necessitates basic world knowledge and logical reasoning complete sentence, while StepGame [82] relies spatial imagination without requiring world knowledge. The dissimilarity in required abilities suggests that models should acquire knowledge through pre-training rather than fine-tuning on an ID dataset and then transferring it to OOD tasks. This approach aligns more reasonably with the task's 's nature.
A.2 Why is SimCSE chosen to measure distribution shift?
We refer to [105] as representative study measuring text distribution shift via semantic vector distance. While [105] employs CLIP's encoder for multimodal contexts, our NLP-specific context steers us towards SimCSE for semantic representation.
To the best of our knowledge, SimCSE and sentence-Bee -ERT [80] are the most widely used models and SimCSE is more advanced according to the SimCSE paper, hence choose SimCSE the semantic representation model. In this discussion period, we also try the other variants of SimCSE, ESimCSE (all-mpnet-base-v21) The
[104], and the best model in the Sentence Transformers leaderboard results are shown in Table 7. From the results, despite different methods varying to some extent, the trend of similarity almost remains unchanged. Therefore, we consider that the essential adopt an advanced semantic representation model, and that which particular model we choose will not lead to differences in our selection.
B Survey
Our survey draws from two NLP taxonomy papers, one focusing robustness [99] and the other on generalization [43]. We meticulously examine all the references cited in these papers, presenting
hatttps:/www.sbart net wwtp://tet/eeeeeeeeeeeeeeeeeeeee models .tml #mode -vvervvee
18

---

Table 8: Survey of papers targeting to address OOD robustness across the five tasks as this paper. Column "Paper" signifies individual papers sorted by date of publication, column "Tasks" details the evaluated tasks, and column "Datasets" lists the encompassed datasets, with each row representing ID dataset + OOD datasets.
Paper
Tiskss
Danasets
[66]
5A
IMDh
Amazon, SemFoal, Yelp
[85
NLI
MNLL SNLI, SICK
MNLL SNLI, SICK
EQA
SQUAD, NewaQA, TaviaQA, SeanchQA, HoppuQQ, Natural Questions
BioASQ, DROP, DuoRC, RACE, RelstinExtrsttion Texibook
SQUAD, NewsQA, SearchQA, TraaQA, HerporQA
CWQ, ComsJA, WikiHop, DROP
[11]]
SQuAD TriviqQA, QuAC. QA-SRL, QA-ZRE
Yelp,
IMDs, Yelp, Amazon
[72]
SA
NLI
MNLI, ANLI -> MNLI, ANLI
Amazon, Yelp,
Amsunn, Yelp,
[7]]
TD
Wiki
Founia, Waseem
NLI
MNLI
HANS
NLI
MNLL FEVER ANLI
Squidshin
[[0]]
SA
SNIJ -> SNLI-CAD, break, DNC, stress diagnottic
Semiival, Yelp. IMDB-Cont,
[1]]
SA
NLI
SST25 - IMDD Cosr, IMD6-CAD
MNLI ANLI, ILANS
SA
SST2 IMDR
NLI
MNLI ANLI
EQA
-3 Symmetric, HANS
[57]
NLI
MNLI + SNLL, ScThil
[52]
EQA
SQuAD -
BiaASQ. New Wikipedia, New York Times, Reduit posts, Amazon Reviews
[97]
5A
SST2, Amazon kitchen, Amazon electronies -+ SST Amtaznn Kitchos, Amazon electmonies
SA
SST2 + Sencil 40, SemEval Yelp, IMDB, Contrast, CAD
NLI
MNLI LL. HANSS, WaNLI SNLL ANLI
[107]
Yelp,
SNLJ, SICK
[62]
TD
HoteXplain
[67]
NLI
MNLLSNLI HANS
[106]
NER
CoNEI + CrosNER NER
[19]
NER
OntoNotes + 1282' 14, CONLL'OS, WNUT GUM, Few NERD
[00]
FQA
SQUAD
NewsQA, SeanhQA, TriviaQA, HorporQA, Natural Questions
subset of references that specifically address OOD robustness in sentiment analysis (SA), toxic detection (TD), natural language inference (NLI), name entity recognition (NER), and extractive question answering (EQA) tasks. The summarized findings are outlined in Table 8. Through the survey, we can observe that there is no single uniform set datasets for evaluation NLP, namely no "standardized OOD benchmark suites tailored for NLP". And we find that existing work includes evaluation datasets without particular considcration while mainly directly using popular datasets, i.c. adopting "popularity-base -based dataset selection strategies".
C Datasets
In section 2.4, we outlined the datasets utilized the BOSS benchmark. In this section, we provide additional descriptions for the datasets not covered in the main body of this paper, and elaborate on data processing procedures for all candidate datasets.
C.1 Description
Sentiment Analysis. DSC [48] consists of product reviews of different categories in Amazon, with binary sentiments. IMDb [64] is another binary classification dataset with movie reviews from the IMDb website. Yelp [116] dataset is constructed by extracting product reviews from the Yelp
website.
Toxic Detection. AbuseAnalyzer [10] collects abusive contents from from Grab, with labels of Hateful and n AdvCivil is new adversarial dataset for toxic detection introduced in this paper. We employ the adversarial attack technique proposed by [12] to simulate real-world ma-
19

---

licious attacks by introducing typos and distracting sentences into the original sentence. Additionally, a post-human inspection process is undertaken to validate the constructed samples. Three human an- notators are employed to filter out label-inconsistent and semantically broken samples. Hate Speech [21] covers posts on various topies and nationalities from several sub-forums on Stormfront. HSOL [20] extracts tweets containing pre-defined toxic words or phrases and categorizes them into three classes: hate speech, offensive but not hate speech, or
of offensin ive nor hate speech. OLID [113] collects tweets and annotates each of them according to whether the text is offensive or not.
Natural Language Inference. [S] is created from abstracts of biomedical publications on PubMed, with each sample labeled as entai led or not-enta entail led and regarding the experimen- tal evidence as the premises. The entai led samples directly adopt the conclusions of mechanistic information in the abstracts as the hypothesis, while the not-entan iled samples manipulate the information generate pseudo conclusions with nine different strategies. CB [22] consists of clause- embedded texts from Wall Street Journal, British National Corpus, and Switchboard, each premise containing an embedded clause and the hypothesis being the extraction of the clause. DocNLI [110] processes five existing datasets of different tasks into NLI formats, all from news or Wikipedia. The premises are all the document level, while the hypotheses have variety of different granularities. SNLI [7] extracts image captions from Flickr30k premises, and generates hypotheses by human annotation.
Name Entity Recognition. CrossNER collect sentences from Wikipedia topics of artificial intelligence, literature, music, natural science, and politics, covering seven coarse-grand sarree-rained entity types in Few-NERD except for the Building.
Extractive Question Answering. HotpotQA [108] is based on Wikipedia, with each question requiring information from two articles to answer. NaturalQuestions [50] takes real queries from Google search engine as questions, corresponding search results as contexts, and -nnootatd texts as answers. Trivia-QA [45] collects long documents from Wikipedia and constructs questions that can not be directly answered by span extraction. SQuAD Shifts [67] inherits the dataset construction pipeline from SQuAD, but collects passages from more diverse sources, including Wikipedia, New York Times, Reddit, and Amazon.
C.2 Processing
C.2.1 Sentiment Analysis
Generally, datasets for this task are label-imbalanced, which is detrimental to model performance. Therefore, If not specified, we tackle this issue by aligning n; with min(n1..... (ellassss) } by discarding redundant samples in each class, where |classes| is the number of classes and nı is the number samples for label i each dataset, E (1...., Jclassel.............
Amazon has 29 subsets of different products. To encourage diversity of the review texts, we aim to merge all the subsets together. However, the scale of the entire dataset is prodigious and the sizes of the subsets vary greatly, making model training consumngg and potentially ignoring the effect of certain types of reviews. Therefore, we first reduce large subsets 20k samples, and then drop samples of stars 2 or 4 and maintain samples of starts 1, 3, and 5. After that, we get temary classification of the entire dataset, split into training and test datasets by 9:1, Finally, for label balance, we sample 10k reviews for each class in the training dataset and discard the rest.
DSC is binary classification dataset, thus samples of both labels will be retained. The final training and test dataset are formed by combining all types of product reviews.
Dynasent has two rounds of training and test datasets, which are incorporated together to constitute unified training and test dataset.
IMDb and EvmE only require label balancing processing.
SST samples annotated with float score indicating the sentiment, ranging from 0 to 1. We follow the common practice of SST-5 to equally divide the score into 5 bins, i.c. mapping score 00.2 to
20

---

label 1, and so on. Then we drop samples of labels and 3, similar to Amazon, to adapt for ternary classification.
Yelp reviews are also rated from to 5 and retained only samples of stars 1, 3, and 5 following Amazon and SST. Due the large amount of data, sample 10k reviews from the training dataset for each label.
C.2.2 Toxic Detection
Datasets for this task are collected from online forums and social media, therefore the texts are commonly dirty, containing some meaningless strings such as @username in the beginning. We clean the texts by removing @username, emoji, tags (with a symbol #), and URLs. Moreover, the label balancing processing the same as for Sentiment Analysis datasets.
A AbuseAnalyzer does not have train/test split, thus we divide the original dataset into training dataset and test dataset by 8:2.
Civil Comments samples with the toxicity score 0.5 considered toxic samples while others whose toxicity score 0.5 are in the benign class.
Hate Speech is divided into training and test datasets by 8:2.
HSOL has three types of label: hate speech, offensive but not hate speech, or neither offensive nor hate speech. We adapt it to binary classification, treating sam- ples with the first two toxic and samples with the label neither offensive nor hate speech as benign.
Implicit Hate annotated into three classes, i.e., not hate, implicit hate and expl 1 icit hate, with implicit hate and explic icit hate being considered toxic. In probing experi- ments, split the training and test dataset by 8:2. After selecting the Implicit Hate dataset into benchmark, we treat the entire dataset as the test dataset.
AdvCivil, OLID and ToxiGen do not require specific processing.
C.2.3 Natural Language Inference
Datasets for NLI do not have a serious label-imbalance problem, hence no corresponding processing applied.
ANLI has three rounds total, we merge them together to form the final training and test datasets respectively.
DocNLI contains some ANLI examples, so we filter those samples out to avoid repeating ANLI
Other datasets, i.e., BioNLI, CB, ContractNLI, MNLI, SNLI, and WANLI, do not require any other specific processing. But note that the test dataset of MNLI the matched validation dataset, and we do not use the unnatcchdd one since the two subsets are too similar.
C.2.4 Name Entity Recognition
Datasets for NER usually have different name entity tags and thus require label mapping for alignment. For the cross-evaluation experiment, we align the labels of each dataset to CoNLLI; while for other experiments, we take the scheme Few-NERD as the standard. Also, we adapt the datasets to the prevalent BIO schema, which indicates whether token is the Beginning, the Inside, the Outside of an entity.
CoNLL contains the four most common entity types which are covered in Few-NERD, thus no label mapping needed.
CrossNER constitutes various entity types from diverse domains. Therefore, we design the following mapping for label alignment:
Label 1 laepplmmmppngggga
"academicjournal" "product",
"album": "product",
"algorithm";
21

---

"astroooomicalcbeee""", "miscellaneour"
"award": "miscellaneous",
"band": "organization",
"book"; *art", "chemicalccoppunnd" "miscellancous",
"chemlcaaleleentt". "miscellancous",
"conference": "event",
"country": "lecation",
"discipline": "miscellaneous",
"election": "event",
"enzyme": "miscelLaneous",
"event": "event",
"field": maiscel,aneuus, "literarygenre": "art", "location": "location", "magazine": "product", "metrics": "miscellaneous",
"misc": "miscellanoouus"
"musicalartist": "person", "musicalinstrrmentt; "product", "musicgenre"; "art", "organisation": "organization", "person""""prrs""""""" "persen", "poem": "art", "politicalparty": "organization", "politician": "persen",
"product": "product,
"programiang": "miscel sassceell,,,s",, "protein": "miscellaneous", "researcher"; "person", "scientist"; "person", "song": "art",
"task": "miscellanecus" "theory": "miscellancous", "university"; "organization", "writer": "person"
Few-NERD requires processing to adapt for the scheme of CoNLL in the probing experiments. Since it has covered all the entity types annotated in CoNLL, the only process is to set other tags, i.c. bui lding, art, product. and event, to be sriscellaneoe Note that this process is also required for CrossNER and WNUT in the evoss-evaluation experiment, since these two datasets have already been aligned with Few-NERD.
WNUT conduct the following operation to align labels with Few-NERD:
labe1 __aappng -
"corporation": "organization", "creative-work": "art", "group": "organization", "Location": "Location",
"person": "person", "product"; "product"
C.2.5 Extractive Question Answering
Datasets are all normalized to a unified extractive setting with the same format as SQuAD, following MRQA [30].
AdvQA has several subsets generated by fooling different models. We adopt the combination of all these subsets for our experiments.
22

---

Table 9: Statistics of toxic detection candidate datasets.
Source
Classes
# Samples
Avg.
Length
Train
Test
Train
Test
AbuseAnalyyer
Grab
2
6,080
1,520
14.24
14.38
AdvCivil
Adversarial
823
70.73
Civil Comments
Civil Comments
60,000
97,320
$0.09
51.15
Hate Speech
Stornfront
2
8,562
2,141
18.07
18.1
HSOL
Twitter
2
5,823
2,485
13.37
13.1
Implicit Hate
Twitter
21,480
16.81
OLID
Twitter
13,240
860
19.62
23.16
ToxiGen
Synthetic
8,960
9400
18.14
18.63
Table 10: SimCSE scores between each pair of datasets regarding the toxic detection task. AA: AbuseAnalyzer, AC: AdvCivil: CC: Civil
Comments; HS: Hate Speech; IH: Implicit Hate; TG: ToxiGen.
Train
Test
AA
AC
CC
HS
HSOL
IH
OLID
TG
AbuseAnalyzer
100
80.05
79.60
78.89
73.97
75.78
85.33
63.06
AdvCivil
80.05
100
95.17
59.07
51.35
69.54
74.49
58.63
Civil Comments
79.60
95.17
100
66.89
50.96
69.05
76.20
62.09
Hate Speech
78.89
59.07
66.89
100
64.96
76.02
70.03
74.00
HSOL
73.97
51.35
50.96
64,96
100
41.73
69.80
43.32
Implicit Hate
75.78
69.54
76.02
41.73
100
64.60
79.79
OLID
85.33
74.49
76.20
70.03
69.80
64.60
100
51.76
ToxiGen
63.06
58.63
62.09
74.00
43.32
79.79
51,76
100
Table 11: The OOD performance of the T5-large when trained on the
Civil Comments dataset. AA: AbuseAnalyzer; AC: AdvCivil; CC:
Civil Comments; HS: Hate Speech; IH: Implicit Hate; TG: ToxiGen.
Train
Test
AC
HS
HSOL
IH
OLID
Civil
57.47
80.43
75.98
63.77
68.83
SQuAD can be used in the original format, while other datasets i.e. HotpotQA, NaturalQuestions, NewsQA, SearchQA, and Trivia-QA. adopted from MRQA to fit the extractive setting.
D Dataset Selection for Other Tasks
In section 2.3, we have taken the task of sentiment analysis example demonstrate how to select ID) and OOD datasets for our benchmark. Next, we will explain how we choose datasets for other tasks.
D.1 Toxic Detection
Candidate Datasets. We use the same approach for searching toxic detection datasets as we do for sentiment analysis. We gather several dataset candidates, including Civil Comments, Hate Speech, HSOL, Implicit Hate, OLID, and ToxiGen. We aim to adopt an adversarial dataset for each NLU task, which is lacking in the current literature. Hence, we later construct new dataset through adversarial attacks on the chosen ID dataset.
Probing Experiments.
The setup is the same as sentiment analysis.
The dataset information can found in Tables 9. The sources are very diverse, with each source, except Twitter, corresponding to only one dataset. For the ID dataset, Civil Comments significantly larger than all the other datasets, and it is the only one containing more than 10k samples for each class. In addition, Civil Comments contain 6 subtypes of toxicity and 24 types of targeted identity, meeting the requirement for dataset diversity. Considering the dataset size and the text source diversity, choose Civil Comments ID dataset for toxic detection.
Results.
23

---

Table 12: Statistics of natural language inference candidate datasets.
Datasct
Source
Classes
Train
Samples
Test
Train
Avg.
Length (p)
Avg.
Train
Length (h)
Tost
ANLI
Adversarial
3
162,865
3,200
54.13
54.42
9.6
10.22
BioNLI
Biomedical
26.91
Wall Street Joural &
British National Corpus
Switchhoard
ContractNLI
Legal
3
7.191
2,091
1673.63
1708.82 82
12.82
12.82
DocNLI
News Wikipedia
942,314
263,886
318.89
385.13
47.5
71.88
MNLI
Open American National Corpus
3
392,662
9,815
19.81
19.27
9.97
9.92
Flickr
549.361
7.42
7.48
WANLI
Synthetic
102,884
5,000
17.49
17.48
9.93
9.83
Table 13: SimCSE scores between each pair of datasets regarding the natural language inference task.
Train
Test
ANLI
BioNLI
CB
ContractNLII
DocNLI
MNLI
SNLI
WANLI
BioNLI
31.32
100
5,51
10.39
28.07
2.28
-4.78
-0.12
CB
ContractNLI
21.99
10.39
17.81
100
19.96
3.62
-7.17
6.12
DocNLI
53.79
28.07
40.95
19:96
100
34,91
21.23
28.00
MNLI
16.27
2.28
68.11
3.62
34,91
100
47.42
85.23
SNLI
21.99
-4.78
41.88
-7.17
21.23
47.42
100
46.80
WANLI
22.41
-0.12
59.17
6.12
28.00
85.23
46.80
100
Table 14: The OOD performance of the TS-large when trained on the MNLI dataset.
Train
Test
ANLI
BioNLI
ContracNLLI
DocNLI
MNLI
SNLI
WANLI
MNLI
36.19
77.63
63.40
37.06
75.67
89,40
87.32
63.32
Next, we consider OOD datasets from other text sources. As aforementioned, first supplement adversarial dataset for toxic detection, i.e. AdvCivil, based on Civil Comments. The construction details can be found in Appendix C.1. Then we consider the semantic similarity among the dataset candidates. From the results in Table 10, we can filter out Analzer and OLID because of their high similarity with Civil Comments. We also observe high similarity between AdvCivil and Civil Comments, but we do not disregard the former dataset at this stage since the high similarity has an acceptable attribution considering the construction process. Moreover, both HSOL and Implicit Hate come from tweets, thus they can not coexist in the benchmark. We are leaning to select Implicit Hate since it contains more challenging implicit toxicity.
Thereafter, we have four datasets left, i.e. AdvCivil, Hate Speech, Implicit Hate, and ToxiGen. We still need drop one dataset based performance degradation. According to results in Table 11, AdvCivil, Implicit Hate, and ToxiGen can provoke a performance drop of over 20 points. By contrast, the ID model still achieve accuracy over 80 Hate Speech, indicating that this shift may be the least challenging. Therefore, we discard Hate Speech and adopt the other three as the OOD datasets. It is also worth noting that Implicit Hate leads to much more severe performance drop than HSOL, supporting our previous claim. Finally, the distribution shift for toxic detection is Civil Comments > (AdvCivil, Implicit Hate, ToxiGen).
D.2 Natural Language Inference
Candidate Datasets. We investigate datasets Paperswitheode and finally include ANLI, BioNLI, CB, ContractNLI, DocNLI, MNLI, SNLI, and WANLI as the candidates.
Probing Experiments. For semantic similarity evaluation, we only feed premises in each dataset to the unsupervised SimCSE model', instead of concatenating the premises and hypotheses together. Note that we do not adopt the supervised SimCSE model here, because was contrastively trained
24

---

Table 15: Statisties of named entity recognition candidate datasets.
i Samples
Avg.
Length
Dutaset
Souree
Classes
Train
Test
Train
CoNLL CrosSNER
14,042
3,454
13.44
Wikipedia
7
701
2,507
38.46
38.22
E-NER
Legal
11,692
34.52
Few NERD
Wikipedia
8
131,768
37,649
24.49
24.47
YouTube Twitter
WNUT
StuckExchange Reddit
6
3,395
1,288
18.48
18.16
Table 16: SimCSE scores between each pair of datasets regarding the name entity recognition task.
Train
Test
CoNLL
CrossNER
E-NER
Fow-NERD
WNUT
CoNLL
62.86
73.76
45.60
CrosNNER
62.86
100
37.19
87.47
44.84
E-NER
43.48
37.19
100
42.29
25.85
Few-NERD
73.76
87.47
42.29
100
45.74
WNUT
45.74
100
on NLI datasets, thus may lead to a bias in evaluating the semantic similarity between NLI datasets. For cross-evaluation experiments, the setup is the same semantic analysis.
Results. Dataset information is shown in Tables 12. These NLI datasets are commonly large and their text sources do not overlap. Based on these results, we start to pick the ID dataset. First all, BioNLI and DocNLI should be excluded from the discussion of ID dataset selection, because they only have two classes but all the other candidates have three categories. Otherwise, the trained ID model may fail distinguish the class Neutral and contradiction a label shift [43]. Then, consider the dataset size. The majority of candidates contain over 100k training samples, while CB only has about 300 samples in total and ContractNLI has less than 10k samples, much smaller than others. Therefore, these two datasets should be excluded too. Among the remaining datasets, ANLI and WANLI mainly contain adversarial and challenging features, instead of general ones; MNLI claims to be more diverse than SNLI since it consists both written and spoken English and contains 10 sub-genres, thus more abundant in text styles and topics. Hence, we pick MNLI as the ID dataset.
namely suffering from
For OOD datasets, since there is dataset drawn from the same source with MNLI, only semantic similarity and performance drop should be involved. From Table 13, the similarities between NLI datasets and MNLI are all relatively low, except for WANLI, CB, and SNLL We will ignore CB and SNLI in the later procedure but still remain WANLI because WANLI is based on MNLI and thus is reasonable be somehow similar to MNLI. Thereafter, examine the performance drop caused by each dataset. As shown in Table 14, the ID model presents little performance degradation on BioNLI and DocNLI, while suffering significant degradation on the other three datasets. Hence, the distribution shifts for NLI will be MNLI + (ANLI, ContractNLI, WANLI).
D.3 Name Entity Recognition
Candidate Datasets. Since NER datasets typically have different sets of entity type labels that require specific domain knowledge, we loosen the search standard to include the datasets that have partially overlapping entity type labels, rather than requiring an exact match. We find five suitable candidate datasets, i.e. CoNLL, CrossNER, E-NER, Few-NERD, and WNUT. To align their label sets, we process them to be consistent with Few-NERD since we consider the label set of Few-NERD to be the most general.
Probing Experiments. The setup for semantic similarity evaluation follows the way we do for sentiment analysis, but the backbone model used the performance evaluation here is DeBERTa- base model, instead of Ta-base. The reason is that T5 requires inputs to be organized with prompts while standard prompt-based tuning methods for NER are still lacking. Hence, T5 does not perform well on NER task, so resort to fine-tuning the encoder-only model, DeBERTa, the alternative.
25

---

Table 17: The OOD performance of the DaBeRTa-large when trained on the Few-NERI dataset.
Train
Test
CONLI.
CrosNER
E-NER
Few OOW-NER
WNUT
Fow-NERD
69.10
66.63
48.01
79.89
45.45
Table 18: Statistics of extractive question answering candidate datasets. AQA: AdvQA; HQA: HotpotQA; NQ: NaturalQuestion;
SQA: SearchQA; SS: SQuAD Shifts; TQA: Trivia-QA.
Samples
Avg. Length
Dataset
Source
Train
Test
Train Test
AdvQA
Adversarial
30,000
3,000
117.36
114.83
HospotQA
Wikipodia
72.928
5,901
153.56
193.21
NaturalQuestion
Google search
104.071
128.36
152.03
158.8
NewsQA
CNN News
74,160
42,12
495.8
492.59
Google search
SearchQA
J'Archive
117,384
16,980
646.87
642.87
SQuAD
Wikipedia
87,599
10,570
119.76
123.95
Wikipedia Amazon &
SQuAD Shifts
Reddit & New York Times
29,753
139.1
Trivia-QA
Wikipedia
61,688
7,785
673.75
672.64
Table 19: SimCSE scores between each pair of datasets regarding the extractive question answering task. AQA: AdvQA; HQA: HotpoiQA;
NQ: NaturalQuestion; SQA: SearchQA; SS: SQuAD Shifts; TQA:
Trivia-QA.
Train
Test
AQA
HQA
NQ
NQA
SQA
SQuAD
SS
TQA
AdvQA
100
66.85
67.66
46.74
62.72
96.29
43.05
69.62
HotpotQA
66.85
100
81.25
65.53
83.99
72.99
39.89
87.81
NaturaQQuestions
67.66
81.25
100
54.67
79.26
73.5
37.06
80.67
NewsQA
46.74
65.53
54.67
100
66.94
51.98
35.94
64.14
SearchQA
83,99
79.26
66.94
94.17
SQuAD
96.29
72.99
73.5
51.98
69.22
100
42.83
75.72
SQuAD Shifts
43.05
39.89
37.06
35.94
48.19
42.83
100
48.12
Trivia-Qe
69.62
87.81
80.67
94.17
75.72
48.12
Table 20: The O0D performance of the T5-large when trained on the SQuAD dataset.
Train
Test
AQA
HQA
NQ
NQA
SQA
SQuAD
TQA
SQuAD
51.19
74.08
64.54
63.77
37.47
93.14
89.02
73.15
The dataset statistics are shown in Table 15. Few-NERD much larger than other datasets and contains the largest variety of samples, with 8 coarse-grained and 66 fine-grained types. Therefore, we select it as the dataset for the name entity recognition task.
Results.
For OOD datasets, since CrossNER shares the same source with the ID dataset Few-NERD, it should not adopted to our benchmark. The results Tables and 17 supports this claim from the perspectives of semantic similarity and performance drop, respectively. Then, the remaining three datasets will be used construct the distribution shifts, i.e., Few-NERD + (CoNLL, E-NER, WNUT).
D.4 Extractive Question Answering
Candidate Datasets. We consider AdvQA, HotpotQA, NaturalQuestions, NewsQA, SearchQA, SQuAD, SQuAD Shifts, and Trivia-QA as the candidates.
26

---

Probing Experiments. In the experiment of semantics similarity, we take the passages as inputs for the supervised SimCSE model. In the experiment of performance evaluation, we choose T5-base as the backbone model and measure Fl scores.
Results. Table 18 shows the dataset statistics. Except for SQuAD Shifts, which does not have a training split, all the other datasets are large in size. However, considering the diversity, assume that Wikipedia provides the most extensive knowledge base. Hence, we restrict the scope of the ID dataset those created from Wikipedia. Among the three datasets, HotpotQA, SQuAD, and Trivia-QA, SQuAD is the largest one. and therefore, we adopt SQuAD as the ID dataset.
Then, since SQuAD has been selected to be the ID dataset, HotpotQA and Trivia-QA QA are considered in the OOD dataset selection. And besides these two datasets, NaturalQuestions also have a high semantic similarity with SQuAD, shown in Table 19. Therefore, for the distinction of the distributions, NaturalQuestions will not be taken into account either. In contrast, despite a high semantic similarity with SQuAD, do not exclude AdvQA for this reason. This is because AdvQA is constructed by human adversaries based on SQuAD, aiming to reveal the weakness of models trained on SQuAD. Therefore, should not disregard AdvQA for this sake. Next, we evaluate the ID model on each candidate dataset examine the challenge of each distribution shift. From Tabel 20, we observe that the ID model performs poorly on AdvQA, NewsQA, and SearchQA while maintaining a high F1 score on SQuAD Shifts, indicating that the shift to SQuAD Shifts is the least challenging. Eventually, we establish the distribution shifts for extractive question answering as SQuAD + (AdvQA, NewsQA, SearchQA).
consider the high similarity between Adv QA and SQuAD reasonable and
E
Additional Descriptions and Results of the Analysis and LLMs Evaluation
E.1
Correlation between ID and OOD performance
E.1.1
Experimental Setting
To measure the correlation between ID and OOD performance, we train models with various ID performances by manipulating model scales, training steps, available training samples, and tunable parameters following [14]. The details are introduced below.
Model scale. We adopt the small, base, and large versions of the T5 and DeBERTa models.
Available training samples. For Sentiment Analysis, Toxic Detection, and Natural Language Inference, we randomly sample samples for each class. For Name Entity Recognition, we adopt the N-way K-shot sampling strategy proposed by [26], ensuring that entities of all N types occur K~~2K times in the sampled dataset. For Extractive Question Answering, we randomly sample K questions. In all experiments, run five repeated experiments for each K.
Number of tunable parameters. We employ parameter-efficieent tuning methods to regulate the tuned parameter for adapting to downstream tasks We prioritize adopting the strong approach Adapter [41]. However, in experiments, we find that Extractive Question Answering models can achieve high ID performance even with a very insignificant amount of parameters being tuned, thus presenting no performance change when the number of tunable parameters increases. Therefore, we turn to an alternative, Soft-prompt [54], for clearer observation.
Training steps. We evaluate model performance at epoch (0,0.1..... ...9) (1.0...., EPOCH), EPOCH), where EPOCH is the total number of training epochs.
E.1.2 Full Results
We present the complete results of ID-OOD correlation in Figure 4. Note that due to the variance of data points, we manually check and remove outlier points to make the fitting results clearer. Next, we will illustrate the results task by task.
Sentiment Analysis. All figures of SA comprise two straight lines. The smoother line, which mainly fits the results of T5-small models with poor ID performances, stretches from around ID accuracy = 60. Meanwhile, the steeper line accommodates T5-base and T5-large model results and begins around ID accuracy = 85. Despite the different slopes, both lines represent a linear positive correlation between ID performance and OOD generalization, thus we still categorize this kind of relation Type I.
27

---

Toxic Detection. This task encompasses two types of ID-OOD relations. AdvCivil stands as an outlier as aforementioned, whereas Implicit Hate and ToxiGen fall under the category Type I relation, characterized by straight line that intersects the diagonal line y z with a flatter slope. This trend indicates that although achieving initially higher performance on OOD examples than on ID examples, the model's performance gap between IDD and OOD decreases and eventually reverses as training proceeds
Natural Language Inference. All relations are classified as Type III, a non-monotonic correlation. Each graph has V-shaped curve, consisting of two straight lines with different slopes. Specifically, for ANLI, the first straight line has shallower slope than the second one, while ContractNLI and WANLI exhibit the opposite trend.
Named Entity Recognition. The ID-OOD relations in this task are all typical Type with all fitted lines lying below the diagonal line. The CoNLL dataset exhibits the highest correlation to perfectly linear relationship (y = I), while the WNUT dataset has the lowest correlation. This phenomenon indicates that improving OOD performance on CoNLL is the easiest while on WNUT is the hardest.
Extractive Question Answering. Type relation observed this task, with varying degrees of curve changes. Specifically, in the SearchQA dataset, the angle of the curve change is the most pronounced, while the NewsQA dataset is the smoothest.
E.2 LLMs Evaluation
Experiment Setting, The templates and instructions we used are shown in Tables 21 and 22. We take Dynasent as an example demonstrate the in-context learning settings for LLMs. The in-context (ID) setting includes examplars drawn from the ID training dataset of the sentiment analysis task, i.e. training dataset of Amazon. However, the in-context (OOD) setting samples the in-context examplars from the training split of the OOD dataset, i.e. training dataset of Dynasent. Note, however, that the latter paradigm may not be applicable to all datasets. All ID datasets are excluded, as well as OOD datasets without training split. Besides, those datasets whose text length is too long average would not be taken into account either, as the contexts could exceed the context window of LLMs.
28

---

Table 21: Templates for TO-3B and T3-series models.
Task
Template
Verbalizer
SA
((input)) All all the sentiment was ("mask").
negative positive / neutral
TD
[(nputt) All in all the toxicity was ("mask")..
benign toxic
Given the two sentences:
(1) (iipput]])..
NLI
Yes Maybe / No
(2) [imputt22 2)
Does the first sentence entail the second ? ("mask").
NER
ner text: [(imput) label: { ("mask")
Extract the answer to the question from the following context.
Question: [imput_]]] 1]
EQA
Context:
{ ((input_2] (oppuut2_]]_ ]
Answer: ("mask")
Table 22: Instructions for LLMs, including LLaMA-7B, Davinci3, and Turbo.
Task
Instruction
### Instruction ###
Solve the sentiment analysis task. Options for sentiment: negative, positive, neutral.
### Format ###
SA
Text: [[Text]) // Prediction: [Prrediction]
### Input ###
Text: [input]] // Prediction:
### Instruction ####
Solve the toxic detection task. Options for toxicity: benign, toxic.
### Format ###
TD
Text: {[Text]] // Prediction: ((Prediction)
### Input ###
Text: [input)) // Prediction:
### Instruction ### Solve the NLI task. Options for entailment relationship: entailment, neutral, contradic-
tion.
NLI
### Format ###
Premise: (Premise)) // Hypothesis: [(Hypothesis)) // Prediction: ((Prediction)
### Input ###
Premise: ([input]]) Hypothesis ((input_2)) // Prediction:
###
Instruction ###
Solve the NER task, identifying the Organization, Person, Location, Miscellaneous, Building, Art, Product, and Event entities from given text.
### Format ###
NER
Text: { Entity: Organization: None |l Person: Word1 || Location: Word6,
Word7 II Miscellaneous: None II Building: None Art: Word 3 || Product: None II
Event: None.
### Input ###
Text: [input]] // Entity:
### Instruction ####
Solve the extractive question answering task. Refering the passage below and extract answer for the question. The answer should be the shortest phrase as it can be. ### Format ###
EQA
Passage: [Passage]) ) // Question: [[Question]] ) // Answer: { Answer)) ). ### Input ###
Passage: [iinput]]] [|] // Question: [(input_2) 21 // Answer:
29

---

Dynasent
Amaron
60
Accuracy SenEval (O(O))
60
53
Accumn on Dynaen
SST
65
Accurayy 0n
43-
60-
50
50
55
63
05
70
75
80
05
40
50
55
60
70
25
02
05
03
50
55
00
05
70
75
05
Accuracy Amazon (ID)
Accuracy on Amazon IID)
Accuracy on IID)
(a) Sentiment Analysis
Civil Comments -n AdvCivil
Accuracy (GOD)
Civil Comments Implich Hate
Civil Comments - ToxiGen
Metcce 1000)
00
ToxCCan (000)
70-
50
Acouracy
50
20
20
93
50
00
80
10 20 30
50
70
90
20 20
30
40
50
70
90
Accurary on Elvil Comments (8D)
Accurary on Eieil Comments (BD)
Accuracy on Civll Comments 183)
(h) Toxic Detection
-NLII
ContraiNu (CO0)
MNJ ContractNLI
MNLI -WANL
Accuratyy en ARU (OOD)
32
Accuracy on WANLI 1000)
15
40
22
Accuracy MNLI (ID)
Accuracy MNLI (IC)
Accuracy on MNLI
(c) Natural Language Inference
Few-NERD CoNii
Fow NERD EERE
Tew-NERD -WNUT
Micr on CONIL 1003)
Nicro F1 on (OOD)
Micro F1 on WINUT (OOD)
00-
20
Micro
Micro Few NERD
NERD
(d) Name Entity Recognition
SQUAD
SQUAD NewsOA
SearchOAA
MMoo on Aivvo (000)
52
Micr 1oof1ooo NeweQA (000)
Micro F1 en SearcROA (000)
40
40-
48
33
k0
22-
12-
40
60
20
(c) Extractive Question Answering
Largersized
Green Available Samples
Blue
Training Steps
Red
Tunable Parameters
Figure 4: Complete results of the relation between ID and OOD performance.
30


Reviews:
[{"review_id": "2vNCFKwqjW", "reviewer": "Reviewer_5TMS", "summary": "", "questions": "", "limitations": "- Proposed evaluation protocol BOSS depends on SimCSE when measuring the similarity of the dataset and it maybe not enough. Well, I know that the paper also used the results of OOD performance gap from Table 3 and I just not sure about the ability of SimCSE in measuring domain shift. Are there any references on that?\n- The paper only focuses on classification tasks for measuring OOD robustness and do not provide some benchmarks for generation tasks that are also essential and common in NLP tasks.\n- To measure the performance among the size of LLMs, it might be used same types of LLM. In other words, it would be better use various sizes of LLaMa. (7B, 13B, 70B)", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "- The paper provides meaningful evaluation protocol for OOD robustness evaluation that is crucial for training NLP systems.\n- Meaningful analysis on three typical learning strategies on OOD robustness.\n- Various evaluations on five LLMs that have various sizes and showed experimental results on the number of ID dataset.\n- Significant benchmark and evaluation methods on language model's OOD robustness that is helpful for NLP community.\n- Showed weaknesses of previous OOD evaluations such as using IMDB and SST.\n- Interesting results on different ID-OOD tendency for various types of tasks.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "haPWuXhkgv", "reviewer": "Reviewer_XEXa", "summary": "", "questions": "", "limitations": "The authors adequately addressed the limitations and potential negative societal impact of their work.", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "**Novel Framework**\n\nThe paper proposes a novel framework for selecting IID and OOD datasets. This framework can be used to evaluate the generalization capabilities of LLMs, compare various small finetuned language model architecture on OOD data and benchmark the robustness methods that works well.\n\n**OOD Experiments**\n\nThe paper has extensive experiments that compares how IID data, tunabale parameters and training steps affects OOD performance and benchmarks recent NLP robustness methods on the proposed BOSS framework. \n\nThe paper has experiments comparing zeroshot, IID and OOD in-context learning in the case of LLMs and discuss tradeoffs when small fine-tuned language models perform better compared to LLMs.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "87TuyAO3mF", "reviewer": "Reviewer_wTiN", "summary": "", "questions": "", "limitations": "- Some NLP researchers/practitioners may feel that the tasks included in BOSS are not representative of all of NLP. Indeed, one could accuse the inclusion of tasks in BOSS as being driven by popularity.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "- The analysis of ID-OOD pair similarity is useful for understanding model performance on OOD data. However, the paper claims that in prior work ID-OOD pairs sometimes come from the same source (or distribution), but only one example of that is given.\n- The paper attempts to establish criteria for what makes a valuable ID-OOD pair. This is an important step in pushing the field forward towards being less arbitrary in the creation/choice of evaluation benchmarks. \n- Inclusion of LLMs in analysis. The paper goes beyond straightforward \"vanilla\" evaluation.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "t0c9RRRgMs", "reviewer": "Reviewer_6XNi", "summary": "", "questions": "", "limitations": "No limitations section in the paper. The major concern for me is the method used for collecting OOD data. Normally, we prepare OOD data by some designed “natural” definitions, e.g., we collect images from different hospitals. In this way, the collected OOD data is meaningful. However, this paper uses pairs similarity and model degradation to guide the OOD collection which is highly model-dependent. Different models used to conduct similarity analysis (in this paper, SimCSE is used) will produce different scores for the same pairs, in turn, results in different ID/OOD data splitting. The same problem could happen in model degradation analysis. Therefore, I am not sure of the usefulness of the prepared datasets in this work.", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "- Novelty: This work is trying to address and clarify an important problem in NLP, OOD Robustness. It helps researchers to better understand and handle OOD data in NLP tasks.\n- Comprehensive Evaluation: The study evaluates various SOTA methods and models on five diverse NLP tasks. Through intensive experiments and comparison, this work shows a better understanding of their performance and draws fascinating and equally important conclusions.\n- Practicality: This study provides valuable guidance for future research through probing experiments, summarizing patterns and detailed analysis. This study’s open source repository will empower researchers to effectively tackle and resolve OOD problems in the future.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: This paper aims to address a very important (and also a very big) problem in NLP. On the one hand, the authors have made significant efforts in dataset analysis, evaluations, and potential solutions. On the other hand, reviewers have raised concerns from many perspectives like the outdated datasets and the potential possibility of data leakage, relatively simple (e.g., sentiment analysis) task, and the limited representations of these few tasks to be claimed as NLP. The use of similarity by a specific model is another valid concern. Overall, there are values in the paper but the authors are encourage to make improvement in terms of paper presentation to avoid the concerns from the readers. It would be considered as a form of over-claim w.r.t. OOD robustness in NLP, if the findings are only supported by the listed datasets and tasks.
Decision: Accept (Poster)