SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lifan Yuan $^{1}$ , Yangyi Chen $^{2}$ , Ganqu Cui $^{1}$ , Hongcheng Gao $^{3}$ , Fangyuan Zou $^{4}$ , Xingyi Cheng $^{4}$ , Heng Ji $^{2}$ , Zhiyuan Liu $^{1*}$ , Maosong Sun $^{1*}$ $^{1}$  NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing  
 $^{2}$  University of Illinois Urbana-Champaign  
 $^{3}$  University of Chinese Academy of Sciences  ${}^{4}$  Tencent lievanyuan173@gmail.com
This paper reexamines the research on out-of-distribution (OOD) robustness in the field of NLP. We find that the distribution shift settings in previous studies commonly lack adequate challenges, hindering the accurate evaluation of OOD robustness. To address these issues, we propose a benchmark construction protocol that ensures clear differentiation and challenging distribution shifts. Then we introduce BOSS, a Benchmark suite for Out-of-distribution robustness evaluation covering 5 tasks and 20 datasets. Based on BOSS, we conduct a series of experiments on pretrained language models for analysis and evaluation of OOD robustness. First, for vanilla fine-tuning, we examine the relationship between in-distribution (ID) and OOD performance. We identify three typical types that unveil the inner learning mechanism, which could potentially facilitate the forecasting of OOD robustness, correlating with the advancements on ID datasets. Then, we evaluate 5 classic methods on BOSS and find that, despite exhibiting some effectiveness in specific cases, they do not offer significant improvement compared to vanilla fine-tuning. Further, we evaluate 5 LLMs with various adaptation paradigms and find that when sufficient ID data is available, fine-tuning domain-specific models outperform LLMs on ID examples significantly. However, in the case of OOD instances, prioritizing LLMs with in-context learning yields better results. We identify that both fine-tuned small models and LLMs face challenges in effectively addressing downstream tasks. The code is public at https://github.com/lifan-yuan/OOD_NLP.
Pretrained language models (PLMs) have excelled in downstream tasks and gained widespread adoption [24, 60]. However, existing evaluation often assumes independent and identically distributed (i.i.d) condition [94, 92], which is often violated in real-world scenarios, highlighting the crucial problem of out-of-distribution (OOD) robustness in NLP models. In this paper, we first revisit the evaluation of PLMs through an examination of evaluation benchmarks. Thereafter, we delve into the ID-OOD performance correlation of fine-tuned models by adopting various model scales, training steps, available training samples, and tunable parameters. Finally, we conduct extensive evaluations of current robustness-enhanced methods and large language models (LLMs).
Definition. There exist multiple definitions of OOD in literature [2, 115], and we define distribution shifts considered in this paper from two perspectives. Firstly, [2] classifies distribution shifts into "semantic shift" and "background shift". Our use of "out-of-distribution" aligns with the concept of
Figure 1: The comparison of previous work and our protocol on dataset selection of OOD benchmarks.
"background shift", which involves changes in the domain or style of the text while preserving the semantic content. Secondly, [115] formally defines three types of distribution shifts: covariate shift, label shift, and concept shift. In our work, we mainly focus on the combination of covariate shift and concept shift. This indicates that the model needs to generalize well to different input features (a.k.a, covariate shift) and adapt to variations in the underlying concepts within the data (a.k.a, concept shift).
**Benchmark.** Our study begins by surveying existing literature on OOD robustness in NLP (Table 8). We observe a lack of standardized OOD benchmark suites tailored for NLP since there is no single uniform set of datasets for evaluation, resulting in the adoption of heuristic and popularity-based dataset selection strategies in previous work [39, 109, 107]. This approach suffers from two main drawbacks: (1) The selected OOD datasets may come from similar distributions as the ID dataset, reducing the OOD evaluation to ID settings and thus hindering rigorous OOD robustness evaluation; (2) The challenge stemming from distribution shifts is limited, deviating from the expectation for difficulty posed by benchmark construction principles [8] and potentially leading to an overestimation of the OOD robustness of language models. As a result, current OOD robustness benchmarks may inadequately assess NLP models.
To address the aforementioned concerns, we establish a protocol as shown in Figure 1, consisting of three fundamental principles, for selecting both ID and OOD datasets: (1) ID datasets should be large and diverse for comprehensive knowledge; (2) Selection for OOD datasets should prioritize distinct distributions and dissimilarity, regarding text sources and semantics; (3) Challenging distribution shifts should be prioritized based on performance degradation, to ensure that the benchmark stands the test of time [8]. Based on the protocol, we compile BOSS, a more holistic and challenging NLP Benchmark suite for OOD robustness evaluation. Unlike existing benchmarks which only consider single task types such as classification [39, 107] or reading comprehension [109], BOSS covers a wider range of task formats, including sentiment analysis, toxic detection, and natural language inference for classification, name entity recognition for structured prediction, and extractive question answering for reading comprehension. We establish one ID and three corresponding OOD datasets for each task.
Analysis. We recognize the lack of analysis of models' learning behaviors regarding ID performance and OOD generalization in the field of NLP, hindering the development and understanding of OOD robustness. Thus, we investigate the correlation between the performance on ID and OOD datasets using the BOSS benchmark. To regulate the ID performance, we manipulate four related factors, i.e. model scale, training steps, available training samples, and tunable parameters. Three typical categories of ID-OOD correlation are observed, namely, monotonic linear positive correlation, monotonic piecewise linear positive correlation, and non-monotonic V-shaped correlation (see Figure 2). We discuss the potential reasons for the causes of these identified correlations in section 3.
**Evaluations.** After examining the learning mechanism of PLMs in vanilla fine-tuning, we scrutinize their performance with existing robustness-enhanced methods and then proceed to prevalent LLMs. Due to the absence of a standard benchmark, previous evaluations of existing methods can be
imprecise and thus misleading the estimations of progress in this field. Moreover, given the increasing focus on LLMs [9, 89] in NLP research, it is essential to evaluate their effectiveness in handling OOD challenges and explore the efficacy of different adaptation paradigms.
For robustness-enhanced methods, we evaluate five representative methods [99] on BOSS. Our main observation is that vanilla fine-tuning (a.k.a, empirical risk minimization) remains a strong baseline, while certain methods may slightly improve OOD performance in some cases. We further evaluate various LLMs and adaptation paradigms. We consider three recent prevailing LLMs, namely LLaMA [89], OpenAI text-davinci-003 [9], and OpenAI gpt-3.5-turbo. We include two relatively smaller models T0-3B [81] and T5-3B [77] for comparison. We apply zero-shot inference, in-context learning, few-shot fine-tuning, and full-data fine-tuning to one or multiple models. Through our experiments, we find that when provided with enough training data, fine-tuning domain-specific models remain the preferable choices for handling ID examples, while leveraging LLMs with in-context learning is superior for tackling OOD instances. In addition, we observe that the impact of in-context learning on generalization ability varies across models. We provide more detailed discussions in section 4.2.
2 BOSS Benchmark
NLP models should exhibit robustness across diverse distributions to ensure reliable applications. To achieve this, a standardized and recognized evaluation benchmark for OOD robustness is imperative. However, previous efforts in constructing benchmarks have predominantly relied on random selections and dataset popularity, lacking a systematic design [39, 109, 107]. Two deficiencies are thus identified: (1) Dataset similarity, as exemplified by the SST and IMDb datasets for sentiment analysis [83, 64], which share movie reviews and exhibit high semantic similarity (see Table 2). This blurs the line between ID and OOD evaluation, hindering rigorous assessment of OOD robustness; (2) Limited distribution shift challenges, exemplified by the high accuracy of a model trained on Amazon [65] when tested on IMDb (see Table 3). However, the significant performance drop on our considered Dynasant [75] suggests that OOD robustness still remains a critical problem in the sentiment analysis task. Thus, there is a need for universally applicable challenges across all dataset selections [8].
2.2 Protocol to Construct OOD benchmark.
We aim to establish a standard benchmark for rigorous evaluation of OOD robustness in NLP. To address the above issues, we first survey and gather existing candidate datasets from Paperswithcode $^2$ , Kaggle $^3$ , and ACL Anthology $^4$  websites. We consider the release date and public availability of datasets. Then we carefully examine three criteria to determine the ID and corresponding OOD datasets. The first criterion focuses on the ID dataset selection, and the other two criteria are proposed for OOD datasets, targeting the two issues in previous work, respectively.
The ID dataset should provide sufficient knowledge for models to handle the task. ID dataset should encompass comprehensive task-level knowledge [44], enabling models to grasp the underlying rationale necessary for task completion. Alternatively, if the model exclusively learns biased features, it may struggle to adapt to other features during distribution shifts. To this end, it is necessary for the ID datasets to possess the following characteristics: (1) Sufficiently large size; (2) Diversity, which is achieved through collection from multiple sources or the inclusion of several subtypes (i.e., styles, topics, levels of formality, et al). Our intuition is in line with [87], which demonstrates that training on large and diverse datasets improves the robustness of vision models.
Datasets within a given task should originate from diverse distributions for a holistic evaluation. We guarantee this through qualitative analysis of data source diversity and quantitative measurement of semantic similarity using SimCSE [31]. To avoid overlap, we select at most one dataset per text source. Additionally, we ensure OOD datasets in the benchmark exhibit relatively low semantic similarity, and thus enhancing distinctiveness.
2https://paperswithcode.com/datasets
<sup>3</sup>https://www.kaggle.com
<sup>4</sup>https://aclanthology.org/
OD shifts should be challenging to provide an accurate assessment of progress in OOD robustness [8]. To quantify the challenge, we train a model on the ID dataset and test it on all candidate datasets. Specifically, we tune a T5-large [77] with manual templates on four tasks, except for NER, on which we adopt DeBERTa-large [38] with conventional fine-tuning due to the lack of a standard prompt-based tuning schema for this task. For this reason, all experiments in this paper follow this choice of model selection. For each text source, we first discard candidates similar to the ID dataset in semantics. Then, to construct challenging distribution shifts, we prioritize the dataset provoking the most severe performance drop of the ID model and adopt it as the OOD dataset in our benchmark.
2.3 Dataset Selection
We take sentiment analysis as a case to demonstrate how we select ID and OOD datasets for each task according to our protocol. The selection process for other tasks can be found in Appendix D.
Candidate Datasets. We first collect all sentiment analysis datasets on Paperswithcode, Kaggle, and ACL Anthology as aforementioned. We filter out datasets released before the 2010s, as they are largely resolved with the advent of pre-trained language models [25]. As a result, seven datasets remain as candidates, i.e., Amazon [65], DSC [48], Dynasent [75], IMDb [64], SemEval [70], SST [83], and Yelp [116]. Considering the inconsistency in the number of categories across the datasets, we align them by converting them into a three-class classification setting. See Appendix C.2 for a detailed explanation of the dataset processing procedure.
Probing Experiments. According to our protocol, dataset size and text sources are assessed for ID dataset selection. Subsequently, semantic similarity and ID model performance degradation guide OOD dataset selection. To this end, two probing experiments are conducted: (1) Comparing semantic similarity using SimCSE for candidate dataset pairs, and (2) Evaluating the
Table 1: Statistics of sentiment analysis candidate datasets.
Dataset	Source	# Classes	# Samples	Avg. Length
Train	Test	Train	Test
Amazon	Product	3	30,000	38,905	71.69	54.84
DSC	Product	2	92,244	11,531	132.29	130.14
Dynasent	Adversarial	3	93,553	4,320	13.19	13.83
IMDb	Movie	2	25,000	25,000	233.79	228.53
SemEval	Twitter	3	6,000	20,622	19.44	19.62
SST	Movie	3	4,004	1,067	18.78	18.75
Yelp	Product	3	30,000	30,000	132.49	131.62
performance of the selected ID model. In the first experiment, for better semantic representation, we resort to the best SimCSE model provided by [31], a supervised RoBERTa-large [60]. We load the model checkpoint from Huggingface<sup>5</sup>. For each dataset, we first encode each sample into a continuous embedding and then average the embeddings across the dataset to obtain a centroid representation of the dataset. Finally, we calculate the cosine similarity between a pair of centroids as the semantic similarity between two datasets. In the second experiment, we train a T5-large model on the selected ID dataset and evaluate its performance on all the candidate datasets.
Dataset Selection. The dataset information and semantic similarities are provided in Table 1 and Table 2, respectively. The text sources of the datasets vary from product reviews, movie reviews, Twitter, and adversarial texts. We observe that datasets originating from the same source tend to exhibit higher SimCSE scores, indicating higher seman
Table 2: SimCSE scores between each pair of datasets regarding the sentiment analysis task.
Train	Test	Amazon	DSC	Dynaset	IMDB	SemEval	SST	Yelp
Amazon	100	86.02	57.30	36.67	24.74	33.70	49.22
DSC	86.02	100	59.15	54.55	31.70	44.40	55.45
Dynaset	57.30	59.15	100	32.69	28.17	19.68	88.99
IMDb	36.67	54.55	32.69	100	46.95	84.62	39.88
SemEval	24.74	31.70	28.17	46.95	100	40.45	24.03
SST	33.70	44.40	19.68	84.62	40.45	100	19.43
Yelp	49.22	55.45	88.99	39.88	24.03	19.43	100
tic similarity. It is worth noting that for IMDb and SST, the widely used ID-OOD dataset pair in sentiment analysis [39, 107], the SimCSE score demonstrates one of the highest levels among dataset pairs. This reinforces the first deficiency of previous benchmarks, where dataset pairs have similar semantics and unclear distribution shifts. Hence, in contrast to existing practices, our benchmark construction considers only one dataset from each source.
For the ID dataset selection, we first exclude DSC and IMDb since they are binary classification datasets, on which the trained model cannot tackle the unseen class neutral. For dataset size,
<sup>5</sup>https://huggingface.co/princeton-nlp/sup-simcse-roberta-large
SemEval and SST are disregarded due to their limited number of samples per class (less than 10k). Among the remaining datasets, Amazon is chosen as the ID dataset for sentiment analysis as it encompasses reviews from 29 distinct product categories, offering greater diversity than Yelp.
For OOD datasets selection, we train a T5-large model on the ID dataset (i.e., Amazon) and evaluate it on all candidate datasets, as illustrated in Table 3. We include Dynasent and SemEval in
Table 3: The OOD performance of the T5-large when trained on the Amazon dataset.
Train | Test	Amazon	DSC	Dynaset	IMDb	SemEval	SST	Yelp
Amazon	90.94	95.63	47.38	92.69	49.90	75.16	89.25
the benchmark suite due to the following reasons: (1) They are the sole adversarial and Twitter datasets available, (2) They demonstrate low semantic similarity, and (3) They exhibit a notable performance degradation, making them crucial for evaluation. For movie reviews, SST is prioritized due to lower SimCSE scores compared to IMDb and larger performance drop of the ID model. Eventually, this yields three distinct and challenging distribution shifts in the sentiment analysis task: Amazon  $\rightarrow$  (DynaSent, SemEval, SST).
2.4 BOSS
Based on the aforementioned protocol, we introduce BOSS, an NLP benchmark suite for OOD robustness evaluation. BOSS comprises five essential NLP tasks: sentiment analysis (SA), toxic detection (TD), natural language inference (NLI), name entity recognition (NER), and ex
Table 4: The datasets included in the BOSS benchmark. Corresponding abbreviations are shown in brackets.
Task	ID Dataset	OOD Datasets
SA	Amazon (AZ)	Dynaset (DS)	SemEval (SE)	SST (SST)
TD	Civil Comments (CC)	AdvCivil (AC)	Implicit Hate (IH)	ToxiGen (TG)
NLI	MNLI (MN)	ANLI (AN)	ContractNLI (CN)	WANLI (WN)
NER	FewNerd (FN)	CoNLL (CoNLL)	E-NER (ENER)	WNUT (WNUT)
EQA	SQuAD (SQuAD)	AdvQA (AQA)	NewsQA (NQA)	SearchQA (QA)
tractive question answering (EQA). These tasks represent diverse practical applications and provide comprehensive coverage for evaluating models' capabilities, from aspects of classification, structured prediction, and extraction. Each task in the benchmark includes one ID dataset and three associated OOD datasets (see Table 4).
Sentiment Analysis. Amazon [65] contains reviews of 29 different categories of products from the Amazon website. DynaSent [75] first identifies naturally challenging sentences from several existing datasets, and then creates adversarial sentences with a human-and-model-in-the-loop annotation approach. SemEval [70] is a three-class sentiment analysis dataset focusing on tweets. SST [83] consists of sentence-level movie reviews from the Rotten Tomatoes website.
Toxic Detection. Civil Comments [6] contains public comments on the Civil Comments platform, with users from diverse groups and various subtypes of toxic texts. AdvCivil, a new toxic dataset introduced in this paper, is generated from Civil Comments by textual adversarial attacks in an automated model-in-the-loop adversarial pipeline. Please refer to Appendix C.1 for details. Implicit Hate [29] contains toxic tweets in both explicit and implicit forms. The latter can circumvent keyword-based toxic detection systems. ToxiGen [36] is synthesized by GPT-3 [9], covering several types of subtly and implicitly toxic texts on 13 minority groups.
Natural Language Inference. MNLI [102] provides ten different categories of written and verbal sentence pairs, with diverse styles, topics, and levels of formality. ANLI [73] is an adversarial dataset collected in a human-and-model-in-the-loop approach, where each premise mainly comes from Wikipedia and the hypothesis is generated by human adversaries. ContractNLI [49] considers each contract as a premise and holds a fixed set of hypotheses throughout the dataset. WANLI [59] is synthesized by GPT-3 [9], each example containing challenging patterns identified in MNLI.
Name Entity Recognition. Few-NERD [26], the arguably largest dataset for NER, labels about 188k Wikipedia sentences into eight coarse-grained entity types. CoNLL [88] takes stories from Reuters news, containing four basic entity types. E-NER [3] is based on legal text. We use the four-category version in this paper, which treats all legal entities as miscellaneous ones. WNUT [23] collects training data from Twitter and mines test data from Reddit, StackExchange, Twitter, and YouTube, containing six coarse-grained entity types in Few-NERD.
(a) Type I
(b) Type II
(c) Type III
Figure 2: Three representative correlations between ID-OOD performance: (a) Type I (monotonic linear positive correlation) indicates consistent linear improvement of OOD performance with increasing ID performance. (b) Type II (monotonic piecewise linear positive correlation) exhibits accelerated OOD performance growth after a turning point. (c) Type III (non-monotonic V-shaped correlation) shows an initial negative correlation, followed by a positive correlation after a turning point. The  $r^2$  value in Figure (a) is 0.9677, and the values of the left and right fits in Figure (b) are 0.9553 and 0.9396 whereas in Figure (c) are 0.7690 and 0.8124 respectively.
Extractive Question Answering. SQuAD [78] constructs question-answer pairs based on Wikipedia passages. AdversarialQA [4] composes adversarial questions for contexts in SQuAD in a human-and-model-in-the-loop procedure, similar to ANLI. NewsQA [90] writes questions for CNN news articles, each of which requires reasoning to answer, rather than relying solely on word overlap and textual entailment. SearchQA [28] adopts a reverse construction pipeline, employing the Google search engine to retrieve relevant contexts for each question-answering pair from the J!Archive website.
3 Analysis of OOD Robustness
Despite OOD robustness in NLP has been extensively studied [43], a potential concern pertains to the usage of nonstandard benchmarks, as discussed in Section 2, resulting in inaccurate conclusions. To address this issue, we conduct a series of empirical analyses and evaluations to gain in-depth insights into OOD robustness in NLP. Previous research primarily concentrates on method comparisons without delving into models' learning behaviors. Therefore, we first analyze the models' learning mechanism by assessing the correlation between ID and OOD performance.
Setting. We assess the correlation between ID and OOD performance across various conditions. We manipulate the ID performance of models by varying their scale, training steps, available training samples, and tunable parameters. Further implementation details can be found in Appendix E.1.1.
Results. We observe that the correlation between ID and OOD performance on datasets of the five tasks is inconsistent, but can be broadly categorized into three types (see Figure 2): monotonic linear positive correlation (Type I), monotonic piecewise linear positive correlation (Type II), and non-monotonic V-shaped correlation (Type III). We also identify an exceptional case in Figure 3, which does not fall into any of the three categories. The full results are shown in Figure 4.
Type I. This is the most prevalent type of correlation observed across all ID-OOD pairs for sentiment analysis, name entity recognition, and the majority for toxic detection. As shown in Figure 2a, in this category, OOD performance is positively and linearly correlated with ID performance, indicating that the task knowledge learned on source distribution can be effectively generalized to other distributions. This observation is consistent with results in the computer vision domain [68], which shows that OOD performance is linearly correlated with ID performance across various model architectures, hyperparameters, training dataset size, and training duration. However, the slope of the line fitted by the least square method is less steep than the  $y = x$  diagram, and it eventually lies below the diagonal, implying that the performance degradation of models under distribution shift will be escalated with the increase of the ID performance.
Type II. This category is observed on ID-OOD pairs for extractive question answering. As presented in Figure 2b, the results can be fitted into a polyline, indicating a piecewise linear correlation. The correlation between OOD performance and ID performance is positive and linear, with a notable differ