SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jiashuo Liu $^{1,*}$ , Tianyu Wang $^{2,*}$ , Peng Cui $^{1}$ , Hongseok Namkoong $^{3}$
$^{1}$ Department of Computer Science and Technology, Tsinghua University  $^{2}$ Department of Industrial Engineering and Operations Research, Columbia University
<sup>3</sup>Decision, Risk, and Operations Division, Columbia Business School
liujiashuo77@gmail.com, tw2837@columbia.edu
cuip@tsinghua.edu.cn, namkoong@gsb.columbia.edu
Different distribution shifts require different algorithmic and operational interventions. Methodological research must be grounded by the specific shifts they address. Although nascent benchmarks provide a promising empirical foundation, they implicitly focus on covariate shifts, and the validity of empirical findings depends on the type of shift, e.g., previous observations on algorithmic performance can fail to be valid when the  $Y|X$  distribution changes. We conduct a thorough investigation of natural shifts in 5 tabular datasets over 86,000 model configurations, and find that  $Y|X$ -shifts are most prevalent. To encourage researchers to develop a refined language for distribution shifts, we build WHYSHIFT, an empirical testbed of curated real-world shifts where we characterize the type of shift we benchmark performance over. Since  $Y|X$ -shifts are prevalent in tabular settings, we identify covariate regions that suffer the biggest  $Y|X$ -shifts and discuss implications for algorithmic and data-based interventions. Our testbed highlights the importance of future research that builds an understanding of why distributions differ.[2]
The performance of predictive models has been observed to degrade under distribution shifts in a wide range of applications, such as healthcare [9, 95, 76, 93], economics [36, 25], education [6], vision [74, 63, 86, 98], and language [62, 7]. Distribution shifts vary in type, typically defined as either a change in the marginal distribution of the covariates ( $X$ -shifts) or the conditional relationship between the outcome and covariate ( $Y|X$ -shifts). Real-world scenarios comprise of both types of shifts. In computer vision [62, 47, 81, 38, 101],  $Y|X$ -shifts are less likely to occur as  $Y$  is constructed from human knowledge given an input  $X$ , unless the labeling noise is severe. For tabular datasets,  $Y|X$ -shifts can be more common because of missing variables and hidden confounders. For example, the prevalence of a disease may be affected by unrecorded covariates whose distribution changes across domains, such as lifestyle factors and socioeconomic status [39, 103, 93].
Different types of distribution shifts require different solutions. When facing  $X$ -shifts, the implicit goal of many researchers is to develop a single robust model that can be generalized effectively across multiple domains. Various algorithms have been developed to align the marginal distributions  $(P_X)$ , including domain adaptation and importance sampling methods. However, under  $Y|X$ -shifts, there may be a fundamental trade-off between learning algorithms: to perform well on a target distribution, a model may have to necessarily perform worse on others. Algorithmically, typical methods for
2More information on the data, codes and python packages about WHYSHIFT are available at https: //github.com/namkoong-lab/whyshift.
Figure 1. Relative regret (1.1) in typical benchmarks [26, 31] (left 5 bars) and seven settings designed in our benchmark (right 7 bars). We use XGBoost  $\mathcal{F}$  here for illustration.
addressing  $Y|X$ -shifts include distributionally robust optimization (DRO) [13, 85, 28, 80, 27] and causal learning methods [72, 8, 83, 46]. Operationally, the modeler can identify and collect an unobserved confounder  $C$  such that  $Y|X, C$  remains invariant across domains, or resort to overhauling the entire model development pipeline to collect more samples from the target.
However, existing distribution shift benchmarks only focus on  $X$ -shifts [74, 47, 107, 81, 101]. To illustrate this concretely, consider popular tabular datasets used to benchmark model performance over demographic subgroups: Adult, BRFSS, COMPAS, ACS Public Coverage, and ACS Income [3, 102, 31]. We take the largest demographic group as training  $P$  and the smallest as target  $Q$  to simulate subgroup shifts, e.g., in Adult,  $P =$  white men and  $Q =$  non-white women. We measure the optimality gap of the model  $f_{P}$  trained on  $P$  as measured on the target  $Q$  using the relative regret
$$
\frac {\mathbb {E} _ {Q} [ \ell (Y , f _ {P} (X)) ]}{\min  _ {f \in \mathcal {F}} \mathbb {E} _ {Q} [ \ell (Y , f (X)) ]} - 1, \quad \text {w h e r e} \quad f _ {P} \in \underset {f \in \mathcal {F}} {\operatorname {a r g m i n}} \mathbb {E} _ {P} [ \ell (Y, f (X)) ] \tag {1.1}
$$
and  $\ell (\cdot ,\cdot)$  is the 0-1 loss. For these widely-used benchmarks, the relative regret is small (left 5 bars in Figure 1), suggesting the  $Y|X$  distribution is largely transferable across those demographic groups.
To study diverse distribution shift patterns, we consider 5 real-world tabular datasets constructed from the US Census (as proposed by Ding et al. [25]) and traffic measurements [64, 65, 2, 1]. We focus on spatiotemporal shifts to model most common natural shifts. Our full benchmark covers 22 settings (see Table 3 in Appendix D), where each setting includes one source (e.g., California) and a number of possible targets (e.g., other states). For illustration purposes, we focus on 7 settings covering 169 possible source-target pairs (see Table 2) and carefully select one target per setting to represent a wide range of  $Y|X$ -shifts (right bars in Figure 1).
We find  $Y|X$ -shifts constitute a substantial proportion of real-world distribution shifts, yet previous (unqualified) empirical findings in the literature only hold over mild  $X$ -shifts and fail to hold over  $Y|X$ -shifts (Section 2). Out of 169 source-target pairs with significant performance degradation ( $>8$  percentage points of accuracy drop),  $80\%$  of them are primarily attributed to  $Y|X$ -shifts.  $Y|X$ -shifts introduce considerable performance variations on the target distribution, leading to different relationships between in- and out-of-distribution performances across settings and datasets. This is in stark contrast to the recently observed accuracy-on-the-line phenomenon [63], where the in- and out-of-distribution performance have been posited to exhibit a strong linear relationship. In Figure 2, we showcase how the accuracy-on-the-line trend fails to hold when  $Y|X$ -shifts are strong. Our results imply that the standard practice of blindly evaluating performance over various shifts is only justified over  $X$ -shifts, where we expect there to be a single model that is robust across domains.
For severe  $Y|X$ -shifts, the training data may not even be informative for modeling the  $Y|X$  relationship in the target. To inform algorithmic and data-based interventions, we must understand why the distribution changed. In Section 3, we illustrate the need for more methodological research that builds a deep understanding of distributional differences. As a concrete example, we show that a simple approach for identifying covariate regions with strong  $Y|X$ -shifts can suggest data-based interventions. Our case study shows it can be useful to collect target data over a particular covariate region, or features  $C$  such that the  $Y|X, C$  distribution is more stable across source and target.
In Section 4, we provide the details of our benchmark, WHYSHIFT. We use five datasets to construct 7 spatiotemporal distribution shifts, and evaluate 22 methods over  $86,000+$  model configurations. We compare a broad range of algorithms including tree ensembles, DRO, imbalance, and fairness methods and summarize our key findings below.
- Rankings of model performance change over different shift patterns. As the validity of empirical findings implicitly depends on the type of shift, any methodological development must be grounded by the specific shifts it addresses.
- Tree ensemble methods are competitive, but still suffer from significant performance degradation.
(a) ACS Income (CA  $\rightarrow$  PR)
(b) ACS Mobility  $(\mathrm{MS}\rightarrow \mathrm{HI})$
(c) Taxi (NYC  $\rightarrow$  BOG)
(d) ACS Pub.Cov (NE  $\rightarrow$  LA)
(e) US Accident (CA  $\rightarrow$  OR)
(f) ACS Pub.Cov (2010→2017)
g) ACS Income (Young  $\rightarrow$  Old)
Figure 2. Target vs. source accuracies for 22 algorithms and datasets in our benchmark. A linear fit (green line) and its corresponding  $R^2$  value is reported on the top left of each figure. Each blue point represents one hyperparameter configuration. (a)-(f): six examples of ACS Income, ACS Mobility, Taxi, ACS Pub. Cov, US Accident datasets. (g): simulated covariate shifts on on sub-sampled ACS Income dataset.
- DRO methods are sensitive to configurations and exhibit significant performance variations.
- Imbalance and fairness methods show similar performance with the base learner (XGBoost).
- A small validation data from the target distribution goes a long way, and more generally, non-algorithmic interventions warrant greater consideration.
2 Distribution Shifts in Tabular Settings
To illustrate how complex distribution shift patterns arise in tabular data, we compare 22 algorithms including tree ensemble methods, robust learning, imbalance, and fairness methods. On 5 real-world tabular datasets (ACS Income, ACS Public Coverage (ACS Pub.Cov), ACS Mobility [25], US Accident [64, 65] and Taxi [2, 1]), we consider the natural spatial shifts between states/cities, e.g., California to Puerto Rico. For the ACS Pub. Cov dataset, we also consider temporal shifts, e.g., from 2010 to 2017. Since all natural distribution shifts we consider are largely induced by  $Y|X$ -shifts, we construct a synthetic subgroup shift from younger people to older people in order to simulate  $X$ -shifts. Deferring a detailed summary to Section 4.1, we focus on introducing representative phenomena in this section.
In Figure 2, we present the source (in-distribution) and target (out-of-distribution) performances of 22 algorithms, each with 200 hyperparameter configurations. To understand shift patterns, we utilize the recently proposed DDistribution Shift DEcomposition (DISDE) framework [16] which decomposes the performance degradation into components attributed to  $Y|X$ - and  $X$ -shifts. Using the best XGBoost configuration as the baseline model for each source-target pair, we present the total performance degradation and the proportion attributed to  $Y|X$ -shifts.
Distribution shifts are predominantly  $Y|X$ -shifts in our empirical study We find performance degradation under natural shifts is overwhelmingly attributed to  $Y|X$ -shifts, as illustrated in the curated list in Figure 2. More generally, out of the 169 source-target pairs whose performance degradation is larger than 8 percentage points,  $87.2\%$  of them have over  $50\%$  of the performance degradation attributed to  $Y|X$ -shifts (70.2% of them have over  $60\%$  of the gap attributed to  $Y|X$ -shifts). We conjecture that  $Y|X$ -shifts are prevalent in tabular data due to missing features. For example, in the context of income prediction, individual outcomes may change due to unobserved economic and political factors whose distribution changes over geographical locations [25]. In contrast, in vision and language tasks, the input (e.g., pixels and words) often encapsulates most of the necessary information for predicting the outcome, making strong  $Y|X$ -shifts less likely unless the labeling noise is severe. Consequently, compared to vision and language data in domain generalization
Figure 3: Performances of typical algorithms of 7 settings in our benchmark.
tasks, tabular data exhibits more pronounced real  $Y|X$ -shifts. Our findings highlight the importance of understanding the cause of the distribution shift.
Accuracy-on-the-line fails to hold over  $Y|X$ -shifts We find significant variation in the relationship between source and target performance throughout all natural distribution shifts presented in Figure 2 (a)-(g). The correlation between source and target performance is relatively weak, and we tend to see poor linear fits (low  $R^2$ ) when the bulk of the performance degradation is attributed to  $Y|X$ -shifts. That is, we see in Figures 2 (a)-(f) that the relationship between the two performances exhibits significant fluctuations across different source-target pairs. In Figure 3, we observe that performance rankings of algorithms substantially vary across different  $Y|X$ -shifts. Our finding highlights the inherent complexity associated with real distribution shifts in tabular datasets, which stands in sharp contrast to the "accuracy-on-the-line" phenomena [63]. The varied shift patterns in tabular data highlight how empirical observations must be qualified over the range of shifts they remain valid over. This is particularly important for  $Y|X$ -shifts which introduce larger variations in the relationship between source and target performance.
Source and target performances are correlated when  $X$ -shifts dominate Across all natural shifts we study, we find  $X$ -shifts are only prominent in temporal shifts (ACS Time dataset; Figure 2f). To better investigate the role of  $X$ -shifts, we subsample the data to artificially induce strong covariate shifts over an individual's age. Specifically, we focus on individuals from California and form two groups according to whether their age is  $\geq 25$ . The source data oversamples low age groups where  $80\%$  is drawn from the age  $\leq 25$  group; proportions are reversed in the target data.
On this synthetic shift we construct, the DISDE [16] method attributes the bulk of the performance degradation to  $X$ -shifts in Figure 2g. Our finding confirms the intuition that unobserved economic factors remain relatively consistent for individuals from the same state (CA). In this synthetic example with  $X$ -shifts, we observe a relatively strong correlation between source and target performance. Moreover, the large performance degradation on these datasets suggests that existing robust learning methods are still severely affected by covariate shifts, indicating the need for future research that addresses covariate shifts in tabular data.
3 Case Study: Understanding Distribution Shifts Facilitates Interventions
Typical algorithmic approaches to handling practical distribution shifts aim to optimize performance over a postulated set of distribution shifts. Causal learning assumes the underlying causal structure can be learned to withstand distribution shifts [72, 83, 82, 77], while DRO methods explicitly optimize worst-case performance over a set of distributions [13, 53, 28, 27]. Despite progress in algorithm design, there are few efforts that examine the patterns of real-world distribution shifts. It remains unclear whether the data assumptions made by algorithms hold in practice, and this mismatch often leads to poor empirical performance [40, 95, 76, 19, 46].
Complementing the active literature on algorithmic development, we present an empirical study that underscores the practical significance of tools that provide a qualitative understanding of the shift at hand. In light of the prevalence of  $Y|X$ -shifts in tabular data, we introduce a simple yet effective approach for identifying covariate regions that suffer strong  $Y|X$ -shifts. We demonstrate our approach on the income prediction task (ACS Income), and show that it can guide operational interventions for addressing distribution shifts. Our case study is not meant to be a rigorous scientific analysis, but rather a (heuristic) vignette illustrating the need for future research on methodologies that can generate qualitative insights on distributional differences.
3.1 Identifying Regions with Strong  $Y|X$ -shifts
Here we propose a simple yet effective method for identifying covariate regions with strong  $Y|X$ -shifts. Despite its simplicity, we demonstrate in the following subsections that our method can inspire operational and modeling interventions. Letting  $(X,Y)$  be random variable supported on the space  $\mathcal{X} \times \mathcal{Y}$ , consider a model  $f: \mathcal{X} \to \mathcal{Y}$  that predicts outcome  $Y \in \mathcal{Y}$  from covariates  $X \in \mathcal{X}$  with the associated loss function  $\ell(f(X), Y)$ . Given samples  $(X,Y)$  drawn from the source and target distributions  $P$  and  $Q$ , our goal is to identify a region  $\mathcal{R} \subseteq \mathcal{X}$  where  $P_{Y|X}$  differs a lot from  $Q_{Y|X}$ .
Since  $P_{Y|X}$  and  $Q_{Y|X}$  are undefined outside of the support of  $P_{X}$  and  $Q_{X}$  respectively, the comparison can only be made on a subset of the common support. To aid comparisons on the common support, Cai et al. [16] introduced a shared distribution approach. The shared distribution has high density when both  $p_{X}$  and  $q_{X}$  are high, and low density whenever either is small. Following Cai et al. [16], we choose a specific shared distribution  $S_{X}$  over  $X$  from the likelihood ratio:
$$
s _ {X} (x) \propto p _ {X} (x) q _ {X} (x) / \left(p _ {X} (x) + q _ {X} (x)\right). \tag {3.1}
$$
We provide more discussion on the choices of  $S_{X}$  and other technical details justifying the correctness of  $s_{X}$  in Appendix C.3. Since we do not have access to samples from the shared distribution  $S_{X}$ , we reweight samples from  $P_{X}$  and  $Q_{X}$  using the likelihood ratios  $s_{X}(x) / p_{X}(x) \propto q_{X}(x) / (p_{X}(x) + q_{X}(x))$  and  $s_{X}(x) / q_{X}(x) \propto p_{X}(x) / (p_{X}(x) + q_{X}(x))$ . The ratio can be modeled as the probability that an input  $x$  is from  $P_{X}$  vs  $Q_{X}$ . Denote  $\alpha^{*}$  as the proportion of the pooled data that comes from  $Q_{X}$  and  $\pi^{*}(x) := \mathbb{P}(\tilde{X} \text{ from } Q_{X} | \tilde{X} = x)$ , we can express the likelihood ratios as:
$$
\frac {s _ {X}}{p _ {X}} (x) \propto \frac {\pi^ {*} (x)}{(1 - \alpha^ {*}) \pi^ {*} (x) + \alpha^ {*} (1 - \pi^ {*} (x))} =: w _ {P} (\pi^ {*} (x), \alpha^ {*}), \tag {3.2}
$$
$$
\frac {s _ {X}}{q _ {X}} (x) \propto \frac {1 - \pi^ {*} (x)}{(1 - \alpha^ {*}) \pi^ {*} (x) + \alpha^ {*} (1 - \pi^ {*} (x))} =: w _ {Q} (\pi^ {*} (x), \alpha^ {*}). \tag {3.3}
$$
With the likelihood ratios, we estimate the best prediction model under  $P$  and  $Q$  over the shared distribution  $S_{X}$  (using XGBoost as the model class  $\mathcal{F}$ ):
$$
f _ {\mu} := \arg \min  _ {f \in \mathcal {F}} \left\{\mathbb {E} _ {S _ {X}} \left[ \mathbb {E} _ {\mu} [ \ell (f (X), Y) | X ] \right] \left(= \mathbb {E} _ {\mu} \left[ \ell (f (X), Y) w _ {\mu} \left(\pi^ {*} (x), \alpha^ {*}\right) \right]\right) \right\}, \text {f o r} \mu = P, Q. \tag {3.4}
$$
Then, for any threshold  $b \in [0,1]$ ,  $\{x \in \mathcal{X} : |f_P(x) - f_Q(x)| \geq b\}$  suggests a region that may suffer model performance degradation with at least  $b$  due to  $Y|X$ -shifts. Without evaluating the performance on the shared distribution  $S_{X}$ , it is hard to distinguish the source of the model performance degradation, i.e. from  $X$ -shifts or  $Y|X$ -shifts.
Empirically, given samples  $\{(x_i^P,y_i^P)\}_{i\in [n_P]}$  from  $P$  and  $\{(x_j^Q,y_j^Q)\}_{j\in [n_Q]}$  from  $Q$ , we estimate  $\hat{\alpha} = \frac{n_Q}{n_P + n_Q}$  and then train a binary "domain" classifier  $\hat{\pi} (x)$  to approximate the ratio  $\pi^{*}(x)$ . Note that the "domain" classifier can be any black-box method, and we use XGBoost throughout. Then we plug these empirical estimands in to obtain the estimated likelihood ratios  $w_{\mu}(\hat{\pi} (x),\hat{\alpha})$  and learn prediction models  $f_{P}$  and  $f_{Q}$  in Equation (3.4). To investigate the model difference under  $S_{X}$ , we pool samples from  $P$  and  $Q$  together and set sample weights as:
$$
\lambda_ {i} ^ {P} = \frac {w _ {P} (\hat {\pi} (x _ {i} ^ {P}) , \hat {\alpha})}{\sum_ {k \in [ n _ {P} ]} w _ {P} (\hat {\pi} (x _ {k} ^ {P}) , \hat {\alpha})} \quad \forall i \in [ n _ {P} ], \quad \lambda_ {j} ^ {Q} = \frac {w _ {Q} (\hat {\pi} (x _ {j} ^ {Q}) , \hat {\alpha})}{\sum_ {k \in [ n _ {Q} ]} w _ {Q} (\hat {\pi} (x _ {k} ^ {Q}) , \hat {\alpha})} \quad \forall j \in [ n _ {Q} ], \tag {3.5}
$$
which are used to learn a prediction model  $h(x)$  to approximate  $|f_{P}(x) - f_{Q}(x)|$  on the shared distribution  $S_{X}$ . The pseudo-code is summarized in the Algorithm 1; To allow simple interpretation and efficient region identification, we use a shallow decision tree  $h(x)$  and consider the region  $\mathcal{R}$  corresponding to the feature range of a leaf node within the tree. More details could be found in Appendix C.5 and Appendix C.6. We show that the node splitting criterion in a standard decision trees training procedure is equivalent with our goal of finding regions with the largest discrepancy in Appendix C.4.
3.2 Data-based Interventions
Using Algorithm 1, we now demonstrate how a better understanding of distribution shifts can facilitate the design of interventions. We focus on the ACS Income dataset where the goal is to predict whether
Algorithm 1: Identify Regions with Strong  $Y|X$ -Shifts.
Input: Source samples  $\{(x_i^P, y_i^P)\}_{i \in [n_P]} \stackrel{\text{i.i.d.}}{\sim} P$  and target samples  $\{(x_j, y_j)\}_{j \in [n_Q]} \stackrel{\text{i.i.d.}}{\sim} Q$ . Model discrepancy threshold  $b$ .
1 Estimate  $\hat{\pi} (x)\approx \mathbb{P}(\tilde{X}\sim Q_x|\tilde{X} = x)$  by training a classifier on the source and target samples.
2 Calculate density ratios  $w_{\mu}(\hat{\pi}(x), \hat{\alpha})$  according to Equation (3.2) and (3.3) for  $\mu = P, Q$ .
3 Fit prediction models  $f_{\mu}$  according to Equation (3.4) replacing  $w_{\mu}(\pi^{*}(x),\alpha^{*})$  there with  $w_{\mu}(\hat{\pi} (x),\hat{\alpha})$  for  $\mu = P,Q$ .
4 Fit a model  $h(x)$  to predict  $|f_{P}(x) - f_{Q}(x)|$  using samples  $\{(x_{i}^{P},y_{i}^{P})\}_{i\in [n_{P}]}$  and  $\{(x_j^Q,y_j^Q)\}_{j\in [n_Q]}$ , each with the weight  $\lambda_i^P$  (or  $\lambda_j^Q$  respectively), according to Equation (3.5).
Output: Region  $\mathcal{R} = \{x\in \mathcal{X}:h(x)\geq b\}$
an individual's income exceeds  $50\mathrm{k}$  ( $Y$ ) based on their tabular census data ( $X$ ). We train an income classifier on 20,000 samples from California (CA, source), and deploy the classifier in Puerto Rico and South Dakota (PR & SD, target), where we get 4,000 samples from PR and SD after deployment. Given the considerable disparities in the economy, job markets, and cost of living between CA and PR/SD, we observe substantial performance degradation due to distribution shifts.
In Figure 4a, we first decompose the performance degradation from CA to PR to understand the shift and find  $Y|X$ -shifts are the predominant factor. The calculation of  $X$ -shifts and  $Y|X$ -shifts is deferred to Appendix C.1. We dive deeper into the significant  $Y|X$ -shifts and identify from CA to PR for the XGBoost and MLP classifier. From the region shown in Figure 4c and Figure 4d, we find college-educated individuals in business and educational roles (such as management, business, and educational work) exhibit large  $Y|X$  differences.
To illustrate how our analysis can inspire subsequent operational interventions to enhance performance on the target distribution, we study two operational interventions.
Collect specific data from the target To improve target performance, the most natural operational intervention is to collect additional data from the target distribution. While a rich body of work on domain adaptation [69, 23, 30, 90, 89] study how to effectively utilize data from the target distribution to improve performance, there is little work that discusses how to efficiently collect supervised data from the target distribution to maximize out-of-distribution generalization. To highlight the need for future research in this space, we use the interpretable region identified by Algorithm 1 as shown in Figure 4c to simulate a concerted data collection effort.