SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Therefore, it is important to understand detailed distribution patterns among these datasets to develop corresponding methodologies to address specific shifts, which is also observed in a recent benchmark [101]. Yang et al. [99] recently introduce one benchmark characterizing different patterns of subpopulation shift. This benchmark focuses on the relationship between a specific attribute and covariates, such as spurious correlation, which can be challenging to identify in tabular datasets. Besides, they focus on changes in the subpopulation, which is a subset of distribution shift patterns. For example, individuals in different states may suffer from little subpopulation shifts but still incur a large distribution shift. Individuals' demographic features are similar but the income level differs greatly between CA and PR in ACS Income, as demonstrated in Table 2. Besides, Kulinski and Inouye [52] use optimal transport to explain the shift between two distributions recently. But their method could not explicitly decompose the performance degradation as in our work. Motivated by the challenges above, our work hopes to fill in the gap by demonstrating one benchmark on tabular datasets with detailed analysis.
Table 2: Descriptive Characteristics under different states in ACS income dataset
State	CA	MA	NY	NE	MT	SD	PR
Total Size	195665	40114	103021	10785	5463	4899	9071
White Man	0.3311	0.4161	0.3687	0.4822	0.4959	0.4691	0.3526
Nonwhite Man	0.1969	0.0878	0.1379	0.0454	0.0428	0.0492	0.1818
White Woman	0.2873	0.4066	0.3464	0.4318	0.4221	0.4297	0.3124
NonWhite Woman	0.1847	0.0895	0.1471	0.0404	0.0392	0.0521	0.1532
P(Y=0)	0.589	0.532	0.585	0.688	0.707	0.730	0.894
Choice of Domains in Distribution Shifts Although it is popular in the fairness literature to set each demographic subgroup in the adult and ACS dataset as one domain [104, 22, 25], the relative regret of models trained on these domains is quite small shown in Figure 1 empirically. This implicitly shows that  $Y|X$ -shifts are not strong across different demographic subgroups. In contrast, the model usually experiences relatively large  $Y|X$ -shifts under different spatial domains, which corresponds to the right-hand side of Figure 1. Besides, from the practical perspective, machine learning models often need to be deployed in spatialtemporal domains (by city, state, and country-wise; by year) only with few observed data while the model is trained with abundant data from the source domain. This is why we mainly choose spatiotemporal domains to benchmark distribution shifts in our case.
A.2 Connection between Non-algorithmic Interventions and Algorithmic Interventions
Here we discuss the connections between our non-algorithmic interventions and several branches of methods addressing the distribution shifts, including active learning, imitation learning, causal learning, feature sensitivity analysis, and feature importance analysis.
Active Learning. Active learning aims to improve model performance by acquiring a limited number of labels from the target distribution. See the survey [84] for a detailed reference. The challenge here is to quantify the value of unlabeled data so that we can select samples better. The selection criterion of existing approaches includes estimated variance [20], influence on the model performance [59], and Shapley value [32, 33, 44], or source-target distance metrics through the importance weighting [18, 61]. Some work also proposed running a regression problem for the query procedure based on learning strategies from the existing dataset [48]. Specifically, the target distribution where we query data and aim to evaluate the model subsequently differs from the source
distribution. Some work also develops active learning methods on domain adaptation to select additional samples from the target distribution [88, 87]. However, these works usually assume some restricted distribution structures between the source and target distribution such as only  $X$ -shifts occur, which may not hold in the tabular data. In contrast, we assume we have a few labeled target data but impose no restrictions on the two distributions. And we sample data in specific regions after identifying regions where we experience the largest  $Y|X$ -shifts.
Imitation Learning Imitation learning aims to mimic the behavior of the expert with some off-policy data. This usually occurs in the reinforcement learning setup where a learner aims to learn the best action given each state [41]. Here, distribution shifts occur due to the mismatch of the state coverage between the observed data (source domain) and the environment to deploy (target domain). This raises the need for the importance sampling method to match the two distributions in imitation learning [92, 94]. In fact, the underlying conditional distribution does not change only with covariate (state) shifts in the imitation learning setup. Therefore, this does not fit into our case where the conditional distribution between the source and target domains differ.
Causal Learning Causal learning methods receive much attention in the field of machine learning. The core idea of causal learning [72, 8] is to learn causally invariant relationships across multiple pre-defined training environments. Arjovsky et al. [8] propose Invariant Risk Minimization (IRM) to learn invariant representations across environments, and follow-up works [17, 49, 5, 50] propose variations with similar invariant regularizations. However, the effectiveness of these methods are challenged both theoretically and empirically. Theoretically, Rosenfeld et al. [78] illustrate that IRM could fail in a nonlinear context, and Liu et al. [58] demonstrate that the learned invariance property largely depends on the quality of pre-defined environments. Empirically, Gulrajani and Lopez-Paz [35] show that when carefully implemented and tuned, ERM still outperforms the state-of-the-art methods in terms of average performance.
Compared with the simple non-algorithmic interventions in this work, causal learning methods rely heavily on the invariance assumptions and have strict requirements on the quality of multi-environment data. This restricts their applicability in practice, since modern datasets are often collected without explicit environment labels, and in many scenarios, it is quite hard to pre-define meaningful environment labels. Our proposed non-algorithmic interventions (collecting features and data) do not rely on the invariance assumption, and it could serve as a "solution" when observing performance degradation, which helps to analyze the model failure and to direct further improvements. And we hope that these simple data-centric interventions could inspire future research in this direction to mitigate the effects of distribution shifts.
Since we are also considering other missing features to improve the performance under distribution shifts, there are some works providing insights on the feature / region importance to the final output.
Feature Analysis: Sensitivity and Importance. There are two streams of literature measuring the relationship between input features and the response variable. These are feature sensitivity analysis and feature importance. Feature sensitivity analysis aims to quantify the sensitivity of the performance metrics to each input. This helps understand the impact of variations in input features on the output of a simulation system. Classical metrics include ANOVA, Sobol indices [68] and Morris methods [43]. Meanwhile, feature importance aims to understand the performance decomposition of different algorithms. Shapley value-based approaches gain the most popularity in understanding the attribution of predictions to each input feature [60, 21]. Some work leverages these ideas to understand the difference in distribution shifts to each existing input feature in the data [15, 106]. They decompose the shift on the joint distribution to particular  $X_{i}$ -shifts or the condition  $X_{j}|X_{i}$ -shifts ( $X_{i}$  and  $X_{j}$  denote different features) under a known causal graph. In contrast, we investigate the additional missing feature beyond existing datasets and aim to reduce the performance drop under distribution shifts in our paper. Specifically, we focus on the local regions where the distribution incurs the largest shifts and add the feature ENG in our main body due to our prior knowledge that "ENG" feature would yield the largest difference in that subpopulation between two distributions. Our region-based
approach, in fact, can be further rigorously extended to investigate the marginal distribution of what features would yield the largest difference in that region between two distributions. We hope this can inspire researchers to apply refined tools in this line such as Shapley value to help understand and mitigate distribution shifts.
Region Analysis: Attribution of Distribution Shifts. Similar to our algorithmic goal in Section 3.1 of identifying regions where model learners are different, Oberst et al. [67] and Lim et al. [56] developed specific methods to identify specific covariate regions where model learners are different. The difference between their setup and ours lies in the sampling and assumption of observed data. In their approach, the observed data is sampled i.i.d. across various prediction models, without any distribution shift. Besides, they can observe only one selected prediction result per sample from all the model learners. However, in our case study, the models are built on datasets from two domains that experience distribution shifts. As a result, we can further isolate the model difference based on the shared input space  $X$  and differentiate it from the total difference, specifically focusing on the  $Y|X$ -shifts.
B Discussion on Limitations
We discuss some limitations of our work.
For Benchmark First, we only consider the source-target transfer pairs from datasets including the economic and transportation domain, and we leave the detailed pattern evaluation of these datasets in other domains such as the medical area (e.g. MIMIC-III [45]) as an interesting direction of the future work. Besides, we only consider the source and target from one fixed domain (i.e. one state or city). In practice, it is reasonable to extend this benchmark to consider the source and target distribution with multiple domains of varying proportions. Our results of characterizing the distribution shift patterns highlight the importance of utilizing other refined tools. These tools can help us understand the difference between real-world distribution shifts and enable further investigation and analysis.
For Algorithm 1 Our Algorithm 1 is proposed to identify risky regions with large  $Y|X$ -shifts when observing severe model degradation between the source target distributions. Therefore, one limitation is that it requires target data to identify the risky regions with large  $Y|X$ -shifts. This algorithm cannot evaluate the generalization performance when the target distribution is completely unknown.
Furthermore, when conducting non-algorithmic interventions based on the risky regions learned by Algorithm 1, researchers should be careful and incorporate more background knowledge to find the proper way. For example, in Section 3.2, we analyze the risky regions and find that the "ENG" feature may be important to mitigate the  $Y|X$ -shifts. This is from our prior knowledge that the official language between the two states is different. Also, when Algorithm 1 is misused (e.g., the learned risky regions are used without destination or not being checked carefully), it might harm some vulnerable groups.
For Data-Collection In Section 3.2, we propose two simple non-algorithmic interventions to mitigate the  $Y|X$ -shifts, one of which is to efficiently collect target data from the risky region. It achieves much better results combined with several typical methods on our benchmark. However, in practice, we acknowledge that this technique could only be used when we can obtain data from the target distribution and the data collection procedure does not raise any privacy concerns and predatory inclusion.
Applicability Across Different Data Modalities In this work, we focus on real-world distribution shifts in tabular data settings. We propose Algorithm 1 to find the risk regions and come up with some simple data-based interventions. Therefore, we do not investigate in-depth the applicability of the proposed methods across other data types. However, we argue that our proposed Algorithm 1 could generalize to complicated data types (e.g., image data) with corresponding deep models.
Specifically, both the domain classifier (to estimate  $\mathbb{P}(X$  from  $Q_{X}|X)$ ) and region learner  $h(X)$  should be replaced by deep neural networks. And our data-based interventions also have the potential to incorporate with more complicated data types. We leave this investigation to future work.
C Case Study Details
In this section, we provide more details about our case study in the main body.
C.1 DISDE to  $X$ -shifts and  $Y|X$ -shifts
When facing performance degradation under distribution shifts, one direct idea is to figure out the reasons why the performance drop. To this end, Cai et al. [16] propose DDistribution Shift DEcomposition (DISDE) to attribute the total performance degradation to  $Y|X$ -shifts and  $X$ -shifts. Specifically, given samples  $(X,Y)$  from distributions  $P$  and  $Q$ , to quantify the discrepancy between  $P_{Y|X}$  and  $Q_{Y|X}$ , they first control the marginal distribution on  $\mathcal{X}$  by introducing the shared distribution  $S_{X}$ . From that, we can estimate the performance degradation caused by  $Y|X$ -shifts and that caused by  $X$ -shifts could also be estimated by comparing  $S_{X}$  with  $P_{X}$  and  $Q_{X}$ , respectively. Note that DISDE could be used in image datasets (see Section 4.2 in [16]). The official code for DISDE could be found at https://github.com/namkoong-lab/disde. We specify the formula of DISDE as follows:
$$
\begin{array}{l} \mathbb {E} _ {Q} \left[ \ell \left(f _ {P} (X), Y\right) \right] - \mathbb {E} _ {P} \left[ \ell \left(f _ {P} (X), Y\right) \right] = \mathbb {E} _ {S _ {X}} \left[ R _ {P} (X) \right] - \mathbb {E} _ {P} \left[ R _ {P} (X) \right] (I) \\ + \mathbb {E} _ {S _ {X}} \left[ R _ {Q} (X) - R _ {P} (X) \right] (II) \\ + \mathbb {E} _ {Q} \left[ R _ {Q} (X) \right] - \mathbb {E} _ {S _ {X}} \left[ R _ {Q} (X) \right], (III) \\ \end{array}
$$
where  $R_{\mu}(x) \coloneqq \mathbb{E}_{\mu}[\ell(f_P(X), Y)|X = x]$  for  $\mu = P, Q$  is denoted as the conditional risks on  $P$  and  $Q$ .  $S_X$  is the share distribution with support contained in both  $P_X$  and  $Q_X$ . Then we see the sum of the two terms (I) and (III) as the performance drop attributed to  $X$ -shifts and the term (II) as the performance drop attributed to  $Y|X$ -shifts. Besides, we also implement DISDE in our released package named WHYSHIFT, which could be found at https://github.com/namkoong-lab/whyshift.
C.2 Prevalence of  $Y|X$ -Shifts
For each of the 7 settings in Table 1, we only select one target distribution in the main body, while our benchmark supports multiple target distributions. Specifically, for ACS Income, ACS Mobility, ACS Pub. Cov, we have 50 target distributions (the other 50 American states) in total for each setting; for Taxi, we have 3 target distributions (other cities Mexico City, Bogotá, Quitio); for the temporal shift version of ACS Pub. Cov (i.e. setting 6), we have 3 target distributions (i.e. the year 2014, 2017, and 2021); and for US Accident, we have 13 target distributions (13 American states). Therefore, we have 169 source-target transfer pairs in total for these 7 settings.
We use XGBoost classifier and calculate the decomposition of performance degradation via DISDE [16]. We calculate the  $Y|X$ -shift ratio from the source distribution  $P$  to the target distribution  $Q$ :
$$
Y | X \text {- s h i f t r a t i o} = \frac {\mathbb {E} _ {S _ {X}} \left[ R _ {Q} (X) - R _ {P} (X) \right]}{\mathbb {E} _ {Q} \left[ \ell \left(f _ {P} (X) , Y\right) \right] - \mathbb {E} _ {P} \left[ \ell \left(f _ {P} (X) , Y\right) \right]},
$$
where  $R_{\mu}(X)$  is defined in Appendix C.1. We first focus on pairs with relatively strong distribution shifts (i.e. performance degradation larger than 8 percentage points). Across these pairs, we find that  $70.2\%$  pairs have over  $60\%$ $Y|X$ -shifts, and  $87.2\%$  pairs have over  $50\%$ $Y|X$ -shifts, indicating the prevalence of  $Y|X$  shifts. To visualize the result better, in all 22 settings in our benchmark (as shown in Table 3), we focus on transfer pairs with performance degradation larger than 5 percentage points and plot the histogram of the ratio of  $Y|X$ -shifts in Figure 8. From Figure 8, we could see that  $Y|X$ -shifts are prevalent in real-world distribution shifts.
C.3 Details of Algorithm 1
In this section, we provide a more detailed introduction of our proposed Algorithm 1. We propose a simple yet efficient method to identify data regions with strong  $Y|X$  shifts, and it could inspire operational and modeling interventions as shown in Section 3.2.
Figure 8: Histogram of  $Y|X$ -shift ratio in our benchmark.
Consider a model  $f: \mathcal{X} \to \mathcal{Y}$  that predicts outcome  $Y$  from covariates  $X$ . Let  $\ell(f(x), y)$  be a loss function denoting a notion of predictive error (e.g., cross-entropy loss, mean-square error). Our goal is to identify a region  $S \subseteq \mathcal{X}$  where  $P(Y|X)$  differs a lot from  $Q(Y|X)$ . In order to directly compare  $P(Y|X)$  and  $Q(Y|X)$ , we must do an apples-to-apples comparison: we cannot know  $Q(Y|X)$  for  $X$ 's that are primarily observed in  $P$  (and vice versa).
To address this, we first construct a shared distribution  $S_{X}$  over  $X$  whose support is contained in that of both  $P_{X}$  and  $Q_{X}$ . We choose a specific shared distribution  $S_{X}$  over  $X$  whose support is contained in that of  $P_{X}$  and  $Q_{X}$  (following [16]). Ideally, the chosen shared distribution would exhibit a higher density when both  $P_{X}$  and  $Q_{X}$  densities are high, and a lower density when either of the two possesses a low density. This strategy effectively allows regions of shared density to be more pronounced. Recall that  $p_{X}, q_{X}, s_{X}$  are the densities of  $X$  under  $P, Q$  and  $S$ , we formulate  $s_{X}$  as
$$
s _ {X} (x) \propto p _ {X} (x) q _ {X} (x) / \left(p _ {X} (x) + q _ {X} (x)\right). \tag {C.1}
$$
Choices of  $S_{X}$  Here we would like to discuss different choices of  $S_{X}$  to provide more intuitions. As demonstrated in [16], we can use many forms of the shared distribution. For example, we could choose the following form:
$$
s _ {X} (x) \propto \min  \left\{p _ {X} (x), q _ {X} (x) \right\}, \tag {C.2}
$$
which guarantees that the support of  $S_{X}$  is contained in that of both  $P_{X}$  and  $Q_{X}$ . Another choice is:
$$
s _ {X} (x) \propto \left\{ \begin{array}{l l} p _ {X} (x) + q _ {X} (x) & \text {i f} \min  \left\{\frac {p _ {X} (x)}{q _ {X} (x)}, \frac {q _ {X} (x)}{p _ {X} (x)} \right\} \geq \epsilon , \\ 0 & \text {o t h e r w i s e}, \end{array} \right. \tag {C.3}
$$
for some  $\epsilon \geq 0$ . This form of  $S_{X}$  defines shared samples as those with high likelihood ratios. Notably, for all the three forms of  $S_{X}$ , if  $P_{X} = Q_{X}$ , then  $S_{X} = P_{X} = Q_{X}$ . And when  $p_{X}(x) \gg q_{X}(x)$  or  $p_{X}(x) \ll q_{X}(x)$ , Equation (C.1) and Equation (C.2) become similar. In our Algorithm 1, we use the form of Equation (C.1), and Cai et al. [16] observe that in practice the qualitative conclusions are not very sensitive to the specific choice of shared distribution.
Intuitions behind Algorithm 1 Since we do not have access to samples from the shared distribution  $S_{X}$ , we reweight samples from  $P_{X}$  and  $Q_{X}$  using the likelihood ratios:
$$
\frac {s _ {X}}{p _ {X}} (x) \propto \frac {q _ {X} (x)}{p _ {X} (x) + q _ {X} (x)} \text {a n d} \frac {s _ {X}}{q _ {X}} (x) \propto \frac {p _ {X} (x)}{p _ {X} (x) + q _ {X} (x)}. \tag {C.4}
$$
Then we define  $\hat{\alpha}$  as the proportion of the pooled data that comes from distribution  $Q$ :
$$
\hat {\alpha} = \frac {n _ {Q}}{n _ {P} + n _ {Q}} \quad \text {a n d} \quad \hat {\pi} (x) = \mathbb {P} (\tilde {X} \text {f r o m} Q _ {X} | \tilde {X} = x), \tag {C.5}
$$
where  $\hat{\pi} (x)$  denotes the probability of a sample to come from  $Q_{X}$ . Using Bayes' rule, we have:
$$
\begin{array}{l} \hat {\pi} (x) = \frac {\mathbb {P} (\tilde {X} = x \mid \tilde {X} \text {f r o m} Q _ {X}) \mathbb {P} (\tilde {X} \text {f r o m} Q _ {X})}{\mathbb {P} (x)} = \frac {\hat {\alpha} q (x)}{\hat {\alpha} q (x) + (1 - \hat {\alpha}) p (x)}, \tag {C.6} \\ = \frac {\hat {\alpha}}{\hat {\alpha} + (1 - \hat {\alpha}) \frac {p (x)}{q (x)}}. \\ \end{array}
$$
Noting that the ratio  $\hat{\pi} (x)$  can be modeled as the probability that an input  $x$  came from  $P_{X}$  vs  $Q_{X}$ , we train a binary "domain" classifier to estimate the ratios. (The "domain" classifier can be any black-box method, and we use XGBoost throughout.)
Then the likelihood ratios that we care about could be reformulated as:
$$
\frac {s _ {X}}{p _ {X}} (x) \propto \frac {1}{\frac {p _ {X} (x)}{q _ {X} (x)} + 1} \text {a n d} \frac {s _ {X}}{q _ {X}} (x) \propto \frac {\frac {p _ {X} (x)}{q _ {X} (x)}}{\frac {p _ {X} (x)}{q _ {X} (x)} + 1}, \tag {C.7}
$$
which gives that:
$$
\frac {s _ {X}}{p _ {X}} (x) \propto \frac {\hat {\pi} (x)}{(1 - \hat {\alpha}) \hat {\pi} (x) + \hat {\alpha} (1 - \hat {\pi} (x))} \quad \text {a n d} \quad \frac {s _ {X}}{q _ {X}} (x) \propto \frac {1 - \hat {\pi} (x)}{(1 - \hat {\alpha}) \hat {\pi} (x) + \hat {\alpha} (1 - \hat {\pi} (x))}. \tag {C.8}
$$
After obtaining the likelihood ratios  $\frac{s_X}{p_X}(x)$  and  $\frac{s_X}{q_X}(x)$ , we could do an apples-to-apples comparison: we estimate  $P_{Y|X}$  and  $Q_{Y|X}$  over the shared distribution  $S_X$  (using XGBoost  $\mathcal{F}$ )
$$
f _ {P} := \arg \min  _ {f \in \mathcal {F}} \left\{\mathbb {E} _ {S _ {X}} \left[ \mathbb {E} _ {P} [ \ell (f (X), Y) | X ] \right] = \mathbb {E} _ {P} \left[ \ell (f (X), Y) \frac {d S _ {X}}{d P _ {X}} (X) \right] \right\}, \tag {C.9}
$$
$$
f _ {Q} := \arg \min  _ {f \in \mathcal {F}} \left\{\mathbb {E} _ {S _ {X}} \left[ \mathbb {E} _ {Q} [ \ell (f (X), Y) | X ] \right] = \mathbb {E} _ {Q} \left[ \ell (f (X), Y) \frac {d S _ {X}}{d Q _ {X}} (X) \right] \right\}. \tag {C.10}
$$
Then, for any threshold  $b \in [0,1]$ ,  $\{x \in \mathcal{X} : |f_P(x) - f_Q(x)| \geq b\}$  suggests a region that may suffer model performance degradation due to  $Y|X$ -shifts.
C.4 Analysis of Decision Tree
In Algorithm 1, we use a shallow decision tree  $h(x)$  to approximate  $y = |f_{P}(x) - f_{Q}(x)|$  on the shared distribution  $S_{X}$  to find the covariate region with highest discrepancy. In our decision tree, we use the squared error as the splitting criterion. And below we demonstrate that this criterion is equivalent to maximizing the discrepancy between two children nodes.
Suppose there are  $N$  samples with outcomes  $\{y_i\}_{i\in [N]}$  belonging to tree node  $fa$ , and these samples are split into two children nodes  $s_1, s_2$ , where the node  $s_1, s_2$  denote the set of sample indices in the two children nodes respectively. The squared error criterion to split  $fa$  into  $s_1$  and  $s_2$  is:
$$
\min  _ {s _ {1}, s _ {2}} \left\{\mathcal {L} \left(s _ {1}, s _ {2}\right) := \frac {1}{N} \left(\sum_ {i \in s _ {1}} \left(y _ {i} - \mu_ {Y, 1}\right) ^ {2} + \sum_ {i \in s _ {2}} \left(y _ {i} - \mu_ {Y, 2}\right) ^ {2}\right) \right\}, \tag {C.11}
$$
$$
\mu_ {Y, 1} := \frac {\sum_ {i = 1} ^ {N} y _ {i} \mathbf {1} _ {\{i \in s _ {1} \}}}{\sum_ {i = 1} ^ {N} \mathbf {1} _ {\{i \in s _ {1} \}}}, \quad \mu_ {Y, 2} := \frac {\sum_ {i = 1} ^ {N} y _ {i} \mathbf {1} _ {\{i \in s _ {2} \}}}{\sum_ {i = 1} ^ {N} \mathbf {1} _ {\{i \in s _ {2} \}}} \tag {C.12}
$$
denote the mean values of the outcome  $Y$  with samples in children nodes  $s_1$  and  $s_2$ . Denote the distribution of the outcome  $Y$  follows the empirical distribution over the  $N$  samples  $\{y_i\}_{i \in [N]}$ . Simplifying (C.11), we have: