SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Since indiscriminately collecting data from the target distribution can be resource-intensive, we concentrate sampling efforts on the subpopulation that may suffer from  $Y|X$ -shifts and selectively gather data on them. For five base methods (logistic regression, MLP, random forest, lightGBM, and XGBoost), we randomly sample 250 points from the whole target distribution and the identified region suffering prominent  $Y|X$ -shifts, respectively. We report the test accuracies in Figure 4e, and observe that incorporating data from this region is more effective in enhancing OOD generalization. While preliminary, our results demonstrate the potential robustness benefits of efficiently allocating resources toward concerted data collection. Future methodological research in this direction may be fruitful; potential connections may exist with active learning algorithms [84, 96, 59].
Add more relevant features We now illustrate the potential benefits of generating qualitative insights on the distribution shift at hand. Our analysis in Figure 4c suggests educated individuals in financial, educational, and legal professions tend to experience large  $Y|X$ -shifts from CA to PR. These roles typically need communication skills, and language barriers could potentially affect their incomes. In California (CA), English is the primary language, while in Puerto Rico (PR), despite both English and Spanish being recognized as official languages, Spanish is predominantly spoken. Consequently, for a model trained on CA data and tested on PR data, incorporating a new feature that denotes English language proficiency (hereafter denoted "ENG") might prove beneficial in improving generalization performances. However, this feature is not included in the ACS Income dataset.
To address this, we went back to the Census Bureau's American Community Survey database to include the ENG feature in the set of covariates. In Figure 4b, we observe that the inclusion of this feature substantially reduces the degradation due to  $Y|X$ -shifts, verifying that the originally missing
(a) Raw setting: drop 10.3
(b) Add ENG: drop 2.1
(c) Region with  $Y|X$  -shifts (XGBoost)
(d) Region with  $Y|X$  -shifts (MLP)
(e) Test accuracies of different ways to incorporate data.
(f) Add ENG (CA→PR)
(g) Add ENG (CA  $\rightarrow$  SD)
Figure 4. Case study illustrations. (a)-(b) Decomposition of performance degradation for the XGBoost classifier from CA to PR. Figure (a) is for the original setting and (b) corresponds to the results post-integration of the "ENG" feature. (c)-(d) Demonstration of Algorithm 1: an interpretable version of the region with strong  $Y|X$ -shifts for the XGBoost and MLP models, respectively. (e) Test accuracies of five typical base methods trained on the source, post addition of 250 randomly selected target observations, and 250 observations from the identified risk region. (f)-(g) Performances of all algorithms prior to and following the addition of the "ENG" feature. Figure (f) corresponds to the CA to PR, and Figure (g) is CA to SD.
Table 1: Overview of datasets and 7 selected settings.
#ID	Dataset	Type	#Samples	#Features	Outcome	#Domains	Selected Settings	Shift Patterns
1	ACS Income	Natural	1,599,229	9	Income≥50k	51	California → Puerto Rico	Y|X ≫ X
2	ACS Mobility	Natural	620,937	21	Residential Address	51	Mississippi → Hawaii	Y|X ≫ X
3	Taxi	Natural	1,506,769	7	Duration time≥30 min	4	New York City→Botoğá	Y|X ≫ X
4	ACS Pub.Cov	Natural	1,127,446	18	Public Ins. Coverage	51	Nebraska → Louisiana	Y|X > X
5	US Accident	Natural	297,132	47	Severity of Accident	14	California → Oregon	Y|X > X
6	ACS Pub.Cov	Natural	859,632	18	Public Ins. Coverage	4	2010 (NY)→2017 (NY)	Y|X < X
7	ACS Income	Synthetic	195,665	9	Income≥50k	2	Younger→Older	Y|X ≪ X
ENG feature may be one cause of  $Y|X$ -shifts. Figure 4f contrasts the performances of 22 algorithms (each with 200 hyperparameter configurations) with original features with those that additionally use the ENG feature. The new feature significantly improves target performances across all algorithms; roughly speaking, we posit that we have identified a variable  $C$  such that  $Y|X, C$  remains similar across CA and PR. However, when we extend this comparison to the source-target pair (CA → SD), we observe no significant improvement (Figure 4g). This highlights that the selection of new features should be undertaken judiciously depending on the target distributions of interest. A feature that proves effective in one target distribution might not yield similar results in another.
Figure 5: Overall performances of all algorithms on the target data in our selected 7 settings.
4 WHYSHIFT: Benchmarking Distribution Shifts on Tabular Data
In this section, we detail our benchmark and summarize the main observations. Our finding highlights the importance of future research that builds an understanding of why the distribution has shifted.
Datasets We explore distribution shifts on 5 real-world tabular datasets from the economic and traffic sectors with natural spatiotemporal distribution shifts. For economic data, we use ACS Income, ACS Mobility, and ACS Public Coverage datasets from the US-wide ACS PUMS data [25], where the outcome is whether an individual's income exceeds  $50\mathrm{k}$ , whether an individual changed the residential address one year ago, and whether an individual is covered by public health insurance, respectively. We primarily focus on spatial shifts across different states in the US. To complement spatial shifts, we derive an ACS Time task based on the ACS Public Coverage dataset, where there are temporal shifts between different years (2010 to 2021). For traffic data, we use US Accident [64, 65] and Taxi [2, 1], where the outcome is whether an accident is severe and whether the total ride duration time exceeds 30 minutes, respectively. We focus on spatial shifts between different states/cities. We summarize the datasets in Table 1 and defer a full description to the Appendix D.1.
Algorithms We evaluate 22 algorithms that span a wide range of learning strategies on tabular data, and compare their performances under different patterns of distribution shifts we construct. Concretely, these algorithms include: (1) base learners: Logistic Regression, SVM, fully-connected neural networks (MLP) with standard ERM optimization; (2) tree ensemble models: Random Forest, XGBoost, LightGBM; (3) robust learning: CVaR-DRO and  $\chi^2$ -DRO with fast implementation [53], CVaR-DRO and  $\chi^2$ -DRO of outlier-robust enhancement [105], Group DRO [79]; (4) imbalanced learning: JTT [57], SUBY, RWY, SUBG, RWG [42], DWR [51] and (5) fairness-enhancing methods: inprocessing method [4] with demographic parity, equal opportunity, error parity as constraints, postprocessing method [37] with exponential and threshold controls. For DRO methods (i.e. (3)), we use MLP as the backbone model. For other algorithms compatible with tree ensemble models (i.e. (4-5)), we use the XGBoost model due to its superior performance on tabular data [34]. For algorithms requiring group labels, we use 'hour' for US Accident and Taxi, and 'sex' for the others. Detailed descriptions for each algorithm can be found in Appendix D.5.
**Benchmarks** We conduct experiments with more than 86,000 model configurations on various source-target distribution shift pairs, and carefully select 7 selected pairs with different distribution shift patterns. In Table 1, we characterize the shift patterns of these 7 source-target pairs, which contain different proportions of  $Y|X$ -shifts and  $X$ -shifts corresponding with plots in Figure 2. The first six settings are natural shifts. In the last setting, we sub-sample the dataset according to age to introduce covariate shift, where we focus on individuals from CA and form two groups according to whether their age is  $\geq 25$ . The source data over-samples the low age group where  $80\%$  is drawn from the group where the individual's age  $\geq 25$ , and the proportions are reversed in the target data.
In Figure 5 and Figure 6, we plot the performance of algorithms using their best hyperparameter configuration on the validation dataset (i.i.d. with the source distribution). Additional results with various source distributions are in the Appendix. Our benchmark is designed to support empirical research, including new learning algorithms and diagnostics that provide qualitative insights on distribution shifts.
Figure 6. Performance drop between source and target data of all algorithms in our selected 7 settings.
(a) DRO & Imb. Methods
Figure 7. (a): Sensitivity of DRO methods and Imbalance Methods w.r.t. configurations. (b) Target performances of 22 algorithms under different validation protocols on ACS Income (CA→PR) setting.
(b) Performances under different validations on ACS Income dataset.
Hyper-parameter Tuning For each model, we conduct a grid search over a large set of hyperparameters. See Appendix D.3 for the complete search space for each method. When one method includes another as a "base" learner (e.g., DRO with MLP, RWY with XGBoost), we explore the full tuning space for the base model (e.g., the cross-product of all MLP hyper-parameters with all DRO hyper-parameters). To control for computational effort, each method is run with 200 configurations for each source-target pair and we select the best configuration according to the i.i.d. validation performance. In Figure 7b, we further compare different choices of validation protocols.
Evaluation Metrics In our benchmark, we include different metrics for a thorough evaluation. Specifically, we use Average Accuracy (micro-average), Worst-group Accuracy, and Macro-F1 score in our main results where we only have one target distribution. For the results with multiple target distributions (i.e. 3 in Taxi, 13 in US Accident and 50 in the others), we present all target accuracies and Macro-F1 scores, as well as the worst-distribution accuracy and Macro-F1 score among all target distributions in Appendix D.6, D.7, D.8, D.9.
Different algorithms do not exhibit consistent rankings over different shift patterns. In Figure 5, we observe the rankings across different shifts are quite different, especially for ACS Income (CA→PR) and ACS Mobility (MS→HI) where  $Y|X$ -shifts dominate. This observation reaffirms the phenomena in Figure 2 that as  $Y|X$ -shifts become stronger, the relationship between source vs target performances becomes less consistent. In Appendix D.3, we also show that even for a fixed source distribution in one fixed prediction task, algorithmic rankings of performances on different target distributions vary a lot.
Tree ensemble methods show competitive performance, but do not significantly improve the generalization drop between source and target data. From Figure 5, tree-based ensembles (yellow bars) show robust and competitive performance on the target distribution in 6 out of 7 settings. However, in Figure 6 which plots the performance degradation between source and target, tree ensembles do not show improved robustness. This suggests that they do not actually achieve better robustness against real-world distribution shifts, and their better performances on target data may simply be due to better fitting the source distribution.
DRO methods are sensitive to configurations, with rankings varying significantly across 7 different settings. From Figure 5, DRO methods exhibit competitive performances on ACS Mobility (MS→HI), Taxi (NYC→BOG), and ACS Income (Young→Old), yet underperform in others. This sensitivity to configurations, as shown in Figure 7a (red points), could be attributed to
the worst-case optimization that perturbs the training distribution within a pre-defined uncertainty set, without any information regarding the target distribution. However, when target information is incorporated for hyper-parameter tuning, as shown in Figure 7b, there is a notable improvement in the performance of DRO methods. Our observations suggest potential avenues for building more refined uncertainty sets in DRO methods.
Imbalance methods and fairness methods show similar performance with the base learner (XGBoost). In our experiments, we choose the XGBoost model as the base learner for imbalance and fairness methods due to its superior performance on tabular data [34]. However, from Figure 5 and Figure 6, imbalance methods and fairness methods do not show a clear improvement upon their base learner (XGBoost, last yellow bar). Further, as shown in Figure 7a, imbalance methods (green) are also quite sensitive to configurations, and their performances do not improve much when their hyperparameters are tuned over the target data (Oracle).
Target information matters in validation. Based on the ACS Income (CA→PR) dataset, we compare different validation protocols, including the best average accuracy, minimum subgroup discrepancy, and best worst-subgroup accuracy on validation data generated from the source distribution. We also use the Oracle validation that chooses the configuration with the best average accuracy on validation data generated from the target distribution. In Figure 7b, we find the first three protocols do not show a significant difference. However, oracle validation with target information substantially improves the effectiveness of both DRO and tree ensemble methods. We conclude using target information for model selection can provide robustness gains even with a small target dataset.
Non-algorithmic interventions warrant greater consideration. Reflecting on Section 3, it is clear that operational interventions yield significant enhancements for various methods, as demonstrated in Figure 4e and Figure 4f. In comparison to algorithmic interventions, such as designing different algorithms (e.g., DRO, Imbalance methods), a data-centric approach can be more effective in addressing distribution shifts. For instance, research on feature collection and feature engineering methods may prove impactful. Another avenue for future work is developing methods that can optimally incorporate expensive samples from the target distribution.
We explore the complexity of distribution shifts in real-world tabular datasets in depth. Using natural shifts from 5 real-world tabular datasets across different domains, we specify each shift pattern and evaluate 22 methods via experiments with over 86k trained models. Our benchmark WHYSHIFT encompasses various distribution shift patterns to evaluate the robustness of the methods. We propose a simple but effective algorithm to identify regions with large  $Y|X$ -shifts, and through a comprehensive case study, we demonstrate how a better understanding of distribution shifts facilitates algorithmic and data-based interventions. Our findings highlight the importance of future research to understand how and why distributions differ in real-world applications.
Our study leaves many open directions for improvements in future work. Our benchmark only includes tabular datasets from the economic and transportation domain. Considering datasets from other domains such as the medicine or those involving feature embeddings may highlight different types of distribution shift. On the algorithmic side, our region-identification algorithm requires some target data to identify risky regions and cannot be used in cases where the target distribution is completely unknown. Furthermore, targeted data collection on regions of  $Y|X$ -shifts may be pose ethical and privacy concerns for marginalized groups. We provide more discussion in Appendix B.
We thank Tiffany Cai for her help with implementing the DISDE method on our benchmarks. Peng Cui was supported in part by National Key R&D Program of China (No. 2018AAA0102004), National Natural Science Foundation of China (No. U1936219, 62141607). Hongseok Namkoong was partially supported by the Amazon Research Award.
[1] Other taxi dataset. https://www.kaggle.com/datasets/mnavas/taxi-routes-for-mexico-city-and-quito.
[2] Us taxi dataset. https://www.kaggle.com/competitions/nyc-taxi-trip-duration/data.
[3] Robert Adragna, Elliot Creager, David Madras, and Richard S. Zemel. Fairness and robustness in invariant learning: A case study in toxicity classification. CoRR, abs/2011.06485, 2020.
[4] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A reductions approach to fair classification. In International Conference on Machine Learning, pages 60–69. PMLR, 2018.
[5] Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar. Invariant risk minimization games. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 145-155. PMLR, 13-18 Jul 2020.
[6] Evelin Amorim, Marcia Cançado, and Adriano Veloso. Automated essay scoring in the presence of biased ratings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 229–237, 2018.
[7] Shushan Arakelyan, Rocktim Jyoti Das, Yi Mao, and Xiang Ren. Exploring distributional shifts in large language models for code analysis. CoRR, abs/2303.09128, 2023.
[8] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019.
[9] Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, et al. From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE transactions on medical imaging, 38(2):550-560, 2018.
[10] Sara Beery, Arushi Agarwal, Elijah Cole, and Vighnesh Birodkar. The iwildcam 2021 competition dataset. arXiv preprint arXiv:2105.03494, 2021.
[11] Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John T. Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, and Yunfeng Zhang. AI fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM J. Res. Dev., 63(4/5):4:1-4:15, 2019.
[12] Sarah Bird, Miro Dudík, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. Fairlearn: A toolkit for assessing and improving fairness in ai. Microsoft, Tech. Rep. MSR-TR-2020-32, 2020.
[13] Jose Blanchet, Yang Kang, and Karthyek Murthy. Robust wasserstein profile inference and applications to machine learning. Journal of Applied Probability, 56(3):830-857, 2019.
[96] Binhui Xie, Longhui Yuan, Shuang Li, Chi Harold Liu, and Xinjing Cheng. Towards fewer annotations: Active learning via region impurity and prediction uncertainty for domain adaptive semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 8058-8068. IEEE, 2022.
[97] Linyi Yang, Shuibai Zhang, Libo Qin, Yafu Li, Yidong Wang, Hanmeng Liu, Jindong Wang, Xing Xie, and Yue Zhang. Glue-x: Evaluating natural language understanding models from an out-of-distribution generalization perspective, 2023.
[98] Yuzhe Yang, Haoran Zhang, Dina Katabi, and Marzyeh Ghassemi. Change is hard: A closer look at subpopulation shift. CoRR, abs/2302.12254, 2023.
[99] Yuzhe Yang, Haoran Zhang, Dina Katabi, and Marzyeh Ghassemi. Change is Hard: A Closer Look at Subpopulation Shift, 2023.
[100] Huaxiu Yao, Caroline Choi, Bochuan Cao, Yoonho Lee, Pang Wei Koh, and Chelsea Finn. Wild-time: A benchmark of in-the-wild distribution shift over time. In NeurIPS, 2022.
[101] Huaxiu Yao, Caroline Choi, Bochuan Cao, Yoonho Lee, Pang Wei W Koh, and Chelsea Finn. Wild-time: A benchmark of in-the-wild distribution shift over time. Advances in Neural Information Processing Systems, 35:10309-10324, 2022.
[102] Han Yu, Peng Cui, Yue He, Zheyan Shen, Yong Lin, Renzhe Xu, and Xingxuan Zhang. Stable learning via sparse variable independence. AAAI, abs/2212.00992, 2022.
[103] John R Zech, Marcus A Badgeley, Manway Liu, Anthony B Costa, Joseph J Titano, and Eric Karl Oermann. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS medicine, 15(11):e1002683, 2018.
[104] Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International conference on machine learning, pages 325-333. PMLR, 2013.
[105] Runtian Zhai, Chen Dan, Zico Kolter, and Pradeep Ravikumar. DORO: Distributional and Outlier Robust Optimization. In Proceedings of the 38th International Conference on Machine Learning, pages 12345–12355. PMLR, 2021.
[106] Haoran Zhang, Harvineet Singh, Marzyeh Ghassemi, and Shalmali Joshi. "Why did the Model Fail?: Attributing Model Performance Changes to Distribution Shifts, 2023.
[107] Xingxuan Zhang, Yue He, Tan Wang, Jiaxin Qi, Han Yu, Zimu Wang, Jie Peng, Renzhe Xu, Zheyan Shen, Yulei Niu, et al. Nico challenge: Out-of-distribution generalization for image recognition challenges. In Computer Vision-ECCV 2022 Workshops: Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part VI, pages 433-450. Springer, 2023.
[108] Sicheng Zhao, Bo Li, Xiangyu Yue, Yang Gu, Pengfei Xu, Runbo Hu, Hua Chai, and Kurt Keutzer. Multi-source domain adaptation for semantic segmentation. arXiv preprint arXiv:1910.12181, 2019.
A Relevant Work 18
A.1 On Distribution Shifts 18
A.2 Connection between Non-algorithmic Interventions and Algorithmic Interventions 19
B Discussion on Limitations 21
C Case Study Details 22
C.1 DISDE to  $X$  -shifts and  $Y|X$  -shifts 22
C.2 Prevalence of  $Y|X$  -Shifts 22
C.3 Details of Algorithm 1 22
C.4 Analysis of Decision Tree 24
C.5 Details of Non-algorithmic Interventions 25
C.6 Potential Alternative Approach of Identifying Risk Region 25
D Benchmark Details 26
D.1 Datasets & Settings 26
D.2 Algorithms & Implementations 27
D.3 Parameter Search Space 28
D.4 Training Details 30
D.5 Detailed Results of 7 Selected Settings in Main Body 30
D.6 Full Results of 22 Settings 32
D.7 Worst-Domain Accuracy 32
D.8 Average Macro-F1 Score 32
D.9 Worst-Domain Macro-F1 Score 32
D.10 Agreement and Maintenance Plan 38
A Relevant Work
A.1 On Distribution Shifts
Distribution Shift Benchmarks. Existing distribution shift benchmarks primarily concentrate on image and language datasets [74, 47, 107, 81, 101] to assess the robustness and efficiency of algorithms in real applications.
We briefly review benchmarks that address distribution shifts across various data types, including image and language data. For image data, several datasets capture natural distribution shifts, such as spatial and temporal variations. PACS [54] and Office-Home [91] categorize environments to be image styles. VLCS [29] and iWildCam [10] set their primary environments as data sources. DomainNet [108] is built on PACS and provides a more extensive selection with additional domains and categories. In recent developments, Koh et al. [47] collect several datasets to establish WILDS, setting a new structure for OOD generalization. Similarly, Yao et al. [100] introduced Wild-Time, highlighting temporal distribution shifts across diverse real-world scenarios. For language data, CivilComments [14] and Amazon [66] consist of individual comments collected from different users and distinctive groups (e.g. male and female). GLUE-X [97] provides a unified benchmark for evaluating OOD robustness in NLP models. For other types of data, OGB-Mo1PCBA [75] collects
molecular graphs in over 100,000 scaffolds and formulates a molecular property prediction task across different scaffolds. Towards auto-engineering, Py150 [73] contains codes from 8,421 git repositories for code completion generalization.
However, these datasets/benchmarks do not specify or investigate in-depth the distribution shift patterns, and there is a noticeable lack of benchmark papers that specifically address real-world tabular datasets. These tabular datasets often present distinct patterns compared to image/language datasets, including prevalent methods [34] and shift patterns.