Title: Counterfactual Residual Data Augmentation for Regression
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: Data Augmentation, Counterfactual Reasoning, Tabular Regression, Residual Modeling, Synthetic Sample Generation

Abstract: Data-driven modeling in real-world regression tasks often suffers from limited
training samples, high collection costs, and noisy observations. Inspired by the
impact of data augmentation in vision and language, we propose a novel Counterfactual Residual Data Augmentation (CRDA) technique for tabular regression.
Our key insight is that once a regressor has modeled the systematic component of
the data, the remaining noise can be viewed as an invariant residual that remains
stable under small perturbations of carefully selected features. We exploit this
residual invariance to generate new, yet realistic, training samples, effectively expanding the dataset without requiring additional real data. Our method is model-agnostic and readily applicable to various types of regressors. In experiments
across datasets from a variety of benchmark repositories, on average, CRDA reduces an MLP Regressor’s MSE by 22.9% and an XGBoost Regressor’s MSE by 6.4%. When compared to existing state-of-the-art synthetic data generators,
CRDA consistently outperforms in MSE reduction. By adding principled counterfactual variations to the training data, our method offers a simple and efficient remedy for noise-prone, small-sample regression settings.