Title: CFA: Causal Feature Augmentation for High-Dimensional Linear Regression
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: multi-task regression, high-dimensional regression, representation learning, low-sample regression, linear regression

Abstract: High-dimensional prediction with limited samples poses a significant challenge due to severe overfitting. While existing approaches tackle this via regularization, clustering, or representation learning, we introduce a novel framework rooted in causal inference, explicitly uncovering and exploiting latent causal structures linking predictors and responses. Our approach employs a new causal-clustering technique guided by a similarity metric that quantifies shared predictor-response dependencies, grouping variables connected through hidden mediators or common confounders. These identified causal features are then incorporated into an augmented regression model, resulting in sparser, more robust, and generalizable predictions. Experiments across synthetic and real-world datasets, including S&P 500 market data, demonstrate that our method achieves higher regression performance and markedly reduces overfitting compared to existing baselines.