Near-Optimal Linear Regression under Distribution Shift
Authors: Qi Lei, Wei Hu, Jason D. Lee
Keywords: minimax estimator, covariate shift, model shift
Abstract: Transfer learning is an essential technique when sufficient data comes from the source domain, while no or scarce labeled data is from the target domain. We develop estimators that achieve minimax linear risk for linear regression problems under the distribution shift. Our algorithms cover different kinds of settings with covariate shift or model shift. We also consider when data are generating from either linear or general nonlinear models. We show that affine minimax rules are within an absolute constant of the minimax risk even among nonlinear rules, for a variety of source/target distributions.