For classification problems with significant class imbalance, subsampling can reduce computational costs at the price of inflated variance in estimating model parameters. We propose a method for subsampling efficiently for logistic regression by adjusting the class balance locally in feature space via an accept-reject scheme. Our method generalizes standard case-control sampling, using a pilot estimate to preferentially select examples whose responses are conditionally rare given their features. The biased subsampling is corrected by a post-hoc analytic adjustment to the parameters. The method is simple and requires one parallelizable scan over the full data set. Unlike standard case-control sampling, which is inconsistent under model misspecification for the population coefficients, our method is consistent provided that the pilot estimate is. Although in severely imbalanced data sets the chosen subsample may contain a tiny fraction of the full data set, we show that under correct specification and with a consistent, independent pilot estimate, the subsampled estimate has exactly twice the asymptotic variance of the full-sample MLE. Moreover, this factor improves to 1 + 1 c if we multiply the baseline acceptance probabilities by c > 1 (and weight points with acceptance probability greater than 1), taking roughly 1+c 2 times as many data points into the subsample. Experiments on simulated and real data show that our method can substantially outperform standard case-control subsampling.
Outside of linear models, local case-control sampling REF provides an effective sampling method for logistic regression based on the residuals as well.
Local case-control sampling: Efficient subsampling in imbalanced data sets
Annals of Statistics 2014, Vol. 42, No. 5, 1693-1724
Mathematics Engineering Medicine