  In many machine learning applications, it is important for the model to
provide confidence scores that accurately captures its prediction uncertainty.
Although modern learning methods have achieved great success in predictive
accuracy, generating calibrated confidence scores remains a major challenge.
Mixup, a popular yet simple data augmentation technique based on taking convex
combinations of pairs of training examples, has been empirically found to
significantly improve confidence calibration across diverse applications.
However, when and how Mixup helps calibration is still mysterious. In this
paper, we theoretically prove that Mixup improves calibration in
\textit{high-dimensional} settings by investigating two natural data models on
classification and regression. Interestingly, the calibration benefit of Mixup
increases as the model capacity increases. We support our theories with
experiments on common architectures and data sets. In addition, we study how
Mixup improves calibration in semi-supervised learning. While incorporating
unlabeled data can sometimes make the model less calibrated, adding Mixup
training mitigates this issue and provably improves calibration. Our analysis
provides new insights and a framework to understand Mixup and calibration.
