  Neural networks trained with class-imbalanced data are known to perform
poorly on minor classes of scarce training data. Several recent works attribute
this to over-fitting to minor classes. In this paper, we provide a novel
explanation of this issue. We found that a neural network tends to first
under-fit the minor classes by classifying most of their data into the major
classes in early training epochs. To correct these wrong predictions, the
neural network then must focus on pushing features of minor class data across
the decision boundaries between major and minor classes, leading to much larger
gradients for features of minor classes. We argue that such an under-fitting
phase over-emphasizes the competition between major and minor classes, hinders
the neural network from learning the discriminative knowledge that can be
generalized to test data, and eventually results in over-fitting. To address
this issue, we propose a novel learning strategy to equalize the training
progress across classes. We mix features of the major class data with those of
other data in a mini-batch, intentionally weakening their features to prevent a
neural network from fitting them first. We show that this strategy can largely
balance the training accuracy and feature gradients across classes, effectively
mitigating the under-fitting then over-fitting problem for minor class data. On
several benchmark datasets, our approach achieves the state-of-the-art
accuracy, especially for the challenging step-imbalanced cases.
