  Real-world data is often unbalanced and long-tailed, but deep models struggle
to recognize rare classes in the presence of frequent classes. To address
unbalanced data, most studies try balancing the data, the loss, or the
classifier to reduce classification bias towards head classes. Far less
attention has been given to the latent representations learned with unbalanced
data. We show that the feature extractor part of deep networks suffers greatly
from this bias. We propose a new loss based on robustness theory, which
encourages the model to learn high-quality representations for both head and
tail classes. While the general form of the robustness loss may be hard to
compute, we further derive an easy-to-compute upper bound that can be minimized
efficiently. This procedure reduces representation bias towards head classes in
the feature space and achieves new SOTA results on CIFAR100-LT, ImageNet-LT,
and iNaturalist long-tail benchmarks. We find that training with robustness
increases recognition accuracy of tail classes while largely maintaining the
accuracy of head classes. The new robustness loss can be combined with various
classifier balancing techniques and can be applied to representations at
several layers of the deep model.
