Informed Mixing -- Improving Open Set Recognition with Deep Dynamic Data Augmentation

Authors: Jiawen Xu, Margret Keuper
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: open set recognition, model generalization, data augmentation

Abstract:
Conventionally trained image classifiers recently excel in accuracy across diverse tasks. One practical limitation is however that they assume all potential classes to be seen during training, i.e., they cannot tell "I don't know" when encountering an unknown class. 
Open set recognition (OSR), which solves this problem of detecting novel classes during inference, therefore remains an open problem and is receiving increasing attention. Thereby, a crucial challenge is to learn features that are relevant for unseen categories from given data, for which these features might not be discriminative. Previous work has shown that the introduction of self-supervised contrastive learning to supervised paradigms can support diverse feature learning and thereby benefit OSR. However, the diversity in contrastive learning is commonly introduced through crafted augmentation schemes. To improve upon this aspect and "optimize to learn" more diverse features, we propose GradMix, a data augmentation method that dynamically leverages gradient-based attribution maps of the model during training. The idea is to mask out the activated areas in previous epochs so that the models can pay attention to broader areas and learn to extract features beyond of what is most discriminative for every class. The resulting models are expected to learn more diverse features from the same data source and thus to improve in OSR and model generalization. Extensive experiments on open set recognition, close set classification, and out-of-distribution detection reveal that our method performs well on these tasks that can often outperform the state-of-the-art. GradMix is also beneficial for increasing robustness to common corruptions.
In self-supervised learning, GradMix can increase the accuracy of downstream linear classifiers compared with baselines, indicating its benefit for model generalization.

Decision: reject

Reviews and discussion:

Metareview: