  While deep learning demonstrates its strong ability to handle independent and
identically distributed (IID) data, it often suffers from out-of-distribution
(OoD) generalization, where the test data come from another distribution
(w.r.t. the training one). Designing a general OoD generalization framework to
a wide range of applications is challenging, mainly due to possible correlation
shift and diversity shift in the real world. Most of the previous approaches
can only solve one specific distribution shift, such as shift across domains or
the extrapolation of correlation. To address that, we propose DecAug, a novel
decomposed feature representation and semantic augmentation approach for OoD
generalization. DecAug disentangles the category-related and context-related
features. Category-related features contain causal information of the target
object, while context-related features describe the attributes, styles,
backgrounds, or scenes, causing distribution shifts between training and test
data. The decomposition is achieved by orthogonalizing the two gradients
(w.r.t. intermediate features) of losses for predicting category and context
labels. Furthermore, we perform gradient-based augmentation on context-related
features to improve the robustness of the learned representations. Experimental
results show that DecAug outperforms other state-of-the-art methods on various
OoD datasets, which is among the very few methods that can deal with different
types of OoD generalization challenges.
