  Approaches based on deep neural networks have achieved striking performance
when testing data and training data share similar distribution, but can
significantly fail otherwise. Therefore, eliminating the impact of distribution
shifts between training and testing data is crucial for building
performance-promising deep models. Conventional methods assume either the known
heterogeneity of training data (e.g. domain labels) or the approximately equal
capacities of different domains. In this paper, we consider a more challenging
case where neither of the above assumptions holds. We propose to address this
problem by removing the dependencies between features via learning weights for
training samples, which helps deep models get rid of spurious correlations and,
in turn, concentrate more on the true connection between discriminative
features and labels. Extensive experiments clearly demonstrate the
effectiveness of our method on multiple distribution generalization benchmarks
compared with state-of-the-art counterparts. Through extensive experiments on
distribution generalization benchmarks including PACS, VLCS, MNIST-M, and NICO,
we show the effectiveness of our method compared with state-of-the-art
counterparts.
