Learning a Non-Redundant Collection of Classifiers
Authors: Daniel Pace
Alessandra Russo
Murray Shanahan
Venue:  (2021)

Abstract: Supervised learning models constructed under the i.i.d. assumption have often been shown to exploit spurious or brittle predictive signals instead of more robust ones present in the training data. Inspired by Quality-Diversity algorithms, in this work we train a collection of classifiers to learn distinct solutions to a classification problem, with the goal of learning to exploit a variety of predictive signals present in the training data. We propose an information-theoretic measure of model diversity based on minimizing an estimate of conditional total correlation of final layer representations across models given the label. We consider datasets with synthetically injected spurious correlations and evaluate our framework's ability to rapidly adapt to a change in distribution that destroys the spurious correlation. We compare our method to a variety of baselines under this evaluation protocol, showing that it is competitive with other approaches while being more successful at isolating distinct signals. We also show that our model is competitive with Invariant Risk Minimization under this evaluation protocol without requiring access to the environment information required by IRM to discriminate between spurious and robust signals.

Under review as a conference paper at ICLR 2021
LEARNING A NON-REDUNDANT COLLECTION OF CLASSIFIERS
Anonymous authors Paper under double-blind review
ABSTRACT
Supervised learning models constructed under the i.i.d. assumption have often been shown to exploit spurious or brittle predictive signals instead of more robust ones present in the training data. Inspired by Quality-Diversity algorithms, in this work we train a collection of classifiers to learn distinct solutions to a classification problem, with the goal of learning to exploit a variety of predictive signals present in the training data. We propose an information- -hheorrticcc measure of model di- versity based on minimizing an estimate of conditional total correlation of final layer representations across models given the label. We consider datasets with synthetically injected spurious correlations and evaluate our framework's ability rapidly adapt change in distribution that destroys the spurious correlation. We compare our method to a variety of baselines under this evaluation protocol, showing that it is competitive with other approaches while being more successful at isolating distinct signals. We also show that our model competitive with In- variant Risk Minimization (IRM) under this evaluation protocol without requiring access to the environment information required by IRM to discriminate between spurious and robust signals.
INTRODUCTION
The Empirical Risk Minimization (ERM) principle (Vapnik 2013), which underpins many machine learning models, is built on the assumption that training and testing samples are drawn i.i.d. from some hypothetical distribution. It has been demonstrated that certain violations of this assumption (in conjunction with potential misalignments between the formulation of the learning objectives and the underlying task interest) lead to models that exploit spurious or brittle correlations in the training data. Examples include learning to exploit image backgrounds instead of objects in the foreground due to data biases (such as using grassy backgrounds to predict the presence cows (Beery al., 2018)), using textural opposed to shape information classify objects (Geirhos et al. 2018). and using signals not robust to small adversarial perturbations (Ilyas et al.) 2019).
Implicit in work that attempts address these phenomena is the assumption that more robust pre- dictive signals are indeed present in the training data, even if for various reasons current models do have the tendency leverage them.
In this work, drawing inspiration from Quality-Diversity algorithms (Pugh et al. 2016 which seek to construct a collection of high- performinf. diverse solutions to a task we aim learn a collection of models, each incentivized to find a distinct, nigh-performing solution to given supervised learning problem from a fixed training set. Informally, our motivation is that sufficiently large collection of such distinct models would exploit robust signals present in the training data in addition to the brittle signals that current models tend to exploit. Thus, given the representations computed by such collection, it may be possible to rapidly adapt to test-time shifts in distribution that destroy the predictive power of brittle features.
Addressing this problem hinges on defining and enforcing appropriate measure of model diver- sity. To this end, we make the following contributions:
We propose and motivate a novel measure of model diversity based on conditional total correlatio (across models) of final layer representations given the label. Informally, this

---

Under review as a conference paper at ICLR 2021
Comminal Ine
Loo,
Loa,
Loee
conditeed beld arokit
gree
dlos
Podake,
Redarko
Zpy,
BAK
Tipe
Figure 1: Graphical representation of our framework. We train multiple models to individually min- imize supervised loss while simultaneously minimizing conditional total correlation of final layer representations given the label.
incentivizes models to each learn non-redundam (in an information-theoree sense) way solve the given task.
We estimate this measure using proxy variational estimator computed using samples from the conditional joint and marginal distributions of final layer representations across models. We train a collection of models be accordingly diverse, alternating between training variational critic to maximize the variational estimator and minimizing a weighted sum of the classification losses across models and the variational estimator.
We empirically validate this framework by training datasets with synthetically injected spurious correlations. We demonstrate that our framework is able to learn a collection of representations which, under linear fine-tuning protocol, are competitive with baselines in being able to rapidly adapt to shift in distribution which destroys the spurious corre- lations. We show that our framework performs favourably relative baselines in being able to isolate distinct predictive signals in different models in the collection, as result being able to make accurate test-time predictions without fine-tuning. We also compare our approach to the Invariant Risk Minimization (Arjovsky et al., 2019) framework which leverages information from multiple training environments to identify signals robust to variations across environments. We show that our approach is able to exploit the robust signals without requiring the metadata needed by IRM to discriminate between spurious and robust signals.
TOTAL CORRELATION
Total correlation (Watanabe) 1960) is a multivariate extension mutual information. The total correlation for random variables XI,... Xn is given by (where KL denotes the Kullback-Leibler divergence):
TC X,,) = KL[p(X,, Xx) PXX)]
For n = this is equivalent to mutual information. We can interpret this as a measure of the amount of redundancy across the X,'s. A total correlation of zero corresponds to the X,'s being mutually independent.
Given classification problem predicting label Y from X, consider the conditional total corre- lation of a collection of vector-valued representations h (X),... ..., h, (X) given Y for differentiable functions h, defined
2

---

Under review as a conference paper at ICLR 2021
(x) = ( (m. II ((((()))
(2)
where is drawn from the conditional distribution given Y.
METHOD
Consider collection of differentiable representation function and linear classifier pairs hel where we denote the vector parameters all these functions as 0 E R", such that for classification problem predicting Y from X, we compute logits e₁ o (X). That is, the h4 computes vector-valued representation of the input X, and e; transforms this into scalar classification prediction.
We target the following optimization problem:
min
Σ Ex, ())
(3)
subject to
(., <
for some >> 0, where is some loss function.
That we seek minimize the risk for each model while keeping the conditional total correlation of final layer representations given the label below some threshold.
We justify the choice of conditional total correlation in place of the unconditional version by ap- pealing to the following argument: if we consider the case where the collection consists of just two models, total correlation equivalent to mutual information. Minimizing mutual information be- tween the two representations is equívalent to enforcing independence. However, it is not possible for the representations to independent while requiring that both are sufficient accurately predict the label. On the other hand, provided that the representations mon cause, they will be conditionally independent given the label. Thus, minimizing the conditional total correlation should provide more useful learning signal.
not dependent due to some com-
In practice, we attempt to solve the following unconstrained problem in place of (3:
min P
Σ Ex.y (TC())))))))))))))))))))))),,,,, (x)))))))))
(4)
where mean).
> hyperparameter and we estimate the risk using the empirical risk (i.e. sample
3.1 TOTAL CORRELATION ESTIMATION
In initial experiments with the two-model case (i.e. where total correlation is equivalent to mutual information), the InfoNCE objective of Oord et al. (2018), shown compute lower bound of the mutual information (Poole et al. 2019), proved effective in computing useful gradient estimates for
the latter term in (4) Consequently, choose construct proxy estimator for the total correlation based on InfoNCE.
The InfoNCE objective for random variables U and V is given by:
INCE ==
log
(5)
K 1=1
1 E==ı efluney)
K
where (u,, v.) are sampled from the joint distribution of U and V. K is the batch size, and f is differentiable, scaler-valued function, commonly referred to as a 'critic' (Poole et al.] 2019). The parameters of f are optimized to maximize JNCE-
3

---

Under review as a conference paper at ICLR 2021
By analogy, our proposed estimator TC(X,, XN) is given by:
K
1 TC(X Xx) = K M 2j=1
log
(6)
where (fe1)..., Fin) are sampled from the joint distribution of XI,..,, Xns K the batch size, is random index of the batch dimension, M is hyperparameter,
Thi Uniform([1,.... and is a critic. The permuted mæ.j indices are intended to yield (Isi.lls... Tx samples drawn approximately from the marginals, derived from the observations in the batch. Similarly we optimize the parameters of maximize TC(X1,..... XN).
We interpret (6) as a ratio of scores between samples from the joint distribution of the X,'s and samples approximately drawn from the marginals, such that solution to the variational objective resulting in large ratio corresponds to being able to easily discriminate between samples from the joint and marginals (i.e. a proxy for large total correlation) and conversely when the ratio is small.
We compute the conditional case as follows:
K Σ 1=1
TC XNlY) = Ey
log
K
M 2==
Ma pl
M
where (Ia1, ..., Fin) are sampled from the conditional joint distribution of X1, Xn given Y. We perform the conditioning step by splitting batch by label, separately computing the estimator for each group, and finally computing weighted average. The weights are estimated using the frequen- cies of each label in the batch.
3.2 MINIMAX TRAINING OBJECTIVE
We replace the latter term in with our variational estimator to obtain:
Exyll(c h4(X), max
where @ are the parameters of the variational critic f, and is hyperparameter.
In practice we use gradient-base based methods to approximately optimize ( by alternating between optimizing & to maximize the variational estimator (giving us local total correlation estimate), and optimizing 0 to minimize the sum of all terms (for fixed d).
EXPERIMENTS
The main hypothesis that we wish test is that our method yields a collection of models that exploit and isolate distinct predictive signals in the training data. Our strategy for testing this hypothesis is consider training distribution with synthetically injected spurious correlations as well test distribution in which the spurious correlation has been destroyed. We then measure our collection of models's ability to rapidly adapt to the new distribution given small amounts of data, relative to number of baselines. The spurious correlations are constructed such that they are easier to exploit than the robust signals.
The central challenge in testing our main hypothesis is in constructing reliable method for mea- suring the extent to which distinct predictive signals have been extracted and isolated in the outputs of trained models. Our evaluation protocol which requires access to small amount of data from a shifted distribution to construct a model selection strategy emerges from a lack alternatives. precludes the use of various real-world benchmarks known or hypothesized to exhibit variety of predictive signals, spurious or otherwise: we simply do not currently have access to appropriate test sets.

---

Under review as a conference paper at ICLR 2021
Looking the related problem of domain generalization for alternate evaluation protocols / model selection strategies, we find approaches either similarly based on oracle test sets or using held-out training domains (Gulrajani & Lopez-Pazi 2020). Here, deliberately consider problem setting which does not assume access to training data from multiple distinct but related domains, so we cannot use model selection strategies based on the latter. Indeed, the underlying motivation for the problem setting we consider is that perhaps we can get away with very little additional information in order isolate distinct predictive signals in training set. central priority for future work is to explore the possibility of model selection strategies that do not require access to data from a shifted evaluation distribution. In this paper, we focus on empirical evaluation of our approach relative to baselines under our proposed fast adaptation evaluation protocols.
We consider variations on the Colored MNIST (C-MNIST) task of Arjovsky et al. (019). This task consists of classifying MNIST digits into one of two classes (0-4 or 5-9). However, the true label is flipped with probability 0.25 and an additional binary signal (colour) is constructed from this corrupted label such that it is more strongly correlated with the corrupted label than the digit. As demonstrated in Arjovsky et al. (201), a model trained with ERM will overwhelmingly tend to exploit the colour information. In Arjovsky al. (2019), the goal was to demonstrate ability to learn to exploit the digit information given the task of identifying a stable signal across environ- ments with varying colour-label correlation. To test our framework, we assume no such access to multiple environments, and collapse the two training environments in the C-MNIST benchmark task of Arjovsky et al. (2119) into one training set The results for this task are collected in Table [
We also consider a modified dataset which introduces an obstacle to extracting mutually exclusive representations of digit and colour. Specifically, we use functions of new binary 'common cause' variable to both rotate the digits and perturb the colour signal, such that minimizing total correlation will depend on an ability to discard the common information. This is to test our framework's 's ability to still extract the predictive signals in the presence of such an obstacle to minimizing conditional total correlation. We refer to this dataset as Rotated Colored MNIST (RC-MNIST). The results for this task are collected in Table 2
We consider a third variation in which more than two predictive signals are present in the training data. Specifically, we construct another binary 'colour' signal from the corrupted label and add it to the input such that new signal can be exploited on the training set for a maximal accuracy of 0.75. We refer to this dataset as Two-Colored MNIST (TC-MNIST). The results for this task are collected in Table
We similarly construct the same three benchmarks using the Fashion-MNIST dataset (Xiao et al. 2017].
We consider the following baselines:
Single ERM classifier
Single ERM classifier with DeCov (Cogswell et al. 2015) regularization. We use this to penalize large non-diagonal entries of the covariance matrix of the output of the represen- tation function
Single ERM classifier with Mutual Angles (MA) (Xie et al. 2015) regularization. We use this to penalize the mean angle between weight vectors corresponding to the output units of the representation function, minus the variance of the angles
Single ERM classifier with a Uniform Figenvalue Regularizer (UER) (Xie al. 2017). We use this encourage the weight matrix computing the output of the representation function to have uniform eigenvalues
Ensemble of ERM classifiers
Collection of ERM classifiers trained to maximize pairwise cosine distances between final layer representations (for a given input), adapted from Yu et al. (2011)
'The two environments are constructed such that the colour signal is computed by flipping the corrupted label with probability 0.1 and 0.2 respectively. Thus, collapsing the two environments into one means that we can at best achieve accuracy of 0.85 on the training set by exploiting the colour signal, versus the maximal accuracy of 0.75 by exploiting the digit.
5

---

Under review as a conference paper at ICLR 2021
We also include results for the following methods for reference (these methods require access to information that the baselines do not require):
Importance--eighhed ERM, with optimal weights (requires knowledge of the test distribu-
tion; optimal importance weights derived in Appendix B
IRM (needs access to additional environment metadata)
Implementation details can be found in Appendix A
4.1 EVALUATION
As discussed above, we evaluate our approach by training relatively simple models on the outputs of the frozen trained models adapt shift in distribution that destroys spurious correlations in the training distribution. We consider the following adaptation models:
Best: choose the single best performing model from the collection, and use its frozen
predictions (0 parameters)
Linear:
train logistic regression model (with L regularization) the concatenated frozen final layer representations (i.e. the outputs of each representation function h,) from each model (n Drepr parameters for models and representation dimension Drepr)
These adaptation models are trained 500 observations (1% the size of training set) from the new distribution. We tune hyperparameters using validation set drawn from the new test distribution and report results on a hidden test set corresponding maximal validation scores achieved. We treat the number of models in our framework as hyperparameter and present results for the best setting. We tune hyperparameters for IRM using the approach used to produce the results in Arjovsky et al.) (201)). Results for each experimental condition are averaged over 10 runs, for which we report the standard deviation. For baselines requiring multiple models, we report results for 10 models as adding further models did not improve results. For single-model baselines, the 'best' protocol com- putes the test accuracy for that single model, while the 'linear' protocol computes the test accuracy for linear classifier trained on the frozen output of the single representation function.
4.2 INTERPRETATION OF RESULTS
For the C-MNIST benchmark (Table D. our approach outperforms all baselines under the 'best' evaluation protocol. Results under this evaluation protocol indicate that the approach has success- fully isolated the 'digit' signal into single model to the extent that, given this model, no further fine-tuning is required to make accurate test time predictions. Across all experimental conditions, approach also produces single model in the collection exploiting the spurious colour signal, idenced by the corresponding model's maximal training set performance and chance performance on the test distribution. The latter is further evidence of our framework successfully isolating predictive signals into distinct models.
As expected, the single ERM classifier baseline fails across all experimental conditions. Single model baselines with regularization that diversifies the components of the single model's represen- tation (i.e. DeCov, MA, and UER) often outperform the ERM baseline but are not competitive with our approach. Perhaps surprisingly. the ERM baselines with multiple models as well as the 'pairwise cosine' baseline are competitive with our approach under the 'linear' protocol, even though no single frozen model from each respective collection can make accurate test-time predictions (as evidenced by the results under the 'best' protocol). This suggests that the superficial notions of model diversity used in these cases are sufficient, given a large enough number of models, to allow for each model to retain different, small amounts of information about the 'digit' signal such that, taken together, all the small sources of information are sufficient to make accurate test-time predictions. While acknowledge that this may just be an artefact of the benchmarks considered in this work, we claim that exploring this phenomenon further in more diverse settings is interesting direction for future work.
The conditional estimator outperforms the unconditional version across all experimental conditions.

---

Under review as a conference paper at ICLR 2021
Table I: Test accuracies for the C-MNIST task. Test data is drawn from distribution in which the
colour signal has been destroyed.
Test aceuracy (s.e.)
Test accuracy (s.e.)
MNIST
Fashion-
Linear
Best
Linear
Best
Optimal
75
75
75
75
Optimal IWERM
70.3 (0.6)
70.7 (0.5)
70.2 (0.7)
70.5 (0.5)
IRM (requires env. info.)
(1.7
68.6 (1.2)
69.0 (0.5)
68.4 (0.6)
Ours (conditional, 2 models)
69.8 (0.4)
69.8 (0.4)
70.0 (0.7)
70.2
Ours (unconditional, models)
66.7 (1.0)
66.6 (0.9)
68.4
(0.6)
ERM (single model)
56.8 (2.6)
50.3 (0.7)
61.4
51.4 (0.5)
DeCov (single model)
(3.4)
54.5 (1.1)
57.7
(0.8)
MA (single model)
63.7
51.0 (0.5)
(0.2
50.7 (0.5)
UER (single model)
57.3 (4.7)
51.2 (0.5)
57.1 (2.3)
50.7 (0.6)
ERM (10 modcls, ensemble)
67.4
51.0 (0.8)
(1.6
50.5 (0.6)
Pairwise cosine (10 models)
70.5 (0.8)
56.4 (0.8)
69.5 (0.5)
54.4 (1.1)
Table 2: Test accuracies for the RC-MNIST task. Test data is drawn from distribution in which the colour signal has been destroyed. Method
Test accuracy (s.e.)
Test accuracy (s.e.) Fashion-MNIS:
MNIST
Linear
Best
Linear
Best
Optimal
75
75
75
75
Optimal IWERM
72.0 (0.6)
72.2 (0.5)
70.9 (0.6)
70.9 (0.4)
IRM (requires env. info.)
70.2
70.3 (0.5)
69.2 (0.4)
69.5 (0.5)
Ours (conditional, 2 models)
(0.7)
71.4 (0.8)
70.5
70.4 (0.7)
Ours (unconditional, models)
(2.6
61.3)
68.1
(8.9)
ERM (single model)
(7.1
50.4
52.0 (1.7
DeCov (single model)
(3.2)
(1.6)
62.5
51.1
MA (single model)
(1.4)
(0.4
(1.2)
(0.7)
UER (single model)
70.9
69.5 (1.00
51.2
ERM (10 models, ensemble)
71.5
52.5
70.1 (0.4)
51.3 (0.9)
Pairwise cosine (10 models)
71.8
53.2 (0.8)
70.7 (0.4)
51.1 (0.6)
Table Test accuracies for the TC-MNIST task. Test data is drawn from distribution in which the two colour signals have been destroyed. Method
Test accuracy (s.c.)
Test accuracy (s.e.) MaSHion-MNIST
MNIST
Linear
Best
Linear
Best
Optimal
75
75
75
75
Optimal IWERM
69.2 (0.8)
61.7 (2.0)
68.9 (2.1)
61.3 (1.9)
IRM (requires info.)
67.5 (1.3)
62.8 (0.8)
68.8 (0.7)
65.5 (0.7)
Ours (conditional, 3 models)
(0.7
67.8 (0.8)
67.9 (0.8)
(2.6)
Ours (unconditional, models)
(1.4)
52.3)
66.8 (0.7)
(3.4)
ERM (single model)
(3.7
56.3 (0.5)
60.8 (2.8)
59.4 (0.6)
DeCov (single model)
(2.6
59.1 (0.9)
63.4
59.6
MA (single model)
58.8
58.1 (0.7)
60.6
59.3
UER (single model)
62.6(1.9)
58.2 (0.9)
65.7 (3.5)
(0.5)
ERM (10 models, ensemble)
68.6 (1.6)
58.5 (1.1)
69.0 (0.9)
58.7 (1.0)
Pairwise cosine (10 models)
68.2
59.2 (0.7)
67.0
59.7 (0.6)

---

Under review as a conference paper at ICLR 2021
We observe similar pattern of results for the RC-MNIST benchmarks (Table 2). For our frame- work, this is an indication that the framework is robust to the obstacle to extracting conditionally independent predictive signals entailed in this benchmark.
A similar pattern of results emerges for the TC-MNIST benchmarks (Table 3 with some excep- tions: relative to the other benchmarks, performance drops for the optimally serportance-wiightee baseline. We have no explanation for this other than to defer to work pointing out potential issues with importance weighting when training overparameterized deep neural networks (Byrd & Lipton 2019).
We note that our approach is competitive with IRM in isolating the causally correct predictive signal across all experimental conditions, despite having no access to the environment metadata needed by the former to identify invariant predictive signals.
5 RELATED WORK
Our work shares motivation with Quality-Diversity (QD) algorithms (Pugh et al. 2016) from the evolutionary computation literature, which also attempt to find diverse, high- perfforrforiin solutions to task. This is typically done by partitioning the search space using predefined behaviour de- scriptor and finding high-pen segh-performing solutions within each behavioural niche (Mouret & Clune, 2015 Cully & Demiris 2017). We do not explicitly partition the search space but instead enforce diversity through an information theoretic objective. A central challenge in directly applying exist- ing QD algorithms the problems considered in this work is the difficulty in defining an appropriate behaviour descriptor. An interesting direction for future work would be to investigate the possibility of learning suitable behaviour descriptors from data, as explored in the context of control problems in Cully (019))
Ensemble-based methods (Polikar 2006 Rokach| 2010) train collections of models using various
techniques (e.g. subsampling of observations and features (Breiman, 2001}. boosting (Friedman, 2001]. penalizing the correlation of model predictions (Liu & Yao 1999)) enforce some notion of model diversity such that aggregate predictions outperform single model predictions. While we have included ensemble baselines, none of the aforementioned approaches are appropriate for the setting considered in this paper.
Various approaches, which we consider as baselines, use regularization intended to diversify some aspect of the model. Yu et al.) (2011) minimize cosine similarity between weight vectors of output components. Cogswell et al. (2015) penalize the magnitude of non-diagonal entries of the covariance matrix of activations, Xie et al. (2015) enforce large mean angles between the weight vectors, minus the variance of the angles. Xie et al.] (2017) enforce eigenvalues of the matrix of weight vectors yield uncorrelated and evenly useful components. We note that
to be uniform, hypothesized diversifying the components of single model without requiring that each are separately sufficient produce good predictions is susceptible to a scenario in which learned components are distinct but only some are predictive. Our approach precludes this scenario by construction.
A number of works (Kim & Mnih 2018 Chen et al. 2018 Poole et al. 2019 Gao et al. 2019) use total correlation based objectives to enforce factorized representations in generative models. Similar to our approach, FactorVAE (Kim & Mnih 2018) uses adversarial training to minimize an estima- tor of total correlation. In contrast to these approaches, we target total correlation for a collection of bigh-dimensional vector-valued representations while also leveraging label information. A pos- sibility for future work is to impose probabilistic structure on the representations in our proposed method that may enable the use of the upper bound on total correlation presented in Poole et al. 2011)), perhaps alleviating the need for adversarial training.
Independent component analysis (ICA) (Comon) 1994) addresses the problem of separating a signal into distinct components using independence based criterion. This is done in unsupervised setting and making different assumptions than we do. Brakel & Bengio (017)) use an adversarial training method similar to ours in the context of non-linear ICA.
Other works (kook& & Adams 2012 Mariet & Sra, 2015)] Pang et al. 2019 2020) use constructs derived from determinantal point processes (Kulesza & Taskar 2012) com-
et al.
8

---

Under review as a conference paper at ICLR 2021
pute diversity measures. These are not directly applicable setting, but exploring these ap- proaches in our context is an interesting direction for future work.
There is large body of work attempting to account for test-time distribution shift (Quionero Candela et al., 2009). The problems considered in this paper are an example of covariate shift, in which the marginal distribution of the label remains unchanged at test time. Approaches from the literature to address covariate shift are based on discarding input features, reweighting training observations, or domain adversarial training (Ganin al.) 2016). Approaches based discarding input features are not readily applicable to our context considering high-dimensional unstructured inputs. We have included results for an optimally importance-weighted weighted baseline and have refrained from exploring baselines which estimate the weights from data due to the challenges entailed in do- ing so in a deep learning context. We were unable to train domain adversarial approach to produce good results under the evaluation protocols considered in this paper; we hypothesize that this may be due the scarcity of adaptation data under our evaluation protocols. We have deliberately refrained from considered standard image domain adaptation benchmarks as these lie outside of the intended scope of our method.
Some works from the GAN literature (Hoang et al. 2018 Ghosh et al. 2018) attempt prevent mode collapse by training a collection of generators to produce easily discriminable samples. Ex- periments adapting such approaches to our context fail completely and we have excluded corre- sponding results. This is not surprising: in our setting it is easy for learned representations to be easily discriminable while still representing the same information content.
Other works drawing from ideas in the causal inference literature attempt to yield models that generalize better under certain violations of the ERM assumptions. Invariant Risk Minimization (Arjovsky et al.] 2019) assumes access to data from multiple environments and attempts to find a representation such that the same classifier is simultaneously optimal all environments (as do sub- sequent works addressing the same similar problem (Ahuja et al. [0200] Krueger et al. (2200). The problem of causal discovery (Pearl (009), Peters et al.) (217]) consists trying identily underly- ing causal structure from data such that the tools of causal inference can then be used to compute predictions that are valid under certain shifts in distribution.
6 CONCLUSION
We have introduced new method for training collection of high-performing supervised learning models each incentivized to learn distinct way to solve the task. We have empirically demonstrated how minimizing our proposed measure of model diversity in addition to the empirical risk suffices to yield a collection of models exploiting distinct predictive signals present in benchmark tasks.
Perhaps surprisingly, find that simple notions of model diversity considered in certain baselines are sufficient to produce competitive results under a linear fine-tuning evaluation protocol. While this may just be artefact of the benchmarks considered in this work, we claim that such strat- egy (i.e. simple, scalable measures of model diversity coupled with linear fine-tuning given small amounts of new data) is interesting avenue explore in future work.
We have focused on evaluating our approach on datasets which we have sufficient control over to enable the testing of basic hypotheses about our proposed framework. To the best of our knowl- edge, there are currently no other more complex, natural datasets for us to evaluate our approach on. Furthermore, as in (Arjovsky et al. (2019), we require access evaluation data from a different distribution than the training data in order to tune hyperparameters. Rather than seeing this as a disadvantage of our approach, we see it as further evidence that confronting the problem of general- ization in a data-driven way may call for the development of new kinds of datasets and benchmarks that are better suited to testing robustness to violations of the i.i.d. assumption on which the empirical risk minimization principle is founded.
REFERENCES
Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar, Invariant risk min- imization games. arXiv preprint arXiv:2002.04692 2020.

---

Under review as a conference paper at ICLR 2021
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv: 1tXXiv1907702892 2019.
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European Conference Computer Vision (ECCV), pp. 456-473, 2018.
Philemon Brakel and Yoshua Bengio. Learning independent features with adversarial nets for non- linear ica. arXiv preprint arXiv:1710.050500, 2017.
Leo Breiman. Random forests. Machine learning, 45(1):5-32, 2001.
Jonathon Byrd and Zachary Lipton. What is the effect of importance weighting in deep learning? In International Conference on Machine Learning, pp. 872-881,
Tian Qi Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. Isolating sources of disentan- glement in variational autoencoders. In Advances Neural Information Processing Systems, pp- 2610-2620, 2018.
Michael Cogswell, Faruk Ahmed, Ross Girshick, Larry Zitnick, and Dhruv Batra. Reducing overfit- ting deep networks by decorrelating representations. arXiv preprint arXiv:1511.06666 2015.
Pierre Comon. Independent component analysis, a new concept? Signal processing. 36(3):287-314, 1994.
Antoine Cully. Autonomous skill discovery with quality and unsupervised descriptors. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 81-89, 2019.
Antoine Cully and Yiannis Demiris. Quality and diversity optimization: unifying modular frame- work. IEEE Transactions on Evolutionary Computation, 22(2):245-259, 2017.
Jerome H Friedman. Greedy function approximation: gradient boosting machine. Annals of
statistics, pp. 1189-1232, 2001.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural net- works. The Journal of Machine Learning Research, 17(1):2096-23 2030, 2016.
Shuyang Gao, Rob Brekelmans, Greg Ver Steeg, and Aram Galstyan. Auto-encoding total correla- tion explanation. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1157-1166, 2019.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cans are biased towards texture; increasing shape bias im- proves accuracy and robustness. arXiv preprint arXiv: /81 12231, 2018
Arnab Ghosh, Viveka Kulharia, Vinay P Namboodiri, Philip HS Torr, and Puneet K Dokania. Multi- agent diverse generative adversarial networks. In Proceedings of the IEEE conference com- puter vision and pattern recognition, pp. 8513-8521, 2018
Gauthier Gidel, Reyhane Askari Hemmat, Mohammad Pezeshki, Rémi Le Priol, Gabriel Huang, Si- mon Lacoste-Julien, and Ioannis Mitliagkas. Negative momentum for improved game dynamics. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1802-1811, 2019.
Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. arXiv preprint a3X10:2007.01434, 2020.
Quan Hoang, Tu Dinh Nguyen, Trung Le, and Dinh Phung. Mgan: Training generative adversarial nets with multiple generators. In International Conference Learning Representations, 2018.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are fea- Processing Systems 32. pp. 125-136.
tures. In Advances in Neural Information
Curran Associates. Inc.. 2019.
URL http:///aapperrnissscccccaaaaaaa
#307aadversaria wew.erreeeeeeaaaeeeeeeeee-eeeeee -etes--r--etttbuuueeeee theerrre----uuuu-te------e--eee org.apapee..............ooooooon augss
10

---

Under review as a conference paper at ICLR 2021
Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In International Conference Machine Learning, pp. 2649-266666 2658, 2018
David Krueger, Ethan Caballero, Henrik Joern-Jacobsen Amy Zhang, Jonathan Binas, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrapolation. arXiv preprint wrXiv:2003.00688 2020.
Alex Kulesza and Ben Taskar. Determinantal point processes for machine learning. arXiv preprint arXiv: 1207:62277.683 2012.
James T Kwok and Ryan P Adams. Priors for diversity in generative latent variable models. In Advances in Neural Information Processing Systems, pp. 2996-3004, 2012.
Yong Liu and Xin Yao. Ensemble learning via negative correlation. Neural networks, 12(10): 1404, 1999.
Zelda Mariet and Suvrit Sra. Diversity networks: Neural network compression using determinantal point processes. arXiv preprint 2oXt:,1511.0500,,201100 2015.
lean-Bap Mouret and Jeff Clune. Illuminating search spaces by mapping elites. arXiv preprint arXiv:1504.04909 2015.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03744 2018
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. International Conference on Machine Learning, pp. 4970-4979, 2019.
Jack Parker-Ho Holder, Aldo Pacchiano, Krzysztof Choromanski, and Stephen Roberts. Effective di- versity in population-bas based reinforcement learning. arXiv preprint arXiv:2002.00632, 2020.
Parker-Holder,
Judea Pearl. Causality. Cambridge university press, 2009.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. MIT press, 2017.
Robi Polikar. Ensemble based systems in decision making. IEEE Circuits and systems magazine, 6 (3):21-55, 2006.
Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On variational bounds of mutual information. In International Conference on Machine Learning, pp. 5171- 5180, 2019.
Justin K Pugh, Lisa B Soros, and Kenneth O Stanley. Quality diversity: A new frontier for evolu- tionary computation. Frontiers Robotics and AI, 3:40, 2016.
Joaquin Quionero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dataset shift in machine learning. The MIT Press, 2009.
Lior Rokach. Ensemble-based classifiers. Artificial Intelligence Review, 33(1-2):1-39, 2010.
Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 2013.
Satosi Watanabe. Information thcoretical analysis of multivariate correlation. IBM Journal of re- search and development, 4(1):66 82, 1960.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mmin a novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint
2017.
Pengtao Xie, Yuntian Deng, and Eric Xing. Diversifying restricted boltzmann machine for document modeling. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1315-1324, 2015.

---

Under review as a conference paper at ICLR 2021
Pengtao Xie, Aarti Singh, and Eric P Xing. Uncorrelation and evenness: new diversity-rromoting regularizer. In International Conference on Machine Learning. pp. 3811-3820, 2017.
Yang Yu, Yu-Feng Li, and Zhi-Hua Zhou. Diversity regularized machine. In Twenty-Second Inter- national Joint Conference on Artificial Intelligence, 2011.
A IMPLEMENTATION DETAILS
In the notation of Section 3 we use multilayer perceptrons (MIP) as representation functions hę with two hidden layers of size 128 and 64 respectively, 32 output units, and ReLu activations. We the cross-entropy as classification loss. Our single ERM classifier baseline is equivalent to a single model from the collection of models in our framework. The baselines calling for multiple models use multiple instantiations of this same model with different initial weights, The variational critic an MLP with hidden layers of size 256 computing a scalar output from an input consisting of a concatenation of all representations computed by the h2 ss. We use Leaky ReLus with a slope of 0.2 as activations in the critic as this appeared to speed up training experiments. We use 3 = 10 throughout. For the 'pairwise cosine', we use coefficient of 0.1 for the loss function term computing the average pairwise cosine distance between final layer representations for a given input.
We use RMSProp with learning rate of 10-5 as optimizer for the models in the collection as well as the critic. We note that not using a momentum-based optimizer appears to be crucial for stable training, following the trend of decreasing momentum values in the GAN literature (Gidel jet al., 2019). We note that training appears to be further stabilized by normalizing representations computed by the h, 's to have unit L norm prior to being input into the critic.
We use a batch size of 256 for training both the models and the critic, as well as M = 64 in Equations (6) and 7 To train the simple classifiers, use 500 training and validation observations from the novel test distribution, the latter used to tune the hyperparameters (if any) of the newly trained classifiers. We report accuracies computed for the remaining 9000 test observations. We train for 250 epochs and report test accuracies corresponding to maximal validation accuracies. We make the following exceptions for the TC-MNIST task: we train for 1000 epochs and alternate between 5 steps of training the critic and 1 step of training the collection of models. This would appear to be necessary for obtaining useful total correlation gradients given the relatively more challenging variational problem that needs to be solved in this case.
These hyperparameter settings were determined from manual tuning of our approach on the C- MNIST benchmark solely for the two-model case using the conditional TC estimator, and reused across all other experimental conditions. Minimal manual tuning of the regularization coefficient for the 'linear' evaluation protocol was subsequently performed for each experimental condition with distinct number of models in the collection. Coefficients for any regularization terms in basclines were similarly tuned.
Hyperparameters for IRM were chosen using the tuning method used to produce the results in Arjovsky et al. (2019), owing the similarity of the datasets consid- ered. Hyperparameters were chosen using random search with 50 trials, sampling values as in https://github. com/ http:////acbooeeeeeee////aaaaaaa siiiattiiiiiiiiiiaaiaai zation/ http:///ooo///oooddmmdddddmd////
hatpeeappperrrrrsseeeeee sh The best model was chosen on the basis of the mean final test accuracy (over 10 runs) under each of our evaluation protocols.
http://ppooouuc_ppppp_ppp
A.1 DATA SYNTIESSIS
C-MNIST
The Colored MNIST dataset constructed as in Arjovsky et al. (2019) MNIST images are sub- sampled to 14x14 size, images are assigned to one of two classes (depending whether the MNIST label is 0-4 or 5-9), and this binary class label is corrupted by flipping with probability 0.25. Subse- quently, binary colour signal is computed from the corrupted label by flipping it with probability
Based
on
code
from:
Invari antRi skminininnnn
12

---

Under review as a conference paper at ICLR 2021
Pe. where is environment index. The colour signal is represented in the final dataset by append- ing a new channel to the grayscale image. The new channel is set to copy the grayscale image if the color signal is equal to I, and zeroed out otherwise. The final dataset consists of these two- channel 'images' and the corrupted label. Two training environment with Pe, = 0.2 and Peg = 0.1 used to train the IRM baseline. Our models and the ERM baseline trained by collapsing these two training environments into one, discarding the environment information.
RC-MNIST
There are a few key changes in synthesizing the Rotated Colored MNIST dataset relative to C-MNIST. Firstly, for each observation new 'common cause' variable is sampled from (er5o Subsequently, the digits in the C-MNIST pipeline are rotated by 45 degrees the common cause variable is equal to 1, and differently than in C-MNIST. Instead of copying the grayscale image in a new channel, the new channel is set to constant value of 1.0 across all spatial entries if the color signal equal and is zeroed out otherwise. Finally, this color channel is shifted up by 0.1 if the common cause variable equal and is shifted down by 0.1 otherwise to produce the final dataset.
rotated otherwise. The color signal is represented
TC-MNIST
Two-Colored MNIST is similar to RC-MNIST (but without the perturbations due the common cause variable), with the addition of new channel representing an additional color signal com- puted from the corrupted label. For each observation, this new color signal channel is perturbed by sampling single value from Normal(O, 0.1) and adding the value to all spatial entries in the channel.
For the purposes of training the IRM baseline, the new color signal is computed as for the original signal but using new environment specific corruption parameter. Similarly, two environments are respectively created using environment parameters of 0.2 and 0.1 for the original color signal, and 0.3 and 0.2 for the new color signal. Both environments are collapsed into one for training our models and the ERM baseline.
13

---

Under review as a conference paper at ICLR 2021
B DERIVATION OF OPTIMAL IMPORTANCE WEIGHTS
Assuming that the supports the training and test distributions overlap, can be shown (Quionero- Candela al. 2009) that the following importance weights for the risk under the training distribution produce an unbiased estimator of the risk under the test distribution:
Ptest (X, Y)
w" (X
Y)
(9)
Ptrain (X, Y)
where Ptrain and Ptest denote training and test distributions respectively.
For the benchmarks considered in this paper, we have access to the complete generative model that produces the data and are thus able to compute these optimal weights in closed form (assuming the input computed from the raw digit image and color signal C). For C-MNIST:
Prest (X,Y)
w" (X,Y)
Pirain (X,Y)
Ptest (Y Presse (X
Dirain Piraain (atraan(((()) (XIY)
Prest (
(since marginal distributions of Y are the same)
Ptrain (Xl
Piest (C, D(Y)
Pirain C, D(Y)
Prest C Ptest (DIY)
(since colour and digit are conditionally independent given Y)
Ptrain C' Ptraain (DlY)
Ptest (ClY) (since conditional distributions of digit given label are the same)
Ptrain (CIY)
Prest (C)
(since colour is independent of label at test time)
Ptrain YC
0.5
(under test distribution colour is Bernoull((0.5)
Ptrain YC
0.5
0.5
else
0.85
0.15
(10)
That is, observations where colour and label do not coincide (i.e. contradicting the spurious correla- tion the training distribution) are relatively upweighted.
14


Reviews:
[{"review_id": "p3SQUyK3vI", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a method for training an ensemble of classifiers for problems where spurious correlations may be present.  The models in the ensemble are encouraged to learn conditionally disentangled representations.  It is suggested that this disentanglement will result in the spurious correlations being consigned to a subset of the models in the ensemble.  The ensemble will then be ready to quickly adapt to a new data distribution without the spurious correlations, via selection of a single model that does not depend on the spurious correlations, or via reweighting of model outputs.\n\nThe author proposes a loss function that combines the standard ERM loss (summed over ensemble constituents), with a variant of the InfoNCE objective that is proposed as an estimate of total correlation.  The total correlation estimator involves a learned variational critic function. The authors argue for the use of conditional total correlation in the supervised setting, since, without conditioning, the minimization of total correlation interferes with the supervised learning objective.\n\nThe empirical evaluations focus on three variants of the MNIST dataset where spurious correlations have been introduced.\n\nThe learning of disentangled representations is highly studied in the context of unsupervised learning, which is alluded to in the paper.  This work is novel in that it focuses on the utility of disentangled learning in supervised learning.  While the proposed method does not directly learn a single model that can discard spurious correlations (like IRM is designed to do when multiple environments are available), it outputs an ensemble where some a priori unknown subset of the models do ignore the spurious correlations.\n\nAt a high level, there is a similarity to ensembling methods such as random forests where correlation between constituent models is deliberately suppressed. However, the motivation is different in that case: individual tree models are high-variance, and averaging uncorrelated trees reduces the variance of the overall model. The authors may want to cite that literature, however.\n\nThe paper points to an interesting direction of research.  Overall there are some ways in which the authors could strengthen the case for their method, and make the presentation clearer.\n\n*    My understanding is that InfoNCE provides a lower bound on mutual information.  Hence it makes sense to maximize it when the goal is to maximize MI.  In this case, minimizing TC would seem to require minimization of an upper bound.  Can the authors justify their use of the proposed estimator, perhaps with a more rigorous derivation that it actually is an effective bound on TC in the context of minimization?  If it is indeed a lower bound, why does minimizing a lower bound work?\n*    The actual definition of the estimator is unclear to me.  I understand $K$ to be the batch size.  It is clear how to plug in to Equation 6 to calculate the unconditioned estimator for a single batch. However, Equation 7 takes an expectation conditional on the label.  How is this estimated?  Are we to assume that the contrastive examples are sampled from the subset of the batch that shares the same value of $Y$ as $Y_i$?  I do not believe this is addressed in the paper.\n*    The argument for conditioning in Section 3 is reasonable; however, it is not backed up by concrete results.  I would agree that, without conditioning, the TC objective would conflict with the ERM objective.  However, I would think some balance would be reached where an unconditioned objective would still yield decently disentangled representations and models with decent accuracies.  Can the authors show a simple simulation that proves the superiority of the conditioned objective?  This becomes more important with a large number of classes, where estimating the conditioned TC becomes more challenging.\n*    The setup for the empirical results is an interesting and appropriate one for the method proposed.  However, the authors do not argue for its practical relevance.  Are there real-world problems where this kind of rapid adaptation of ensembles is required?\n*    On a similar note, the paper left me curious whether this disentangled learning is useful for the original supervised task. For example, analogous to random forests, does the disentangled model improve upon the performance of a single model in any datasets?\n*    The results on the MNIST datasets are compelling.  By my reading, the most impressive thing is the “Best” performance for “ours”.  I think the results could be further strengthened by including a simulation example where the function form is fully known, and the internal mechanics of the proposed method could be more fully explicated.  For example, the simulation setup of the IRM paper could be borrowed.  Then, the authors could show whether one of the members of the ensemble actually learns the correct causal model.  Right now, there are some compelling results on MNIST, but it could be made clearer what the method is actually doing.\n*    A minor point: On first reading, I found the results to be confusing, since ERM (single model) differed between “Linear” and “Best”.  It seems the difference is that for “Linear”, the top layer of the model is retrained, and for “Best” the model is unchanged.  The authors may want to clarify what “Linear” and “Best” refer to for a single model.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "bEM85KekHy", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper discusses the problem of inducing complementary representations to improve classification and isolate relevant signals among possible spurious/artificial ones. This is certainly an interesting agenda, and the experiments seem to provide some empirical evidence that the proposed method seems to work reasonably well, yet I must say that from the theoretical side, I did not manage the identify the reasons for this. Also, the paper is sometimes difficult to follow, as it is not totally rigorous and also sometimes assume strong background knowledge about very specific points. \n\nMain remarks:\n\n* While the announced agenda of the authors seems to get rid of spurious correlations in the original features and to identify informative, non-redundant signals issued from those, the whole paper concerns possibly complex transformations h_n of those (by the way, has the number of functionals to be the same as the one of features?). How does this compare to, say, find an efficient Binary Coding Matrix with low overlap or with feature selection approaches?  \n\n* P4, top: the estimator is built using an analogy with the pairwise case, however nothing is said about the properties of this analogy. Does it remain a lower bound? How good an approximation of the original goal can we expect it to be? \n\n* P4, top half: a lot of sampling needs to be done to get the estimates (again, are we speaking of X_i or h_i(X)? Since we are in the same section as equation (5), why Y has now turned into what seems to be the target classification variable?), how are they actually achieved, since most of them require access to theoretical distributions?\n\n* The results are a bit disappointing when allowing for linear combination of the classifiers, and from the description it is unclear to me whether Optimal IWERM is an obtainable model or not (I understand that it is, but it is not explicitly mentioned in the list, or is the IWERM the last model mentioned at the bottom of page 4? And in this case what is \"optimal\")? Because it performs quite comparably to the proposed model in the Best case. Also, why is there a difference between linear combination of models and best models in the case of ERM (single model)?\n\n\nMinor remarks:\n\n* P2, it should be said that KL is for Kullback-Leibler divergence\n\n* P3, top, what is k? Are c_i vectors or functions (from where to where in the last case)? The composition dot symbol suggest the latter. \n\n* P3, top: are \\theta parameters of h_i or c_i?\n\n* P3, before equation (5): suddenly Y becomes a feature and not the target class variable. This is confusing at first. \n\n* P3, bottom: what is a differentiable variational critic computing unnormalised score? Can a reference be given? \n\n* P4, bottom: \"and only included for comparison\" --> \"and are only included for comparison\"\n\n* P5, bottom: \"we tune ... drawn from the new test distribution\". There is unclarity about whether the test distribution is a subset of the training set or data issued from a test set (in which case certainly one cannot tune hyper-parameters using them). Maybe a synthetic drawing would help to understand how are created/used the noisy data sets? ", "extra_scores": {}, "extra_text": {}}, {"review_id": "Zi_tTAyEuaV", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an approach of training an ensemble of DNN classifiers while also minimizing the total correlation (TC) between the last layers (learned feature representations) of the classifiers to increase robustness to spurious correlations.  To compute gradients for this new TC regularization term, they use the InfoNCE objective as a proxy, and minimize this, and then use alternating minimization to learn the parameters. The authors then test this on a variation of the Colored MNIST task, showing gains over baseline approaches.\n\nOverall, this is a simple and intuitive approach.  Simple and intuitive is good- but requires (A) rigorous comparison to prior work to show why something like this has not been done before, and (B) rigorous experimental comparison to other approaches.  Unfortunately this paper does not do enough here.\n\nRegarding related work- the idea of adding a term that optimizes for diversity of the learned features is not a new one (random sample of papers: https://openreview.net/pdf?id=Hy1QdPyvz, https://www.researchgate.net/publication/330257555_Regularizing_Deep_Neural_Networks_by_Enhancing_Diversity_in_Feature_Extraction, http://www.cs.cmu.edu/~epxing/papers/2017/Xie_Singh_Xing_ICML17.pdf; and many others).  Note in particular that while the authors discuss an \"ensemble\" of classifiers, there is not a major distinction between this and a single model where the diversity of that model's features is regularized.  The authors here compare to a wide range of approaches- QD algorithms, ensemble-based methods, factorized generative models, ICA, and more- but seem to miss more basic comparables.\n\nMore importantly: the experiments compare to only one other method that is explicitly not a valid apples-to-apples comparison because of different input information required; and only examine one fairly bespoke semi-synthetic dataset.  The authors would need to compare to (A) a broader range of similar feature diversity objective approaches, and (B) should have explored whether this actually works on standard real-world datasets (where spurious correlations naturally occur), if they wanted compelling and general results.", "extra_scores": {}, "extra_text": {}}, {"review_id": "DLjglPu3EkU", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to learn a collection of classifiers, each of which is incentivized to use distinct features. The motivation is that among these distinct models, some would leverage robust signals present in the training data that can be generalized to test-time distribution shift; in contrast to ERM training where a model will use a mixture of robust and spurious signals and thus fail to generalize.\n\nIn particular, the authors train multiple models that minimize both their ERM losses and the total correlation (TC) of their final layer representations conditioned on the label. TC is an extension of mutual information (MI) that considers n random variables, and the authors similarly extend the InfoNCE estimator of MI to n variables to estimate TC. \n\nExperiments are conducted on the MNIST and Fashion-MNIST datasets, with three types of spurious signals injected into training data. Test data is constructed such that the spurious correlations no longer exist. Methods are tested in two adaptation setups: choosing the single best performing model from the collection, or training a linear model on top of the final layer representations from each model.\n\nOverall, I think the idea of training a set of distinct models to separate robust and spurious features is interesting, and results under the \"best\" setup show that the proposed method has isolated a single model that can generalize to the test data, which is promising. However, under the widely adopted \"linear\" adaptation setup, the proposed method has no advantage over a simple ensemble or a collection of classifiers trained to maximize pairwise cosine distances between final layer representations, which makes the value and practicability of the proposed method unclear. In addition, can you provide analysis of which features are learned by each model in the collection? I think this is important for understanding the model behavior.\n\nAbout TC estimation. InfoNCE is a lower bound of MI. In representation learning where one wants to maximize the MI between the feature representation and the label, it makes sense to maximize a lower bound. But here you want to minimize TC, why you choose a lower bound instead of an upper bound? The choice of lower bound also forces you to use a minmax objective, which might be a roundabout way.\n\nLastly, I think it would be interesting if there is a way to identify the robust models from the spurious ones (perhaps based on some inductive bias) without having to access the test distribution. This would also show the superiority of your method over the ensemble baselines that require fine-tuning on test examples.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper tackles a major problem of supervised ML, that of the minimisation of the risk of a set of classifiers. This problem has received attention in numerous work over the past decades, much of which spans the formal aspects of the problem. The paper tackles the problem from a “diversity” standpoint. My main concern is, for such a problem and exhaustive formal and experimental SOTA, one cannot just evacuate any formal understanding of a contribution to future work (Authors’ reply to R2). The argument is then a victim of its own content, ending up in a sloppy vocabulary where “speculation” and “intuition” are called forward as justification to the calls for “rigorous” (R1) and “theoretical” understanding (R2 + answer to R2). I am confident the authors can find formal merit to their contribution, but this needs to be addressed. R1 + R4 hint on avenues to understand the contribution. 

Decision: Reject