Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes a method for training an ensemble of classifiers for problems where spurious correlations may be present.  The models in the ensemble are encouraged to learn conditionally disentangled representations.  It is suggested that this disentanglement will result in the spurious correlations being consigned to a subset of the models in the ensemble.  The ensemble will then be ready to quickly adapt to a new data distribution without the spurious correlations, via selection of a single model that does not depend on the spurious correlations, or via reweighting of model outputs.

The author proposes a loss function that combines the standard ERM loss (summed over ensemble constituents), with a variant of the InfoNCE objective that is proposed as an estimate of total correlation.  The total correlation estimator involves a learned variational critic function. The authors argue for the use of conditional total correlation in the supervised setting, since, without conditioning, the minimization of total correlation interferes with the supervised learning objective.

The empirical evaluations focus on three variants of the MNIST dataset where spurious correlations have been introduced.

The learning of disentangled representations is highly studied in the context of unsupervised learning, which is alluded to in the paper.  This work is novel in that it focuses on the utility of disentangled learning in supervised learning.  While the proposed method does not directly learn a single model that can discard spurious correlations (like IRM is designed to do when multiple environments are available), it outputs an ensemble where some a priori unknown subset of the models do ignore the spurious correlations.

At a high level, there is a similarity to ensembling methods such as random forests where correlation between constituent models is deliberately suppressed. However, the motivation is different in that case: individual tree models are high-variance, and averaging uncorrelated trees reduces the variance of the overall model. The authors may want to cite that literature, however.

The paper points to an interesting direction of research.  Overall there are some ways in which the authors could strengthen the case for their method, and make the presentation clearer.

*    My understanding is that InfoNCE provides a lower bound on mutual information.  Hence it makes sense to maximize it when the goal is to maximize MI.  In this case, minimizing TC would seem to require minimization of an upper bound.  Can the authors justify their use of the proposed estimator, perhaps with a more rigorous derivation that it actually is an effective bound on TC in the context of minimization?  If it is indeed a lower bound, why does minimizing a lower bound work?
*    The actual definition of the estimator is unclear to me.  I understand $K$ to be the batch size.  It is clear how to plug in to Equation 6 to calculate the unconditioned estimator for a single batch. However, Equation 7 takes an expectation conditional on the label.  How is this estimated?  Are we to assume that the contrastive examples are sampled from the subset of the batch that shares the same value of $Y$ as $Y_i$?  I do not believe this is addressed in the paper.
*    The argument for conditioning in Section 3 is reasonable; however, it is not backed up by concrete results.  I would agree that, without conditioning, the TC objective would conflict with the ERM objective.  However, I would think some balance would be reached where an unconditioned objective would still yield decently disentangled representations and models with decent accuracies.  Can the authors show a simple simulation that proves the superiority of the conditioned objective?  This becomes more important with a large number of classes, where estimating the conditioned TC becomes more challenging.
*    The setup for the empirical results is an interesting and appropriate one for the method proposed.  However, the authors do not argue for its practical relevance.  Are there real-world problems where this kind of rapid adaptation of ensembles is required?
*    On a similar note, the paper left me curious whether this disentangled learning is useful for the original supervised task. For example, analogous to random forests, does the disentangled model improve upon the performance of a single model in any datasets?
*    The results on the MNIST datasets are compelling.  By my reading, the most impressive thing is the “Best” performance for “ours”.  I think the results could be further strengthened by including a simulation example where the function form is fully known, and the internal mechanics of the proposed method could be more fully explicated.  For example, the simulation setup of the IRM paper could be borrowed.  Then, the authors could show whether one of the members of the ensemble actually learns the correct causal model.  Right now, there are some compelling results on MNIST, but it could be made clearer what the method is actually doing.
*    A minor point: On first reading, I found the results to be confusing, since ERM (single model) differed between “Linear” and “Best”.  It seems the difference is that for “Linear”, the top layer of the model is retrained, and for “Best” the model is unchanged.  The authors may want to clarify what “Linear” and “Best” refer to for a single model.

review_title: A novel area of investigation, but the presentation could be strengthened

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes to learn a collection of classifiers, each of which is incentivized to use distinct features. The motivation is that among these distinct models, some would leverage robust signals present in the training data that can be generalized to test-time distribution shift; in contrast to ERM training where a model will use a mixture of robust and spurious signals and thus fail to generalize.

In particular, the authors train multiple models that minimize both their ERM losses and the total correlation (TC) of their final layer representations conditioned on the label. TC is an extension of mutual information (MI) that considers n random variables, and the authors similarly extend the InfoNCE estimator of MI to n variables to estimate TC. 

Experiments are conducted on the MNIST and Fashion-MNIST datasets, with three types of spurious signals injected into training data. Test data is constructed such that the spurious correlations no longer exist. Methods are tested in two adaptation setups: choosing the single best performing model from the collection, or training a linear model on top of the final layer representations from each model.

Overall, I think the idea of training a set of distinct models to separate robust and spurious features is interesting, and results under the "best" setup show that the proposed method has isolated a single model that can generalize to the test data, which is promising. However, under the widely adopted "linear" adaptation setup, the proposed method has no advantage over a simple ensemble or a collection of classifiers trained to maximize pairwise cosine distances between final layer representations, which makes the value and practicability of the proposed method unclear. In addition, can you provide analysis of which features are learned by each model in the collection? I think this is important for understanding the model behavior.

About TC estimation. InfoNCE is a lower bound of MI. In representation learning where one wants to maximize the MI between the feature representation and the label, it makes sense to maximize a lower bound. But here you want to minimize TC, why you choose a lower bound instead of an upper bound? The choice of lower bound also forces you to use a minmax objective, which might be a roundabout way.

Lastly, I think it would be interesting if there is a way to identify the robust models from the spurious ones (perhaps based on some inductive bias) without having to access the test distribution. This would also show the superiority of your method over the ensemble baselines that require fine-tuning on test examples.
review_title: Interesting idea, experimental results and analysis need to be strengthened

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes an approach of training an ensemble of DNN classifiers while also minimizing the total correlation (TC) between the last layers (learned feature representations) of the classifiers to increase robustness to spurious correlations.  To compute gradients for this new TC regularization term, they use the InfoNCE objective as a proxy, and minimize this, and then use alternating minimization to learn the parameters. The authors then test this on a variation of the Colored MNIST task, showing gains over baseline approaches.

Overall, this is a simple and intuitive approach.  Simple and intuitive is good- but requires (A) rigorous comparison to prior work to show why something like this has not been done before, and (B) rigorous experimental comparison to other approaches.  Unfortunately this paper does not do enough here.

Regarding related work- the idea of adding a term that optimizes for diversity of the learned features is not a new one (random sample of papers: https://openreview.net/pdf?id=Hy1QdPyvz, https://www.researchgate.net/publication/330257555_Regularizing_Deep_Neural_Networks_by_Enhancing_Diversity_in_Feature_Extraction, http://www.cs.cmu.edu/~epxing/papers/2017/Xie_Singh_Xing_ICML17.pdf; and many others).  Note in particular that while the authors discuss an "ensemble" of classifiers, there is not a major distinction between this and a single model where the diversity of that model's features is regularized.  The authors here compare to a wide range of approaches- QD algorithms, ensemble-based methods, factorized generative models, ICA, and more- but seem to miss more basic comparables.

More importantly: the experiments compare to only one other method that is explicitly not a valid apples-to-apples comparison because of different input information required; and only examine one fairly bespoke semi-synthetic dataset.  The authors would need to compare to (A) a broader range of similar feature diversity objective approaches, and (B) should have explored whether this actually works on standard real-world datasets (where spurious correlations naturally occur), if they wanted compelling and general results.
review_title: Intuitive and simple approach but experiments + related work are somewhat weak

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper discusses the problem of inducing complementary representations to improve classification and isolate relevant signals among possible spurious/artificial ones. This is certainly an interesting agenda, and the experiments seem to provide some empirical evidence that the proposed method seems to work reasonably well, yet I must say that from the theoretical side, I did not manage the identify the reasons for this. Also, the paper is sometimes difficult to follow, as it is not totally rigorous and also sometimes assume strong background knowledge about very specific points. 

Main remarks:

* While the announced agenda of the authors seems to get rid of spurious correlations in the original features and to identify informative, non-redundant signals issued from those, the whole paper concerns possibly complex transformations h_n of those (by the way, has the number of functionals to be the same as the one of features?). How does this compare to, say, find an efficient Binary Coding Matrix with low overlap or with feature selection approaches?  

* P4, top: the estimator is built using an analogy with the pairwise case, however nothing is said about the properties of this analogy. Does it remain a lower bound? How good an approximation of the original goal can we expect it to be? 

* P4, top half: a lot of sampling needs to be done to get the estimates (again, are we speaking of X_i or h_i(X)? Since we are in the same section as equation (5), why Y has now turned into what seems to be the target classification variable?), how are they actually achieved, since most of them require access to theoretical distributions?

* The results are a bit disappointing when allowing for linear combination of the classifiers, and from the description it is unclear to me whether Optimal IWERM is an obtainable model or not (I understand that it is, but it is not explicitly mentioned in the list, or is the IWERM the last model mentioned at the bottom of page 4? And in this case what is "optimal")? Because it performs quite comparably to the proposed model in the Best case. Also, why is there a difference between linear combination of models and best models in the case of ERM (single model)?


Minor remarks:

* P2, it should be said that KL is for Kullback-Leibler divergence

* P3, top, what is k? Are c_i vectors or functions (from where to where in the last case)? The composition dot symbol suggest the latter. 

* P3, top: are \theta parameters of h_i or c_i?

* P3, before equation (5): suddenly Y becomes a feature and not the target class variable. This is confusing at first. 

* P3, bottom: what is a differentiable variational critic computing unnormalised score? Can a reference be given? 

* P4, bottom: "and only included for comparison" --> "and are only included for comparison"

* P5, bottom: "we tune ... drawn from the new test distribution". There is unclarity about whether the test distribution is a subset of the training set or data issued from a test set (in which case certainly one cannot tune hyper-parameters using them). Maybe a synthetic drawing would help to understand how are created/used the noisy data sets? 
review_title: Interesting goal, but not entirely clear why, theoretically, the method should work. 

Meta-review: The paper tackles a major problem of supervised ML, that of the minimisation of the risk of a set of classifiers. This problem has received attention in numerous work over the past decades, much of which spans the formal aspects of the problem. The paper tackles the problem from a “diversity” standpoint. My main concern is, for such a problem and exhaustive formal and experimental SOTA, one cannot just evacuate any formal understanding of a contribution to future work (Authors’ reply to R2). The argument is then a victim of its own content, ending up in a sloppy vocabulary where “speculation” and “intuition” are called forward as justification to the calls for “rigorous” (R1) and “theoretical” understanding (R2 + answer to R2). I am confident the authors can find formal merit to their contribution, but this needs to be addressed. R1 + R4 hint on avenues to understand the contribution. 

Decision: Reject