Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a method to mitigate the pervasive shortcut learning issue. The authors first make the observation that deep generative classifiers tend to encode spuriousness when the bottleneck is present. Based on this insight, authors propose a two-pronged approach where a generative classifier is trained while only a partition of the latent space is used for classification, and this part of the latent space absorbs the spuriousness. In the second stage, only the other latent code/representation is used to predict the label.
strengths_and_weaknesses: Strengths
- The paper tackles an important long standing problem. The motivation is intuitive and the method is simple and easy for the community to reproduce and build on top.
- The paper is laid out clearly and easy to follow, the logic flows smoothly
- Empirical results show large benefit over the chosen baselines
- The analyses are illuminant

Weaknesses
- The choice of baselines seems a bit slim. The main baselines are the vanilla ERM and the generative classifier. The only stronger baseline was JTT. The results will be more convincing if put into context w.r.t. other recent work that tries to mitigate spuriousness such as NURD (Out-of-distribution Generalization in the Presence of Nuisance-Induced Spurious Correlations; https://openreview.net/pdf?id=12RoR2o32T).
questions: - In the paper, the observation that the presence of the bottleneck affect how much the spuriousness will be picked up. Can this be shown as an ablation for the main experiments that when the z1-clf has larger bottleneck capacity (i.e., varying $z_p$), it carries less spuriousness and thus results in worse OOD performance? 
limitations: Yes.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors present an approach that aims to prevent "generative classifiers" from learning spurious correlation, which the authors refer to as short-cuts. They achieve this by splitting the latent representation in two, z1 and z2. The latent, z1, is passed to a classifier, while z1 and z2 are used to reconstruct the image. The model is trained using gradients from the classifier and the ELBO loss. The motivation being that z1 should contain the short-cut info, while z2 should not.
strengths_and_weaknesses: Originality:
This work proposes using a classifier on only a sub-set of the latent representation rather than the whole representation.

Quality:
It is very encouraging and highly commendable that the authors show results on a real world dataset, this is excellent! While their approach does much better on worst group accuracy, it still does very poorly on the average accuracy. Can the authors please elaborate more on this and explain why this result is still encouraging? Or if this suggests that their approach is not suitable for real world problems? Why do the other methods perform better?

It is not clear that there are sufficient pressures to ensure that the short-cut is not also encoded in z2, especially in non-trivial settings. This may explain why the model performs poorly on real world data.

There is very high variance between accuracy on different groups of data.

Clarity:
Aside from some terminology confusions mentioned in the box below, this work is exceptionally well motivated in the introduction.

It would be very helpful to have a diagram showing the encoding, decoding, classification with losses and gradients. The exact implementation and gradients are not 100% clear.

Significance:
The authors tackle an important and challenging problem. However, they fail to show convincing results on real world data.

Main strengths:
(1) The work is very well motivated.
(2) The authors evaluate on real-world data.

Main weaknesses:
(1) It is not clear how the approach works to mitigate short-cut information being encoded in z2.
(2) The approach does not work well on real-world data.
(3) The positive results are on simple datasets.
questions: The authors describe short-cut learning as “using simple features to achieve low training loss”. Surely it is best for the model to learn the simplest explanation since this is more likely to generalise? It would be helpful here to make the connection between “short-cuts” and spurious correlations.

Line 39-40: By “generative classifier” do you mean a classifier trained on representations learned via a generative model? Could you provide some (anecdotal) example of this?

To what extend is short-cut learning a failure to collect sufficiently diverse and unbiased data?

Line 90-95: In an unsupervised representation learning setting there are no labels, do these short-cuts still appear here? And if not, why not just fine-tune a classifier on top of representations learned without supervision? 

Figure 2(a): Reconstructions look poor in all cases. The do **not** appear to be especially bad in the case with the blue square. 

Figure 2(b): These results are very interesting and do motivate the problem well. This figure could also be improved by clearly showing an x-axis with the model capacity increasing and a box around the square.

This approach can only be used to factor out the information specific to the classes you have labels for?

In Figure 3: what is the class label here? Is it hair colour? Please clarify in the caption.

In Figure 3(d)-(e) please show the original faces too. This figure does show that this approach is “working”. However, it also suggests that gender is being encoded? Is there bias in the data to blonde women?

Table 2: Is it worth pointing out that the authors expect z1-clf to be poor on D_out? And that this further goes to show that z1 contains the spurious correlations?

It is not clear to me how using z1 for classification helps to ensure that z2 is better than a representation learned using a VAE without supervision (other than the fact that z1 and z2 share parameters, but to me this does not appear to be sufficient)? Is this achieved because z1 and z2 are needed for reconstruction and the assumption is that the information would not need to be encoded in both z1 and z2? This is very unlikely to hold in most cases. It may hold for a beta-VAE (or other disentangled representation). However, then there may not be sufficient information in z2 for classification? 

Does the approach only work for single label classification?

Table 3 and 4: why is there such high variation in accuracy between groups?

Table 4: Include the dataset information in the caption.

**I am happy to significantly increase my score if the authors can address the following:**
(1) How does the current setup explicitly prevent "short-cut" information being encoded in z2.
(2) Add a clear diagram with losses and gradients.
(3) It is highly commendable that the authors include results on real world data and I do not want to discourage this. Can the authors please explain any benefits their approach is having on this dataset or why we should be optimistic about the results.
limitations: Limitations are not addressed in the main body of the paper.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors presented a method for mitigating shortcuts using generative classifiers. The main idea is to isolate the latent factors corresponding to the shortcut in the VAE latent space and use complementary space for learning the classifier. The underlying intuition is that the shortcuts are easy to compress features, so they will only be present/detectable in a small fraction of the latent space embedding. Experiments show the efficacy of the method on multiple datasets.
strengths_and_weaknesses: ### **Strengths**

1. The paper is well written and easy to follow.
2. The idea is well-motivated and explained using information bottleneck theory and GradCam. The method separates the shortcut in the latent space of VAE and uses the remaining space to train the classifier invariant to the shortcuts.
3. The paper is trying to solve an important problem of mitigation shortcuts and spurious correlations.  
4. The authors presented intensive experiments with detailed analysis. The methods show clear performance gains and robustness to shortcuts.

### **Weakness**

The paper requires more benchmarks, in particular with causal learning methods. One proper benchmark would be the WILDS dataset [1]. Comparison with other methods such as [2] would be helpful.



[1] Koh et al.,  WILDS: A Benchmark of in-the-Wild Distribution Shifts
[2] Lee et al., Diversify and Disambiguate: Learning From Underspecified Data

questions: 1. It is not clear why gender would be a shortcut and not the hair color, if both are correlated with the output labels.
2. In Figure 4. C, the dimensions of z_1 is more than z_2. Shouldn’t it be smaller since z_1 is correlated to shortcuts and thus more compressible? 
3. What does ‘invariant’ means in  Table2.
4. Why do you think KNN gives better accuracy? 
5. “MLPs with smaller hidden layers have better accuracies, and our top performance is a k-nearest neighbor classifier with k = N^(½).” What does N denote here?

limitations: Yes, the authors have addressed the limitations and potential negative societal impact in the appendix.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper addresses the problem of shortcut learning in deep neural networks. First, the paper makes the observation that shortcut features tend to be the ones that are preferentially encoded when a limited amount of information can be encoded in one of the hidden layers of the network. The paper then builds on this observation to propose a two-stage approach for learning a classifier robust to shortcut solutions: first a VAE classifier is learned whereby the classification is only based on a small part of the latent, second an independent classifier is trained on the part of the latent space of the VAE that did not feed into the classifier. The efficacy of the method is demonstrated using several hard versions of ‘shortcut benchmarks’ in which the shortcut features are strongly or completely correlated with the targets.
strengths_and_weaknesses: The paper is well written, insightful and a pleasure to read. I am not very familiar with shortcut learning benchmarks, but I had the feeling I was able to follow the entire paper without too much effort.

The proposed solution for overcoming shortcut learning, the Chroma-VAE, is well motivated, clearly explained and to the best of my knowledge novel and original.

The performance of the Chroma-VAE is very strong on the toy dataset benchmarks. The paper provides a clear intuition for why the method is able to perform so well, which is backed up by insightful visualizations

In general I would like to support acceptance of this paper, but I do have one important concern that should be addressed first. My concern is that I suspect that the proposed Chroma-VAE method will only work well in ‘artificial’ situations where the shortcut feature that should be avoided is very strongly correlated with the target. If I’m not mistaken, in general, the part of the VAE’s latent space connected to the classifier should encode all features relevant for the classification task (i.e., both the shortcut feature and the true signal) while the unconnected part of the latent space should encode only irrelevant features. It seems to me that only when the shortcut features are very strongly correlated with the target, some other features with relevance for the classification problem will be left to be encoded in the unconnected part of the latent space. Perhaps by restricting the size of the part of the latent space connected to the classifier it is possible in practice to encourage that mostly the shortcut feature is encoded there, but on softer / realistic benchmarks I do not think this will be straightforward and I expect it might lead to low performance.
Therefore, to investigate this, I suggest that the authors additionally include experiments on “softer” versions of the shortcut benchmarks in which the shortcut features are less strongly correlated with the target.

The authors do include one experiment on a “real-world”, X-ray dataset, but the results of the Chroma-VAE method on this experiment are not very convincing, as there is a large drop in average accuracy. I am not familiar with this benchmark, but it seems to me that this is not a result with any practical value. (If I’m mistaken, please correct me. But in what practical situation would we prefer the model with the lower average accuracy?) I therefore think that the claim that “our method is viable on a natural, real-world data set” must be moderated, or else better motivated.

Minor points:
- The result mentioned in L270-271 seems to be missing from Table 3.
- It would be good to make the caption of Table 4 clearer (e.g., to include that these are the results for the X-Rays experiment).

questions: I would like to ask the authors to include evaluations of the proposed Chroma-VAE method on “softer” versions of the benchmarks in which the shortcut feature is only weakly correlated with the target and/or the authors should provide convincing evidence that the Chroma-VAE method can be useful in real-world settings (or discuss clearly why the currently reported real-world experiment has practical value).
limitations: If my main concern with this study holds (see above), the limitation that the proposed method depends on the shortcut feature being very strongly correlated with the target should be discussed.

I think the potential negative societal impacts are satisfactorily addressed.
review_title: null

Meta-review: metareview: This paper explores a method to learn generative classifier while mitigating 'shortcut learning'---relying on spurious correlations. They do so by modelling two latent variables, only one of which is used to classify, and both used to generate data. The underlying intuition is that 'shortcuts' will only manifest in a small fraction of the latent space embedding as they compress. Experiments show the efficacy of the method on multiple datasets.

The reviewers all agreed that the paper tackles an interesting and relevant problem, that the exploration of the performance and experiments considered are quite thorough.

The biggest issue raised in this instance is that the experiments largely only deal with restricted/constrained settings. While it is understandable that the problems tackled here are well accepted benchmarks within this subfield, the paper should make clear it's contribution and include a discussion about the potential for real-world application.
In particular, the X-ray example was something that was highlighted as something that, as currently presented, potentially can imply that the proposed method is viable for actual real-world use. It is strongly suggested that the authors moderate the interpretation / claims / discussion around this example.

The authors also provided additional experiments over specific questions on the information in the non-classifier latent to address reviewer concerns, which was good.  

Overall the paper has merit and explores an interesting idea with a good set of experiments that cover a range of settings, and should be accepted.

confidence: Certain
Decision: Accept