Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes a novel feature-level data augmentation technique to train image classifiers in a debiased manner. The approach consists of disentangling “intrinsic” and “bias” attributes from training set images and then synthesizing new “data points” by combining intrinsic and bias features from different images. Since, a lot of image datasets contain a large amount of bias-aligned data points (high correlation between intrinsic and bias attributes), the proposed feature-swapping approach’s aim is to generate bias-conflicting data points. The approach results in better classification performance on 2 synthetic and 2 real-world datasets.
main_review: Strengths:
- The paper tackles an important problem — spurious correlations in image datasets that result in poor generalization in real-world applications. 
- The proposed approach is simple yet effective.
- The paper presents promising quantitative results for 4 different datasets, supporting the robustness of their approach.


Weaknesses:
- I think the paper needs much more explanation in lines 159-160 about what GCE is and how it enables learning bias attributes as compared to intrinsic attributes? Similarly, I think the paper should additionally discuss how cross-entropy loss enables learning intrinsic attributes? If CE is enough to learn intrinsic attributes, any typical image classification systems trained with CE should be able to learn disentangled representations and there wouldn’t be any need for debasing?
- Figure 3: Could the authors please include the visualization of z_b for BAR? Since the target labels are available, its coloring with target labels would still make sense. Also, I disagree that this figure (in its current form) shows that the proposed approach successfully disentangles the intrinsic and bias attributes for BAR dataset (referring to line 269).
- Figure 4: are these examples random or cherry-picked? Could the authors please include a randomly chosen 10x10 matrix?
- How are the ratio of bias-conflicting samples determined for each dataset?
- Lines 228: Both encoders and classifiers are MLPs?
- Lines 50-51: The thickness or the degree of tilt are mentioned as intrinsic attributes. How is that? Shouldn’t they be bias attributes? They should not be contributing to the “digit” classes. This raises a bigger question — how should intrinsic and bias attributes be determined for each dataset? By humans? I understand that the proposed approach doesn’t require it but its qualitative evaluation does require it.
- Section 3 shows that 1) larger diversity ratio is better and that 2) larger sampling ratio is better. Why is there a need to compare these two very different things with each other (lines 129-131)?

Overall, although the experiment section shows promising quantitative results, I believe that the paper lacks enough explanation of how their approach works. Therefore, I am a bit hesitant in recommending the paper for acceptance.


-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
I thank the authors for their response. It answers all my concerns/questions. Therefore, I am increasing my rating to accept.
limitations_and_societal_impact: Yes, the authors discuss their work’s limitations and societal impact.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: Dataset bias significantly degrades the generalization performance of deep neural networks. To alleviate this problem, the authors propose a novel approach for developing the debiased representation, where the model learns the disentangled representation and synthesizes diverse bias-conflicting samples via feature-level data augmentation. They demonstrate its effectiveness through extensive experiments on the image classification task.

main_review: [Strengths]

The idea is simple and interesting. They propose a straightforward and effective debiasing approach via disentangled feature augmentation. Also, the proposed method handles debiasing without presuming a certain bias type, which is advantageous for a practical setting.
They synthesize diverse bias-conflicting samples with feature-level data augmentation. And via a toy-set experiment, they show that the diversity of bias-conflicting samples is an important factor in debiasing.
They demonstrate the effectiveness of the proposed method through extensive experiments on the image classification task with synthetic and real-world datasets. (i.e., Colored MNIST, Corrupted CIFAR-10, BAR and bFFH)
 

[Weakness]

It is questionable whether the experiment on LfF [1] was performed properly. There is a significant gap between the reproduced performance of LfF in Table 2 and the reported performance in LfF. On corrupted CIFAR-10, the reported performance in LfF outperforms the proposed method. For validating a fair comparison, the authors should clarify this.

EnD [2] also handles debiasing based on the disentangled representation. Thus, at least the discussion with EnD is necessary; (conceptually and if possible empirically) comparing  the proposed method with it.

To show the effectiveness of debiasing, it should evaluate the performance on bias-conflicting samples as LfF [1] and EnD [2]. However, evaluations and comparisons are conducted only on unbiased samples in experimental results. By observing and evaluating on bias-conflicting samples, it would be clear where the effectiveness of the proposed method comes from.

Unlike LfF [1], the objective function of Equation (2) does not include the relative difficulty score W. Hence, it is not clear how E_i and C_i can learn intrinsic attributes. It should be clearly explained whether missing W in Eq (2) has any meaning (or reason)-- if so, how E_i and C_i can learn intrinsic attributes.
 
[Minor]

For readability, it would be better to add explanations for (i) and (ii) in Figure 3.
It would be great to show the effect of the proposed method by visualizing before and after applying the proposed method using CAM (or Grad-CAM) in the real-world dataset, BAR.
For example, in a bias-aligned sample such as ‘birds on the sky’, the proposed method may lead the model focus on the birds, not the sky.

[1] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. “Learning from failure: Training debiased classifier from biased classifier.” In Advances in Neural Information Processing Systems, 2020.

[2] Tartaglione, Enzo, Carlo Alberto Barbano, and Marco Grangetto. "EnD: Entangling and Disentangling deep representations for bias correction." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.

limitations_and_societal_impact: yes.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper presents a way to generate bias-conflicting samples at feature level, which help de-bias the model. For this, the authors first propose learning two separate (disentangled) representations: a) representation of implicit attributes (i.e., necessary to predict the target) learned through the normal cross-entropy loss and b) representation of biased attributes learned through generalized cross entropy loss (also used by LfF for bias amplification). The empirical results demonstrate that generating such samples outperforms oversampling/reweighting.
main_review: The proposed idea is a very neat adaptation of the code swapping technique [1] applied for bias mitigation. This is a good contribution to the field, as it provides a simple way to generate bias-conflicting samples at the representation level without relying on generative models or annotations for the bias attributes/labels.

Overall the paper is very well written and properly structured. I like the upfront analysis on ColoredMNIST and CIFAR10, motivating the need to for diversity. Ablations and comparisons against other methods e.g., LfF clearly demonstrate that training with the bias-conflicting samples generated in the proposed way improves over simply up-weighting the bias-conflicting/down-weighting the bias-aligned samples.

The 2D visualizations and reconstructed samples do a good job of showcasing the disentanglement (though not always perfect) between the intrinsic/bias attributes.

Next, I have a few comments/suggestions:

[C1] The main weakness is that the benchmarks used in the work seem to contain only one type of spurious correlation. I wonder what happens if there are multiple types of biases. Would the same GCE-based biased model capture all of those biases? Would one need to create multiple bias models and tune the GCE loss individually to capture those biases?

[C2] Following up on [C1], it would have been a huge plus to have stronger benchmarks with different kinds/levels of biases e.g., GQA-OOD [2].

[C3] Fig. 3 was a bit confusing and could be labeled/captioned better. I think both rows (target type) and columns (representation type) can be labeled to make things clearer. The BAR plot was a bit confusing -- I suppose the bottom right for column ‘c’ is all green due to the unavailability of bias labels?

Other Minor comments:

[MC1] I would prefer the phrase: ‘intrinsic attributes’ to be replaced by something more obvious e.g., ‘signal’ or ‘task-relevant attributes’ or perhaps ‘non-spurious attributes’. However, this is not a major concern, since it is well defined in the abstract/introduction.

[MC2] Line 175/Algorithm 1: The phrase ‘Randomly shuffled bias attributes’ was slightly confusing, as it could also mean shuffling the dimensions within the representation itself as opposed to swapping the entire ‘code’ between samples. Perhaps the text could say something like: "replace $z_b$ in $z$ with  $\tilde{z_b}$ from a randomly chosen sample in the batch to create the swapped representation: $z_{swap}$ ..."


Having said this, all in all, this is a good paper, with potential for high practical impact.

[1] Park, Taesung, et al. "Swapping autoencoder for deep image manipulation." arXiv preprint arXiv:2007.00653 (2020).

[2] Kervadec, Corentin, et al. "Roses Are Red, Violets Are Blue... but Should Vqa Expect Them To?." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
limitations_and_societal_impact: The paper acknowledges the imperfect disentanglement, but could also mention that the work does not study more realistic cases where biases can stem from different sources and the correlations between bias attributes and target labels may vary, which may pose greater difficulty for disentanglement.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposed a feature-level augmentation method to disentangle the bias and intrinsic features for debiasing image classification. This paper pointed out that the diversity of augmented bias-conflicting (i.e., few-shot, tailed) data is crucial for debiasing. The proposed framework consists of two encoders that encode the image into disentangled intrinsic and bias features. (1) To achieve the disentanglement, the modules for intrinsic and bias features are trained using cross-entropy (CE) loss and generalized cross-entropy (GCE) loss, respectively. (2) To generate augmented data, the intrinsic and bias features are swapped in each mini-batch. Experiments conducted on two synthetic datasets Colored MNIST and Corrupted CIFAR-10) and two real-world datasets (BAR and Biased FFHQ) demonstrate that the proposed method achieves the best performance compared to those that do not require a bias type. The quantitative evaluation further illustrates the disentanglement.
main_review: (+ Strengths, - Weaknesses, * Typos/Comments)

Overall, I like and value the research topic and motivation of this paper and lean positive. However, some details are not clear enough. I would update my rating depending on the authors' feedback. The details are as follows.

+ \+ Interesting and important research problem. This paper focuses on how to obtain disentangle representations for feature-level augmentation. This topic is interesting and important, and will attract many interests of the NeurIPS community.

+ \+ Good quality of writing and organization. Overall, the writing quality is good and the paper is well organized. It is comfortable to read this paper, although some details are not clear.

+ \+ Comprehensive experiments. Experiments are conducted on two synthetic datasets Colored MNIST and Corrupted CIFAR-10) and two real-world datasets (BAR and Biased FFHQ). 

- \- Relative difficulty score and generalized cross-entropy (GCE) loss. It is not clear how the relative difficulty score $W(x)$ in Eq. (1) is used in the pipeline. W(x) is not mentioned again in both the overall objective functions Eq. (2) or Algorithm 1. Since readers may not be familiar with the generalized cross-entropy (GCE) loss, it is encouraged to briefly introduce the formulation and key points of the GCE loss to make this paper more self-contained.

- \- How bias-conflicting samples and bias-aligned samples are selected. This weakness follows the first one. It seems that the "bias-conflicting" is determined based on the relative difficulty score, but the details are missed. Also, the ablation study on how the "bias-conflicting" is determined, e.g., setting the threshold for the relative difficulty score, is encouraged to be considered and included.

- \- Disentanglement. It is not clear how disentanglement is guaranteed. Although "Broader Impacts and Limitations" stated that "Obtaining fully disentangled latent vectors ... a limitation", it is still important to highlight how the disentanglement is realized and guaranteed without certain bias types.

- \- Inference stage. It is not clear how the inference is conducted during testing. Which encoders/decoders are preserved during the test stage?

- \- Figure 1 is not clear. First, it seems that the two $y$ towards $L_{\text{CE}}$ are the outputs of $C_i$, but they are illustrated like labels rather than predictions. Second, the illustration of the re-weighting module is not clear. Does it represent Eq. (4)?

- \- Table 4 reported a much lower performance of "swapping" on BAR compared to the other three datasets. Is there any explanation for this, like the difference of datasets?

- \- Sensitivity to hyperparameters. The proposed framework consists of three important hyperparameters, $(\lambda_{\text{dis}},$ $\lambda_{swap_b},$ $\lambda_{\text{swap}})$. It is not clear whether the framework is sensitive to these hyperparameters and how these hyperparameters are determined.

* \* (Suggestion) Illustration of backpropagation. As introduced in  Line 167-168, the loss from $C_i$ is not backpropagated to $E_b$. It would be clearer if this can be added in Figure 1.

* \* Line 280. Is "the first row and column ... respectively" a typo? It is a little confusing for me to understand this.

* \* Typos in Algorithm 1. Are $\lambda_\text{dis}$ and $\lambda_{swap_b}$ missed in $L_{\text{dis}}$ and $L_{\text{swap}}$?

* \* Typo in Line 209. Corrputed -> Corrupted.





============================= After rebuttal ===================================

After reading the authors' response to my questions and concerns, I would like to vote for acceptance.

The major strengths of this paper are:

* The research problem, unbiased classification via learning debiased representation, is interesting and would attract the NeurIPS audience's attention.
* The proposed method is simple but effective. The method is built on top of LfF [12] and further considers (1) intrinsic and bias feature disentanglement and (2) data augmentation by swapping the bias features among training samples.
* The paper is clearly written and well organized.

These strengths and contributions are also pointed out by other colleague reviewers.

My main concerns were:

* Unclear technical details of the GCE loss and the relative difficulty score. This concern was also shared with Reviewer 8Ai1 and iKKw. The authors' response clearly introduced the details and addressed my concern well.

* Sensitivity to hyper-parameters. The authors' response provided adequate results to show the sensitivity to hyper-parameters.
Other details of implementation and analysis of experimental results. The authors' responses clearly answered my questions.

Considering both strengths and the weakness, I am happy to accept this paper.
limitations_and_societal_impact: The authors have adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

Meta-review: All reviewers are in agreement that the paper tackles an important problem in a simple yet effective, and enthusiastically recommend acceptance. The authors provided extensive experiments and explanations in their rebuttal which should be incorporated into the final version. 
Decision: Accept (Oral)