Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a method to de-bias classifiers and make them robust to spurious correlations, which is mainly based on learning an ensemble of intentionally biased models which will help to identify minority (bias-conflicting) examples in train data and increase their importance / weight while training a final de-biased model. Additional details of the method include (1) pre-training representations with self-supervised learning and learning shallow models on top of these representations, and (2) alternate updating of the biased ensemble and de-biased classifier where the knowledge is transferred from the classifier to the ensemble. The approach shows strong performance on a number of benchmarks: CelebA, ImageNet-9, ImageNet-A, BAR and NICO.
strengths_and_weaknesses: ### Originality
The proposed method is novel, to the best of my knowledge. Although prior works proposed to use an internationally biased model to detect minority examples in the past (e.g., [1], [2]) this papers is the first to proposed a biased bootstrapped ensemble and shows that it increases robustness of the final classifier model. The paper is based on a combination of known techniques (bootstrapped ensemble, using auxiliary module to identify biases, self-supervised learning, knowledge distillation), however, the proposed approach shows strong performance in practice, and, thus, I believe is a valuable contribution.


### Quality

1. My main concern for the paper evaluation is regarding the hyperparameter tuning: the authors described which hyperparameters they picked in lines 192-195, but it is not mentioned how exactly they were tuned and which range of values was considered for each hyper-parameter. It was shown in prior works on spurious correlation robustness that the worst-group results may vary a lot depending on whether "in-distribution" or "out-of-distribution" data was used for hyper-parameter tuning (i.e. which portion of the validation set was bias-guiding vs bias-conflicting) and whether the hyper-parameters were chosen based on average accuracy or worst-group accuracy on validation (the latter would imply that we assume having access to bias labels for validation data). Also e.g. on Figure 3(b) it is shown that worst-group performance varies quite a bit depending on the number of classifiers in the committee and it is mentioned that $m$ was chosen to be the one corresponding to the best performance -- but it is not clear whether this number was chosen based on best performance on validation or test.
 
It would be helpful to know which assumptions on validation data the authors made and the exact strategy for hyper-parameter tuning. 
 
2. While the observation that self-supervised (SSL) training improves the robustness to biases on most benchmarks is interesting, it sounds a bit disconnected from the rest of the paper. I think it is also not fair to compare SSL+LBC to other prior methods which were not using SSL pretraining (especially considering that SSL+ERM is already beating most methods on all benchmarks except CelebA). I think to perform a careful and fair ablation it would be needed to (1) compare LBC w/o SSL pretraining to prior works (to isolate the effect of improved representations), (2) with fixed SSL features, train and compare different prior methods and LBC on top of these features. 

3. I found it a bit surprising that such small subsets of data $S_l$ were used to train individual networks in the committee, I guess it is related to the fact that the authors train quite shallow MLP models on top of SSL features. I would be really interested to see how important boostrapping is in general for the method, e.g. what if one swapped bootstrapped ensemble with regular deep ensemble? Would the performance change much in cases when SSL pretraining is used and when there is no SSL pretraining (so we are training not a shallow but deep model)? It would also be interesting to see how the two parameters the size of $S_l$ and the number of members in the committee interact: is it the case that we might get the same performance with the smaller number of networks in the committee if they are trained on bigger splits of train data?

4. It would be helpful for understanding of the method to see the plot similar to Figure 1(a) but for the proposed approach showing what the ratio of bias-conflicting examples among the examples identified by a biased committee is.

5. Since the method consists of a few independent parts (biased committee, SSL, knowledge distillation) it would be helpful to see more ablation studies varying just one of those components, e.g. how much would the KD be helpful when using only one biased model as opposed to the committee? Similarly, what would the performance be with a single biased model on SSL features? (That would be quite related to reproducing a method like JTT [2] on SSL featured as mentioned in point 3 above) 

6. It would be interesting how this method works on other data modalities, e.g. NLP spurious correlation tasks such as CivilComments or MultiNLI.


### Clarity
The paper is clearly written and easy to follow. A minor note is that it might be more natural to swap 4.1 and 4.2 and first describe the datasets in the Experiments section since they are mentioned in 4.1. 


### Significance
I think although the method is based on a combination of known techniques, it significantly advances the performance on robustness benchmarks and especially if the authors address the evaluation concern described above the paper will be interesting for the robustness community.


References

[1] Nam et al, Learning from Failure: Training Debiased Classifier from Biased Classifier

[2] Liu et al, Just Train Twice: Improving Group Robustness without Training Group Information

[3] Lee et al, Diversify and Disambiguate: Learning From Underspecified Data
questions: Please see section "Quality" in "Strengths And Weaknesses".
limitations: Yes
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper considers the problem of learning to classify from spuriously-correlated data when annotations for the spurious attributes are absent. The authors propose to use the disagreement between an ensemble of classifiers trained on random subsets of the data (bagging) 
to generate sample-weights for the classification loss. In addition to being trained on their respective subsets with supervision
from the ground-truth labels, they are provided with an additional error signal from the remaining data in the form of a distillation loss
with respect to the 'main' classifier, trained using the full dataset. The authors make use of self-supervision (BYOL) to train the backbone, on top of which the auxiliary classifiers are trained, finding this alone to enhance robustness and complementary to the aforementioned reweighting. The proposed method, LBC, is evaluated on five datasets exhibiting with known spurious correlations and compared against a variety of baselines that are both supervised and unsupervised w.r.t. the spurious attribute. Qualitative results are additionally illustrated for the CelebA dataset to provide insight into the sample weights and whether they align with the biases in the dataset.

strengths_and_weaknesses: ### Strengths
- The idea of using a disagreement between ensemble members to generate sample-weights with the view to
improve fairness seems to be relatively novel
-  The authors do a good job of setting up the problem, establishing sufficient context for it in light of previous work, and motivating the use of a committee for identifying bias-conflicting samples through the analysis conducted in Figure 1.
- The proposed method is simple, intuitive, and demonstrably effective at alleviating spurious correlations.
- Evaluation was conducted using a good range of relevant datasets and baseline methods.
- The paper is well-presented, in that the results are easy to interpret, the proposed method can be understood based on its visual representation in Figure 2 alone, and the pseudocode is sufficiently detailed to make clear the more technical elements.
 - The paper is generally generally clear and well-written/structured, though there are some moderate grammatical errors, such as on line 225 (elaborated on below).
 - Good set of ablation studies demonstrating the contribution of each component of LBC, in turn, as well as the relationship
 between the size of the committee and performance. That said, it would be of interest to ablate the self-supervised component
 on its own to garner insight into how complementary it is to the proposed method, particularly as the dataset-task combination on which
 the ablations were performed show worse performance for SSL+ERM than ERM on its own (by a significant margin). Also of interest would be  how LBC compares with deep ensembling, where all ensemble members are trained on the full dataset with no distillation component.
 - Results are computed as the average over several replicates, alongside the with associated error intervals.

### Weaknesses
- The claim that this is the first paper to show the relationship between self-supervision and debiasing isn't entirely justified. 
There are, for instance, [1] showing that large-scaling pre-training with SWAV improves fairness on downstream computer vision tasks,
and [2] showing that models trained with self-supervision are more robust to adversarial perturbations and label corruptions.
- Figure 4 isn't referenced in the main text. There should be some discussion as to what is shown here and whether it aligns with expectation (which it does seem to given that disagreement is highest for those samples from the minority group - Blond-Male).
- While the related work section is reasonably thorough, there is a gap when it comes to discussion 
ensemble-based-debiasing methods (e.g. [3, 4]) and ensembling in the context of robustness generally, having obvious relevance to the proposed method.
- Despite the motivation encapsulated in Figure 1, there is a lack of theoretical guarantees regarding why and when should this bootstrapping approach should work 
for spurious correlations in general.
- The term 'enrichment', as used in Figure 1, is not explicitly defined.

### Misc.
- Liu et al. (Just Train Twice) is cited using two different references.
- 'Self sup' should be SSL in the table of ablations for consistency with 'SSL+ERM'.
- The parenthetical remark "e.g., most dog images are appeared ‘on grass’ the others are appeared on 6 context" doesn't make complete sense -- I assume this is supposed to read something to the effect of "More dog images appear on the 'on grass' context than any of the other 6 contexts"

questions: - Is the lack of a stop-gradient on the main classifier, $g$, intentional? Generally one doesn't allow the gradient to propagate through to the teacher network when performing knowledge distillation. If this is intentional, was it the result of testing or is there some theoretical justification for it?
- How essential is bagging? Could different random initialisations alone provide sufficient diversity, as suggested by the literature on deep ensembles?
- Why does the performance degrade past 40 ensemble members, rather than just plateauing?
- How important is the warm-up and how many iterations of it are typically required for stable performance?

limitations: I am generally satisfied with the extent to which the limitations of the proposed method have been discussed.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes LBC to mitigate biases without bias labels. Motivated by the limitation of LfF’s single biased classifier, LBC overcomes this limitation by learning a committee of biased classifiers. The LBC is also trained with the knowledge of the main classifier through knowledge distillation. Furthermore, the paper explored the role of self-supervised representation in bias mitigation. The experiments are conducted on multiple datasets, showing better performance of the proposed method.
strengths_and_weaknesses: Strengths

* The paper is well-written (except for a few typos) and easy to follow.
* The paper is well-motivated by the limitation of LfF’s single biased classifier (Fig. 1 and Sec. 1). The proposed method uses a committee with multiple biased classifiers to address this problem well, which has a novel contribution to this field.
* Using self-supervised representation for bias mitigation is also a novel exploration.


Weaknesses
* A few methods should also be cited and discussed in Section 2.2: PGI (Ahmed et al. 2021), ARL (Lahoti et al. 2020), George (Sohoni et al. 2020), and SD (Pezeshki et al. 2021), which are all debiasing methods without using spurious attribute labels.
Could the authors clarify why the results of ERM and LfF differ between Table 1 and Table 2? For example, ERM’s unbiased accuracy in Table 1 is 70.25 but becomes 95.6 in Table 2.

* Could the authors clarify which split is used to report the results on CelebA? The paper mentioned that the configuration follows [31] (L203), which uses the validation set to report the results. However, the experiment also uses CelebA’s validation set to tune the hyperparameter (L452-453).

* Could the authors promise to release the new split on BAR (L220-221)? Otherwise, it will become hard for future methods to benchmark.

Typos

* L51, 54, 68, and caption of Fig. 3: LCB -> LBC
* L160: identity -> identify
* L188: celebA -> CelebA


Although I have a few questions about the paper, the proposed method is generally novel and performs well on various datasets. My current rating is “Borderline accept.” I am willing to raise my rating if the authors can address my concerns in the response.

Reference

Ahmed, Faruk, Yoshua Bengio, Harm van Seijen, and Aaron Courville. 2021. “Systematic Generalisation with Group Invariant Predictions.” In International Conference on Learning Representations. https://openreview.net/forum?id=b9PoimzZFJ.

Sohoni, Nimit S., Jared A. Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. 2020. “No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems.” In Advances in Neural Information Processing Systems. http://arxiv.org/abs/2011.12945.

Lahoti, Preethi, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. 2020. “Fairness without Demographics through Adversarially Reweighted Learning.” In Advances in Neural Information Processing Systems. Vol. 33. https://papers.nips.cc/paper/2020/hash/07fc15c9d169ee48573edd749d25945d-Abstract.html.

Pezeshki, Mohammad, Sékou-Oumar Kaba, Yoshua Bengio, Aaron Courville, Doina Precup, and Guillaume Lajoie. 2021. “Gradient Starvation: A Learning Proclivity in Neural Networks.” In Advances in Neural Information Processing Systems. https://openreview.net/forum?id=aExAsh1UHZo.





questions: Questions
* Clarify the inconsistent results of ERM and LfF between Table 1 and Table 2.
* Clarify the split used on CelebA.
* Clarify if the new split of the BAR dataset will be released for reproducibility.


Suggestions
* Cite and add discussion on other debiasing methods without spurious attribute labels.

limitations: Yes. The limitations are discussed in Sec. 5.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a new approach to learning a debiased classifier with no spurious attribute label. Samples are weighted according to the consensus. The main classifier and committee classifier interacts. Committee classifier uses information about samples difficult to classify by the main classifier as distilled knowledge and integrates this into the loss function used to train the committee classifiers. Self-supervised representation obtained by BYOL is used as backbone.
strengths_and_weaknesses: - Distilling the knowledge of difficult samples from the main classifier to train committee classifiers is an intriguing idea. Doing this with a self-supervised backbone is also carefully considered.

- Random subsets of training may not necessarily be dominated by bias-guiding samples especially in long-tailed datasets

- Figure 1 does not demonstrate that a committee of classifiers focus more on bias-conflicting samples. Higher enrichment can possibly be achieved due to other factors as well?



questions: - BYOL is trained on the target dataset. It would be interesting to see how the representations learned this way on a limited data would compare to those learned from DINO (base or small VIT) pretrained on ImageNet. 

- It looks like different hyperparameters are used for each dataset. How are these (learning rate, batch size, committee size, \lambda, \tau  etc.} optimized?

- LBC outperforms SSL+ERM on the CelebA dataset but is on par with it on the ImageNet (as well as BAR and NICO) datasets. Is this because bias is not as strong in texture-heavy images of ImageNet as in CelebA dataset?

- In the light of the limitations stated in the paper how can you ensure the improvement in CelebA dataset is real and is not due to fluctuations of randomness in the experimental process that affect smaller datasets more?

- Lines 206-207: “Therefore, we use HairColor as the target attribute and Gender as a spurious attribute, the same as HeavyMakeup.” This sentence is confusing. The proposed approach is not supposed to use the explicit knowledge of attributes causing bias. It is not clear  what it means to select these attributes as “target” and “spurious” attributes and how this information was used during training. 


Corrections
Line 160: We identity
limitations: Algorithmic limitations are explained but societal impact is not.  What happens if the proposed algorithm fails to debias the data? This should have some ethical implications.

-------------------------------
Authors responses address my main concerns. As a result I update my score to Weak Accept.
review_title: null

Meta-review: metareview: The manuscript considers the problem of learning to classify from spuriously-correlated data when annotations for the spurious attributes are absent. The general idea is to learn an ensemble of intentionally biased models which will help to identify minority (bias-conflicting) examples in train data and increase their importance / weight while training a final de-biased model. Additional details of the method include (1) pre-training representations with self-supervised learning and learning shallow models on top of these representations, and (2) alternate updating of the biased ensemble and de-biased classifier where the knowledge is transferred from the classifier to the ensemble. The approach shows strong performance on five datasets: CelebA, ImageNet-9, ImageNet-A, BAR and NICO. Qualitative results are additionally illustrated for the CelebA dataset to provide insight into the sample weights and whether they align with the biases in the dataset.

Reviewers acknowledged several positive aspects of the manuscript including the proposed idea of distilling the knowledge of difficult samples from the main classifier to train committee classifiers. Performing this with a self-supervised backbone was also carefully considered. The manuscript is based on a combination of known techniques (bootstrapped ensemble, using auxiliary module to identify biases, self-supervised learning, knowledge distillation), however, the proposed approach shows strong performance in practice.

Discussion phase has addressed many concerns related to experimental evaluations inc. hyperparameter tuning, importance of self-supervised backbone, and importance of bootstrapping. 

confidence: Certain
Decision: Accept