SEBRA : Debiasing through Self-Guided Bias Ranking

Authors: Adarsh Kappiyath, Abhra Chaudhuri, AJAY KUMAR JAISWAL, Ziquan Liu, Yunpeng Li, Xiatian Zhu, Lu Yin
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Spurious Correlations, Bias Mitigation, Sub-population shift, Fairness

Abstract:
Ranking samples by fine-grained estimates of spuriosity (the degree to which spurious cues are present) has recently been shown to significantly benefit bias mitigation, over the traditional binary biased-vs-unbiased partitioning of train sets. However, this spuriousity ranking comes with the requirement of human supervision. In this paper, we propose a debiasing framework based on our novel Self-Guided Bias Ranking (Sebra), that mitigates biases via an automatic ranking of data points by spuriosity within their respective classes. Sebra leverages a key local symmetry in Empirical Risk Minimization (ERM) training -- the ease of learning a sample via ERM inversely correlates with its spuriousity; the fewer spurious correlations a sample exhibits, the harder it is to learn, and vice versa. However, globally across iterations, ERM tends to deviate from this symmetry. Sebra dynamically steers ERM to correct this deviation, facilitating the sequential learning of attributes in increasing order of difficulty, ie, decreasing order of spuriosity. As a result, the sequence in which Sebra learns samples naturally provides spuriousity rankings. We use the resulting fine-grained bias characterization in a contrastive learning framework to mitigate biases from multiple sources. Extensive experiments show that Sebra consistently outperforms previous state-of-the-art unsupervised debiasing techniques across multiple standard benchmarks, including UrbanCars, BAR, and CelebA.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 5

Reviews and discussion:
<thread>
creation date:
1730039941306

thread_type:
official_review

<thread_object>
creator: reviewer_apod

<content>
summary:
This work introduces SEBRA (Self-Guided Bias Ranking) to reduce bias in machine learning models without human intervention. Typically, models can develop biases by over-relying on spurious patterns in the data. SEBRA works by identifying these spurious patterns and ranking data points based on how likely they are to contain these misleading cues. During training, SEBRA adjusts the order in which samples are learned, focusing first on those with stronger biases and gradually moving to unbiased samples. This method allows the model to better distinguish between real patterns and biases, resulting in a more accurate and fair model.

SEBRA further incorporates this ranking into a contrastive learning framework, which contrasts data points with strong biases against those with fewer biases, enhancing the model’s ability to learn unbiased representations. Tests across datasets such as UrbanCars, CelebA, and BAR show that SEBRA outperforms other debiasing methods, particularly when multiple sources of bias are present.

strengths:
The paper is easy to understand, especially for readers unfamiliar with the area. The approach is interesting and addresses a problem of importance.

The proposed algorithm seems simple, intuitive yet effective. 

The illustration of results in the appendix is quite helpful in building intuition.

weaknesses:
The findings in the SEBRA heavily rely on Assumption 1, termed the Hardness-Spuriosity Symmetry, which states that the difficulty of learning a sample and its spuriosity (presence of misleading features) are inversely correlated. SEBRA uses this assumption to rank and prioritize samples in training, guiding the model to focus on samples with high spuriosity initially. The approach optimizes a ranking objective derived from this assumption, implying that SEBRA's effectiveness in reducing bias is largely contingent on this assumption holding true in practice. If assumption 1 does not hold, SEBRA’s ranking mechanism and bias-mitigation strategy might lose validity, as the model's ranking would no longer accurately reflect the spuriosity of samples. 

Please add more references in the related work section.

questions:
Can you delve on the motivation a bit? Why is this kind of de-biasing important?

Is Assumption 1 a part of known literature or is it something discovered during the behavioral exploration of this work? If it is a known fact in debiasing literature, please cite relevant works.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730126315665

thread_type:
official_review

<thread_object>
creator: reviewer_rg64

<content>
summary:
This paper proposes a bias mitigation method by ranking data points based on the assumption that the harder a data point is to learn, the less spuriosity it has and the overall ranking process is achieved by selecting and upweighting the samples that have not yet been learned, which relive the need of human supervision compared to existing method used to identify biased features.

strengths:
- The paper addresses the bias mitigation problem by introducing a fine-grained estimation method instead of the traditional binary biased-*vs*-unbiased partitioning approach, providing a more precise way to evaluate spuriosity and the combination of the contrastive learning framework is a nice attempt to further reduce the reliance on the few unbiased samples compared to existing methods by utilizing all the samples in the dataset.
- The intuition behind the proposed method is clearly with the definitions and theorems. Although I think the assumption (assumption 1, Hardness-Spuriosity Symmetry) is strong, it still demonstrates a clear logic explaining why the ranking mechanism is designed in this way.

weaknesses:
- The proposed method heavily relies on the assumption 1 (Hardness-Spuriosity Symmetry), which states that samples with large losses (difficult to learn) are likely to be less spurious. This raises concerns about the method’s effectiveness when the samples are outliers or have label noise issues....so if the large loss is caused by these samples, then utilizing them for training would adversely affect the performance on the downstream tasks. I think such condition should be discussed. 
- The proposed method has three parameters to optimize, so when the dataset has high dimensionality, the optimization process may become very slow, and potential convergence issues should be discussed.
- The overall writing of this paper is not very straightforward, as it repeats the main concepts too many times (e.g., the concept of spuriousness, the assumption, selection, upweighting.....) and the method section (3.1 to 3.3) has very weak connections, making each part appear independent of the others, which weakens the logical flow when introducing the method. I suggest the authors consider merging some of the repeated content (or summarize when it repeat) and adding some transitional words between sections to improve coherence.

questions:
- When selecting points that have not yet been learned, which are defined by large loss, how can the method avoid selecting data points that are outliers or affected by label noise issues?

- Is Assumption 1 (Hardness-Spuriousness Symmetry) supported by any literature or empirical evidence?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730442161383

thread_type:
official_review

<thread_object>
creator: reviewer_cwjj

<content>
summary:
The authors mainly aim to address spurious correlations by ranking instances according to the extent of biases automatically and conducting contrastive learning with these rankings. Specifically, based on the observation that the extent of biases is proportional to learning speed, they modify the learning process of ERM to progressively capture biases according to their extent. This modification process consists of three phases: selection, upweighting & training, and ranking. Then, with acquired rankings, they conduct contrastive learning to debias models by constructing negative pairs with similar levels of spurious correlations and positive pairs with different levels of spurious correlations. In experiments, the authors demonstrate the effectiveness of their method.

strengths:
* The proposed method can acquire the ranking of instances based on the extent of spurious correlation without human annotations.
* The authors show that their ranking is similar with the ground truth ranking and the overall framework mitigates performance gaps.
* The paper is well-written.

weaknesses:
- The paper lacks a detailed discussion on the utility of ranking instances according to the strength of spurious correlation. In this paper, the ranking is used for debiasing models via contrastive learning, yet it remains unclear how this approach offers high-level advantages over debiasing methods that do not utilize ranking. It would enhance the paper to discuss potential applications of their fine-grained spuriosity rankings beyond the contrastive learning framework. For example, the recent method, B2T [1], can identify and mitigate biases without human supervision by leveraging keywords. Since keywords are natural language, humans can interpret identified biases as well. Would the authors clarify the advantages over B2T?
- The experimental validation of the proposed method is not convincing. First, the comparison does not include recent debiasing methods [1, 2, 3]. Additionally, the model is validated only on a few datasets within the visual domain. The paper would be strengthened by including experiments on other widely-used datasets for spurious correlation, such as Waterbird, and NLP tasks like MultiNLI and CivilComments. Finally, the metrics are not commonly used in the spurious correlation domain, so reporting worst-group accuracy would enhance comparability.

[1] Kim, Younghyun, et al. "Discovering and Mitigating Visual Biases through Keyword Explanation." CVPR, 2024.

[2] Liu, Sheng, et al. "Avoiding spurious correlations via logit correction." ICLR, 2023.

[3] Deng, Yihe, et al. "Robust learning with progressive data expansion against spurious correlation." NeurIPS, 2023.

questions:
* Would the authors provide how much additional training time is required compared to training ERM. It would be helpful if you could provide a breakdown of the time spent on ranking and contrastive learning separately. Although there is a discussion in the appendix, specific values are not provided.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730697092168

thread_type:
official_review

<thread_object>
creator: reviewer_gzfw

<content>
summary:
This paper aims to rank samples by the degree of spuriosity and debias accordingly. The authors propose an unsupervised framework, Sebra, which ranks samples automatically by spuriosity in their classes. An important assumption of this paper is the negative correlation between spuriosity and the ease of learning a sample via ERM. Further, they dynamically steer ERM to satisfy this assumption during iteration training and sequentially learn samples in increasing order of difficulty. Experiment results show superiority of Sebra over other unsupervised debiasing methods.

strengths:
- The motivation of this paper is simple and clear: biased samples are harder to learn.
- The design of methods is well described and easy to understand.
- Performance improvement is significant.

weaknesses:
- The method parts are a little too long and not eye-catching, written in a way that follows the authors' thought processes. I would prefer it to be more concise and add more examples or illustration figures.

- Some sentences are way too long to understand, e.g., lines 340-343. Also, plz pay attention to distinguishing \cite and \citep.

- Class labels are required in this method.

questions:
- Can this method be extended to semi-supervised, especially when encountering new classes?
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper introduces a new bias ranking method that organizes data in descending order of spuriosity without requiring human supervision. Initially, the paper received mixed reviews before the rebuttal. However, following a constructive rebuttal, most concerns are successfully addressed. The only remaining major issue is raised by the reviewer 'APoD', who questions the validity of the assumption. After reviewing the paper and discussions, I believe this concern is not a big issue for the following reasons:

- The authors have conducted additional experiments in their latest response, supporting the assumption.
- Similar assumptions have been utilized in other studies.

Overall, I believe the advantages of this paper outweigh its disadvantages. I recommend accepting it, with the authors required to include all rebuttal details in the final version.