BAdd: Bias Mitigation through Bias Addition

Authors: Ioannis Sarridis, Christos Koutlis, Symeon Papadopoulos, Christos Diou
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: fairness, bias, spurious correlations

Abstract:
Computer vision (CV) datasets often exhibit biases in the form of spurious correlations between certain attributes and target variables that are perpetuated by Deep Learning (DL) models. While recent efforts aim to mitigate such biases and foster bias-neutral representations, they fail in complex real-world scenarios. In particular, existing methods excel in controlled experiments on benchmarks with single-attribute injected biases, but struggle with complex multi-attribute biases that naturally occur in established CV datasets. Here, we introduce BAdd, a simple yet effective method that allows for learning bias-neutral representations invariant to bias-inducing attributes.  It achieves this by injecting features encoding these attributes into the training process. BAdd is evaluated on seven benchmarks and exhibits competitive performance, surpassing state-of-the-art methods on both single- and multi-attribute bias settings. Notably, it achieves +27.5% and +5.5% absolute accuracy improvements on the challenging multi-attribute benchmarks, FB-Biased-MNIST and CelebA, respectively.

Decision: reject
Reviewer scores: 5, 6, 8, 5

Reviews and discussion:
<thread>
creation date:
1729743902286

thread_type:
official_review

<thread_object>
creator: reviewer_cpaj

<content>
summary:
The paper introduces BAdd (Bias Addition), a method for mitigating bias in deep learning models for computer vision tasks. Bias in datasets often arises from spurious correlations between certain attributes (e.g., gender, race, background) and target variables, leading models to make decisions based on these irrelevant features. Existing bias mitigation methods often struggle in complex, real-world scenarios, especially when multiple biases are present.

BAdd addresses this issue by injecting bias-capturing features into the model's training process. Specifically, it involves adding features that encode the protected (bias-inducing) attributes into the penultimate layer of the model during training. This approach aims to decouple the learning of biased features from the optimization process, allowing the model to learn bias-neutral representations of the data.

The method is evaluated on seven benchmarks, including both artificially biased datasets (e.g., Biased-MNIST, Corrupted-CIFAR10) and more complex, multi-attribute biased datasets (e.g., FB-Biased-MNIST, CelebA, UrbanCars). Notably, it achieves accuracy improvements of +27.5% on FB-Biased-MNIST and +5.5% on CelebA.

strengths:
Versatility: The approach is architecture-agnostic and can be easily integrated into any deep learning model without extensive modifications or pre-processing.
Visualization of Results: Use of activation maps and GradCam visualizations helps to intuitively demonstrate how BAdd shifts the model's focus away from bias-inducing features.

weaknesses:
Assumption of Bias Representability: The method assumes that biases can be captured and represented explicitly. In cases where biases are subtle or unknown, this approach may be less effective.
Potential Increase in Model Complexity: Adding bias-capturing features might increase computational overhead during training and potentially during inference if not properly managed.
Fine-Tuning Requirement: The necessity of a fine-tuning step adds an extra phase to the training process, which might not be ideal in time-constrained or resource-limited applications.
Limited Analysis on Real-World Biases: While the paper includes evaluations on datasets like CelebA, further analysis on real-world datasets with complex, less-defined biases would strengthen the applicability of BAdd.

questions:
Availability of Protected Attributes: How does BAdd perform when protected attribute labels are unavailable or unreliable?

Handling Unknown Biases: In real-world applications where biases may be unknown or multifaceted, how can BAdd be adapted to mitigate biases that are not explicitly identified?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730216529759

thread_type:
official_review

<thread_object>
creator: reviewer_e7ck

<content>
summary:
The paper introduces a BAdd approach to mitigate the effects of biased data during model training. The idea is to incorporate captured bias features into the final layer of the model, which helps the model be invariant for these features and create a bias-neutral feature representation.

strengths:
1-	The problem is described clearly.

2-	The investigated problem is important.

weaknesses:
1-	The BAdd approach seems similar to FLAC [1]. 

2-	On page 3, The author claims that BAdd is easily applied to any network architecture and to any CV dataset. This claim needs to be proven.

3-	The paper does not discuss the computational complexity or scalability of the proposed approach in detail, which could be a concern for large-scale applications.

4-	Limited Ablation Studies: While the paper includes some ablation studies, more extensive ablations could strengthen the claims about the individual contributions of each component of the proposed method.

[1] Ioannis Sarridis, Christos Koutlis, Symeon Papadopoulos, and Christos Diou. Flac: Fairnessaware representation learning by suppressing attribute-class associations. arXiv preprint arXiv:2304.14252, 2023a.

questions:
1-Describe the key differences between the BAdd and  FLAC [1].

2-Conduct experiments using different architectures, such as ViT with BAdd, to prove that the approach works properly with different architectures. Also, use the ImageNet dataset to show the approach performance with the balanced dataset.

3-Discuss the computational complexity of BAdd approach.

4-We encourage the author to do more ablation studies on the proposed approach to show consistency, such as changing the batch size.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730595620514

thread_type:
official_review

<thread_object>
creator: reviewer_jpua

<content>
summary:
The model proposed in this paper learns unbiased features by adding specific elements that capture potential biases during training. This ensures that the model doesn’t rely on biased information when making predictions. BAdd was tested on various datasets, including those with single and multiple biases, and showed strong improvements in reducing bias and overall model performance.
So,  this paper:
1.Introduced BAdd, an easy and effective method for learning unbiased features by incorporating bias-detecting elements into training.
2.Evaluated BAdd on several benchmarks, including different types of biased datasets, showing that it outperforms other state-of-the-art methods.

strengths:
1.BAdd reduces bias effectively by adding bias-related features into the training, helping the model avoid being influenced by biased data.
2.The authors showed that BAdd works well on different datasets, with consistent improvements in various bias situations, proving that the method is scalable and works in different applications.

weaknesses:
1.BAdd requires a classifier or labels that identify the bias, which may not always be available. The authors could consider ways to detect and handle biases automatically without needing predefined labels.
2.The method mainly addresses visual biases, and it’s unclear if it works for other types of biases, like those in text, limiting its use beyond visual data.
3.The paper is clear, but lacks a detailed comparison to standard deep learning training. It’s not explained how the bias-detecting classifier fits into the training process.
4.In line 728, the tense is inconsistent—“were” should be changed to “are.”

questions:
please check the weakness part, and give some explanations.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730630383621

thread_type:
official_review

<thread_object>
creator: reviewer_itn8

<content>
summary:
The document introduces BAdd, a method for mitigating bias in deep learning models by injecting bias-capturing features into the training process. This approach aims to create bias-neutral representations by incorporating features that encode bias-inducing attributes, thus preventing the model from relying on these biases during training. 


The core idea is to divide the training loss between bias-aligned and bias-conflicting samples and study their behavior. To mitigate the loss spike due to underfitting on bias-aligned samples, the authors introduce the attributes themselves as features in the final layer, which mitigates the loss spikes and leads to proper training.

strengths:
- The paper is clear and well-written.

- The core idea seems interesting and is supported by various experiments.

weaknesses:
The idea of Bias Injection to Mitigate Bias has been used in the following work [1].

Although the idea is that a bias injection module, can prevent the loss spike. However, when the loss is forced to be zero, it needs to overcorrect the bias injection module, does it lead to correct features?

For the bias-aligned examples, the network can probably take the shortcut. Hence, the learning needs to happen just which Bias Corrected samples. In case B_c >> B_a won’t it affect the learning of diverse features? It would be great if the authors could clarify this aspect more.


The introduction section mentions that the method is more suitable for real-world. However, on closely examining the method, I found that BAdd also requires knowing the bias attributes. Could the authors please clarify on this aspect?

questions:
Please see the questions in the weakness section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732359951718

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
Dear Reviewers,

We sincerely hope our responses have clarified your concerns. We understand that the discussion period may be a busy time for you, but we wanted to kindly follow up to check if there are any further points you would like us to address.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733145661126

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
Dear Reviewers,

We would like to inform you that we have uploaded a revised version of the manuscript based on your valuable feedback.

We would be grateful if you could let us know if there are any remaining points or additional concerns that we should address during this discussion period.

Thank you once again for your time and effort in reviewing our work.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733310671196

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of the reviewers' feedback and our corresponding responses

comment:
We sincerely thank the reviewers for the time and effort they dedicated to reviewing our work. We are confident that our responses have sufficiently addressed the concerns raised. Below, we provide a concise summary of the reviewers' feedback and our corresponding responses:

**Reviewer iTN8** requested clarification on the concept of bias injection and the role of bias intensity in the data. In our response, we provided a detailed explanation of how our method facilitates learning unbiased features and showed that BAdd achieves consistent performance across varying levels of data bias, as illustrated in Table 13 of the paper's appendix.

**Reviewers iTN8, JPua, and cPAJ** raised concerns about BAdd’s reliance on a classifier or labels for bias-related attributes. To address this, we first presented a taxonomy of relevant literature, categorizing methods based on bias-label awareness and the number of biases addressed. We highlighted that mitigating multiple biases is an open and very challenging problem, even for bias-label aware approaches. Additionally, we emphasized BAdd’s flexibility compared to other methods due to its use of a bias-capturing model. Furthermore, while bias identification is beyond the scope of this work, we discussed how such methods could be integrated with BAdd to infer bias labels when they are unknown. Finally, we conducted new experiments to assess how the reliability of bias labels impacts BAdd’s performance. The results showed that BAdd maintains competitive performance even when the bias labels are of low quality.
Reviewer JPua inquired about the applicability of BAdd to non-visual data. Our response outlined the technical requirements for applying BAdd to other modalities, while noting that this work specifically focuses on visual biases. Nonetheless, we expressed our willingness to include baseline experiments in the camera-ready version to inspire researchers in other domains to employ BAdd for their preferred data modalities.

**Reviewer JPua** asked for clarification on integrating the bias-detecting classifier into the training process. In our response, we explained that the bias-capturing model functions as a feature extractor, independent of the main model’s training. To enhance clarity, we included a new figure (Figure 7) in the revised paper illustrating the training pipeline.

**Reviewer e7ck** requested a detailed comparison between BAdd and FLAC. In our response, we provided a thorough description of both methods, highlighting their fundamental differences.

**Reviewer e7ck** suggested testing BAdd using additional architectures. Our response involved experiments with various architectures, including CNNs and transformers, demonstrating BAdd’s consistent performance.

**Reviewers e7ck and cPAJ** asked for a computational complexity analysis. In our response, we presented a detailed analysis of FLOPs, showing minimal overhead during training and equivalent inference costs compared to baseline (biased) models.

**Reviewer e7ck** requested experiments analyzing the impact of batch size on BAdd’s performance. We conducted these experiments and included results demonstrating that BAdd maintains consistent performance across varying batch sizes.

**Reviewer JPua** pointed out a minor inconsistency in the text (line 728), which we corrected in the revised version.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper received mixed reviews. The reviewers appreciated the interesting and model-agnostic approach to debiased training, its consistent improvement across multiple datasets, and extensive experiments and analyses. However, they at the same time raised concerns with the demand for manually annotated (well-represented) bias attributes for training data (iTN8, JPua, cPAJ), incremental novelty (iTN8, e7ck), a potential over-correction issue caused by the proposed method (iTN8), lack of empirical validation for the versatility of the method (e7ck), additional space-time complexity imposed by the method (e7ck, cPAJ), limited ablation study (e7ck), limited analysis on real-world biased data (cPAJ), and some clarity issues (JPua). 

The authors' rebuttal and subsequent responses in the discussion period address some of these concerns but failed to fully assuage all of them. After the discussion period, Reviewer iTN8 still had concerns about the manual annotation of multiple bias attributes for training data and the potential technical issue of the proposed method; the former was pointed out by other two reviewers too, they did not come back for discussion though. The AC agrees with the authors that the manual multi-bias labeling can be execused regarding the fact that the task of mitigating multiple biases is extremely challenging. However, the authors are strongly encouraged to at least report results with existing bias discovery methods (e.g., [M2]) to demonstrate that the proposed method is ready to be applied to real-world complex scenarios as the authors argued in the paper; this is also essential since a few prior methods mitigate multiple biases with no manual bias labels [M4, M5], and because these methods exist, someone might think that the authors' claim in the rebuttal are overstated. Moreover, considering that the paper strongly emphasizes the ability of the proposed method to mitigate multiple biases as its contribution, experiments with multiple biases need to be strengthened. Currently, the datasets used for the purpose, except the CelebA variant, are all artifical (relevant to the concern of Reviewer cPAJ on limited analysis on real-world biased data), and details of the CelebA variant (e.g., the number of bias attributes and their populations) have not been reported in either the main paper or the appendix although such details are crucial to examine if the benchmark is valid. In this context, the AC strongly encourages the authors to adopt existing benchmarks and/or evaluation protocols of previous work on mitigating multiple biases [M1, M2, M3] in addition to the UrbanCar dataset; of course, the novelty and contribution of the proposed method compared to the all aforementioned papers [M1, M2, M3, M4, M5] should be clearly discussed in the paper, while the current version does not.

Putting these together, the AC considers that the remaining concerns outweigh the positive comments and the rebuttal, and thus regrets to recommend rejection. The authors are encouraged to revise the paper with the comments by the reviewers and the AC, and submit to an upcoming conference.


[M1] Men Also Do Laundry: Multi-Attribute Bias Amplification, ICML 2023

[M2] Discover and Mitigate Multiple Biased Subgroups in Image Classifiers, CVPR 2024

[M3] Improving Robustness to Multiple Spurious Correlations by Multi-Objective Optimization, ICML 2024

[M4] ExMap: Leveraging Explainability Heatmaps for Unsupervised Group Robustness to Spurious Correlations, CVPR 2024

[M5] Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias, AISTATS 2024