BAdd: Bias Mitigation through Bias Addition

## Abstract

Abstract Computer vision (CV) datasets often exhibit biases that are perpetuated by deep learning models. While recent efforts aim to mitigate these biases and foster fair representations, they fail in complex real-world scenarios. In particular, existing methods excel in controlled experiments involving benchmarks with single-attribute injected biases, but struggle with multi-attribute biases being present in well-established CV datasets. Here, we introduce BAdd, a simple yet effective method that allows for learning fair representations invariant to the attributes introducing bias by incorporating features representing these attributes into the backbone. BAdd is evaluated on seven benchmarks and exhibits competitive performance, surpassing state-of-the-art methods on both single- and multi-attribute benchmarks. Notably, BAdd achieves +27.5% and +5.5% absolute accuracy improvements on the challenging multi-attribute benchmarks, FB-Biased-MNIST and CelebA, respectively.

## 1 Introduction

Convolutional Neural Networks (CNNs) have demonstrated impressive capabilities, as evidenced by groundbreaking research across various Computer Vision (CV) tasks [, , ].
However, a concerning issue has emerged alongside these advancements: the potential for bias in AI systems, disproportionately impacting specific groups [, , ].
Specifically, when AI systems base their decisions on attributes like age, gender, or race,
they become discriminatory. Considering the profound impact AI decisions can have on individuals’ lives, such biases should be eliminated from any decision-making system [, , , , ].

For AI systems processing visual data, bias often originates from the quality of the datasets used for training [].
One of the main types of bias they exhibit is the presence of stereotypes, which means that the datasets are constructed in such a way that specific groups of people are largely associated with certain attributes (e.g., women are illustrated wearing earrings in the majority of images). When such data is used to train Deep Learning (DL) models, these biases can act as “shortcuts”, leading the model to prioritize irrelevant attributes in its decision-making process []. Motivated by this issue, sophisticated approaches have been proposed to enable learning fair representations that are independent of the biased attributes [, , ].
Such methods often leverage labels associated with protected attributes to guide model training towards learning fair representations [, , , , , ]. Techniques like adversarial training [, ], regularization [, ], and fairness-aware loss functions [, ] are commonly employed.
While these approaches aim to mitigate biases in established CV datasets, they largely rely on evaluation benchmarks that do not reflect the real-world complexities encountered in practice, as they typically comprise (i) uniformly distributed and artificially generated single-attribute biases, found in datasets like Biased-MNIST [] and Corrupted-CIFAR10 [];
(ii) enforced biases through data selection from established datasets, such as UTKFace [] and CelebA [].
However, established CV datasets, such as CelebA, exhibit vastly different properties in terms of biases:

- •
Multiple attributes contribute to creating complex bias patterns. For instance, in CelebA, attributes like WearingEarrings, WearingLipstick, HeavyMakeup, and BlondHair are all highly associated with gender.
- •
The bias intensity also differs across different target classes. In the example of CelebA, the BlondHair attribute has a 23.9% and 2.1% co-occurrence with females and males, respectively.

These fundamental differences between existing benchmarks and real-world datasets significantly complicate the task of mitigating bias, leading to significantly reduced performance of most existing approaches.

In this paper, we recognize the need to enhance the applicability of fairness-aware CV models in complex application settings and we introduce BAdd, a simple yet effective methodology designed to mitigate bias stemming from single or multiple attributes effectively. Particularly, the proposed approach suggests that the injection of bias-capturing features into the penultimate layer’s output enables learning representations invariant to the bias-capturing features (see Fig. 1).
At the core of BAdd lies a core insight into the mechanism by which bias is introduced to the DL models during training via the minimization of the loss function. The model optimizes its parameters by taking advantage of biases present in the data, as doing so reduces the overall loss. Consequently, the model learns to prioritize features associated with the biased attributes, reinforcing and perpetuating the bias within its representations.
The intentional inclusion of bias-capturing features within the model’s features ensures that the model can effectively encode the biases without them exerting undue influence on the loss function optimization. In essence, BAdd decouples the learning of biased features from the optimization process and thus allows for learning fair representations. BAdd outperforms state-of-the-art methods on a wide range of experiments involving four datasets with single attribute biases (i.e., Biased-MNIST, Biased-UTKFace, Waterbirds, and Corrupted-CIFAR10) and three datasets with multi-attribute biases (i.e., FB-Biased-MNIST, UrbanCars, and CelebA). Where BAdd shines is on datasets involving multi-attribute biases, where it outperforms the state of the art by +27.5% and +5.5% absolute accuracy improvements on FB-Biased-MNIST and CelebA, respectively.
In summary, the paper makes the following contributions:
(i) we introduce BAdd, an effective methodology for learning fair representations concerning one or more protected attributes by incorporating bias-capturing features into the model’s representations (ii) providing an extensive evaluation involving seven benchmarks, showcasing the superiority of BAdd on both single- and multi-attribute bias scenarios.

## 2 Related Work

#### Fairness-aware image benchmarks.

Most standard benchmarks for evaluating bias mitigation techniques in CV involve single-attribute artificially generated biases. Biased-MNIST [], derived from the original MNIST dataset, associates each digit with a specific colored background. Similarly, Corrupted-CIFAR10 [] introduces biased textures across its classes. The Waterbirds [] dataset, constructed by cropping birds from the CUB-200 [] dataset and transferring them onto backgrounds from the Places dataset [], introduces biases through correlating bird species with certain backgrounds (i.e., habitat types).
On the other hand, datasets like Biased-UTKFace [] and Biased-CelebA [] are carefully selected subsets of UTKFace [] and CelebA [], respectively. These subsets are designed to showcase an association of 90% between specific attributes, such as gender and race.
However, all these benchmarks share a crucial limitation: they are far from emulating the complexities of realistic scenarios, as they typically exhibit uniformly distributed single attribute biases.
To address this limitation, recent works focus on introducing benchmarks that involve multi-attribute biases, such as Biased-MNIST variations [, , ] and UrbanCars []. Specifically, UrbanCars introduces a multi-attribute bias setting by incorporating biases related to both background and co-occurring objects and the task is to classify car body types into urban or country categories.

In addition to the above benchmarks, in this paper, we create a variation of Biased-MNIST, termed FB-Biased-MNIST, which builds on the bias introduced by background color in Biased-MNIST by injecting an additional bias through foreground color. Furthermore, we consider a benchmark that utilizes the original, unmodified CelebA dataset but focuses on evaluating performance against the most prominent bias-inducing attributes within the dataset. This allows for evaluating fairness-aware methodologies on multiple biases in a more realistic setting - without artificially injected biases.

#### Fairness-aware approaches.

Efforts on learning fair representations using biased data encompass techniques like ensemble learning [, ], contrastive learning [, ], adversarial frameworks [, , , , , ], and regularization terms [, , , ].
For instance, the Learning Not to Learn (LNL) approach [] penalizes models if they predict protected attributes, while the Domain-Independent (DI) approach [] introduces the usage of domain-specific classifiers to mitigate bias. Entangling and Disentangling deep representations (EnD) [] suggests a regularization term that entangles or disentangles feature vectors w.r.t. their target and protected attribute labels. FairKL [] and BiasContrastive-BiasBalance (BC-BB) [] are contrastive learning-based approaches that try to mitigate bias by utilizing the pairwise similarities of the samples in the feature space. Finally, there are several works that can be employed without utilizing the protected attribute labels, such as Learned-Mixin (LM) [], Rubi [], ReBias [], Learning from Failure (LfF) [], and FLAC []. The latter achieves state-of-the-art performance by utilizing a bias-capturing classifier and a sampling strategy that effectively focuses on the underrepresented groups.
It is worth noting that methodologies for distributionally robust optimization [, , , , , ] are highly relevant, as they aim at mitigating biases arising from spurious correlations in the training data.
Recent efforts suggest approaches that are designed to be effective in scenarios involving multiple biases. For instance, OccamNets [] propose modifying the network architecture to impose inductive biases that make the network robust to dataset bias. Furthermore, the Last Layer Ensemble (LLE) [] employs multiple augmentations to eliminate different biases (i.e., one type of augmentation for each type of bias). However, it requires extensive preprocessing (e.g., object segmentation), which makes it challenging or even infeasible to apply to different CV datasets. On the other hand, BAdd is a simple yet effective approach that can be easily applied to any network architecture that processes images, as well as to any CV dataset.

## 3 Methodology

### 3.1 Problem formulation

Consider a dataset $\mathcal{D}$ comprising training samples $(\mathbf{x}^{(i)},y^{(i)})$, where $\mathbf{x}^{(i)}$ represents the input sample and $y^{(i)}$ belongs to the set of target labels $\mathcal{Y}$. Let $h(\cdot)$ denote a model trained on $\mathcal{D}$ and $\mathbf{h}$ the model representation (e.g., output of penultimate model layer). Let also $\mathcal{T}$ be the domain of tuples of $q$ protected attributes (e.g., the set $t=\left\{\text{\emph{male}},25,\text{\emph{black}}\right\}$ for protected attributes “gender”, “age” and “race”). The protected attributes should not be used to predict the targets in $\mathcal{Y}$, either because there is no causal relationship between them, or due to ethical or legal constraints. In addition, we consider a bias-capturing model $b(\cdot)$, which has been trained to predict the value of protected attribute(s) $t\in\mathcal{T}$ from $\mathbf{x}$, and the corresponding representation $\mathbf{b}$.

We define $\mathcal{D}$ as biased with respect to the protected attributes in $\mathcal{T}$ if there is a significant imbalance in the co-occurrence of certain values in $\mathcal{Y}$ with a value or a combination of values of protected attributes in $\mathcal{T}$.
Within a batch $\mathcal{B}$, samples exhibiting the dataset bias are termed bias-aligned ($\mathcal{A}$), while those that deviate from it are referred to as bias-conflicting ($\mathcal{C}$).
The set $\mathcal{D}$ is assumed to include at least some bias-conflicting examples.
Using such a biased dataset for training often introduces model bias, by leading $h$ to rely on features that encode information related to $t$. Thus, the objective is to mitigate these dependencies between representations $\mathbf{h}$ and $\mathbf{b}$, and consequently learn fair representations.

### 3.2 Method description

When training a model $h(\cdot)$ on a biased dataset $\mathcal{D}$, it often prioritizes learning features related to the protected attributes instead of features directly characterizing the target class. As already discussed, this phenomenon arises if there is a high correlation between protected attributes and targets and if the protected attribute’s visual characteristics are easier to capture than the visual characteristics of the target [].
Below, we delve into the details behind a vanilla model’s limited ability to mitigate this kind of bias and explain how the proposed approach addresses this limitation.

First, let us consider the Cross-Entropy loss on a batch of samples $\mathcal{B}=\mathcal{A}\cup\mathcal{C}$:

$$ $\begin{split}\mathcal{L}=&-\frac{1}{N}\sum_{i\in\mathcal{B}}\sum_{k=1}^{K}y_{k }^{(i)}\log\hat{y}_{k}^{(i)}=\\ &-\frac{1}{N}\sum_{i\in\mathcal{A}}\sum_{k=1}^{K}y_{k}^{(i)}\log\hat{y}_{k}^{( i)}-\frac{1}{N}\sum_{i\in\mathcal{C}}\sum_{k=1}^{K}y_{k}^{(i)}\log\hat{y}_{k}^ {(i)}=\\ &\mathcal{L}_{\mathcal{A}}+\mathcal{L}_{\mathcal{C}},\end{split}$ (1) $$

where $N$ is the number of samples within a batch, and $K$ the number of target classes. The predictions $\hat{y}_{k}^{(j)}$ are computed as follows:

$$ $\hat{y}_{k}^{(j)}=\sigma_{k}(\mathbf{z}(\mathbf{x}^{(j)};\boldsymbol{\theta})),$ (2) $$

where $j$ the index of input sample $\mathbf{x}^{(j)}$, $\boldsymbol{\theta}$ the learnable parameters of model $h(\cdot)$ and $\sigma_{k}$ the $k$-th class probability after applying the softmax function on the logits $\mathbf{z}(\mathbf{x}^{(j)};\boldsymbol{\theta})$.

Given that $||\mathcal{A}||>>||\mathcal{C}||$, we can assume that there exists a point in the training process at which
the model has learned to be accurate on the bias-aligned samples $\mathcal{A}$ misguidedly relying on protected attributes’ features, so that $\mathcal{L}_{\mathcal{A}}\approx 0$, while at the same time $\mathcal{L}_{\mathcal{C}}>>0$.
Consequently, backpropagating the gradients of $\mathcal{L}$ will update the parameters $\boldsymbol{\theta}$ in a way that steers the model towards accurately predicting the bias-conflicting samples $\mathcal{C}$ in order to further reduce $\mathcal{L}$, which stops reliance of $h(\cdot)$ on the protected attributes.
The major limitation of a vanilla model is directly connected to the loss behavior when the model processes the next mini-batch. In particular, the step the model makes towards correctly predicting the samples in $\mathcal{C}$, thus reducing $\mathcal{L}_{\mathcal{C}}$, severely affects the loss w.r.t. the bias-aligned samples, which is now $\mathcal{L}_{\mathcal{A}}>>0$, as $h(\cdot)$ relies less on the protected attributes and at the same time it is impossible to learn to encode the target with only one batch of $||C||$ bias-conflicting samples. This leads to a loss spike for the bias-aligned samples that in the next iteration will restore the model’s parameters $\boldsymbol{\theta}$ to their initial state (encoding the protected attributes’ features) in order to again achieve a much lower $\mathcal{L}$. Figure 2 illustrates the loss w.r.t. the bias-aligned samples for approximately 12k training steps. As one may observe, the vanilla model tends to present these loss spikes that prevent the model from mitigating the bias.

Figure: Figure 2: Vanilla vs BAdd: Bias-aligned samples loss on Biased-MNIST dataset.
Refer to caption: x2.png

To support these arguments in more concrete terms, let us consider the derivative of the loss of Eq. (1) with respect to a parameter $\theta_{0}$ for the $i$-th sample:

$$ $\begin{split}\frac{\partial\mathcal{L}^{(i)}}{\partial\theta_{0}}=&y_{\kappa} \frac{\partial\log\sigma_{\kappa}(\mathbf{z}(\mathbf{x}^{(i)};\boldsymbol{ \theta}))}{\partial\theta_{0}}=\\ &y_{\kappa}\frac{1}{\sigma_{\kappa}(\mathbf{z}(\mathbf{x}^{(i)};\boldsymbol{ \theta}))}\frac{\partial\sigma_{\kappa}(\mathbf{z}(\mathbf{x}^{(i)}; \boldsymbol{\theta}))}{\partial\theta_{0}},\end{split}$ (3) $$

where $\kappa$ is the correct class, according to the ground truth (i.e., $y_{\kappa}=1$). Setting $A_{0}^{(i)}=\frac{\partial\sigma_{\kappa}(\mathbf{z}(\mathbf{x}^{(i)};
\boldsymbol{\theta}))}{\partial\theta_{0}}$ the derivative for a batch becomes

$$ $\frac{\partial\mathcal{L}}{\partial\theta_{0}}=-\frac{1}{N}\Big{(}\sum_{i: \mathbf{x}^{(i)}\in\mathcal{A}}\frac{1}{\sigma_{\kappa}^{(i)}}A_{0}^{(i)}+\sum _{j:\mathbf{x}^{(j)}\in\mathcal{C}}\frac{1}{\sigma_{\kappa}^{(j)}}A_{0}^{(j)} \Big{)}$ (4) $$

After the model has learned to predict the targets based on the protected attributes, $\sigma_{\kappa}^{(i)}$ is large (close to 1) while $A_{0}^{(i)}$ is small, as $h(\cdot)$ already correctly predicts samples in $\mathcal{A}$. In contrast, $\sigma_{\kappa}^{(j)}$ is small while $A_{0}^{(j)}$ is large. The model update therefore strongly depends on the samples in $\mathcal{C}$. After the update step, however, $\sigma_{\kappa}^{(i)}$ becomes smaller, $A_{0}^{(i)}$ becomes larger and given that $||\mathcal{A}||>>||\mathcal{C}||$, the derivative is now dominated by samples in $\mathcal{A}$, and the parameters revert back to their previous values. In other words, any progress the model makes towards reducing its bias is counteracted by the loss function, which is lower when the model focuses on the easier-to-learn, biased samples. This essentially traps the model in a vicious cycle being condemned to encode the protected attributes instead of the targets.

To address this limitation, BAdd proposes combining the features capturing the protected attribute, represented by $\mathbf{b}$, with the main model’s representation, represented by $\mathbf{h}$. This combined representation $\mathbf{h}+\mathbf{b}$ is then fed to the final classification layer.(^1^11The $\mathbf{b}$ features can be obtained either by using a bias-capturing classifier (i.e., protected attribute extractor) or by using a regressor that encodes the protected attribute labels into a feature vector matching the dimension of $\mathbf{h}$. The former is considered less prone to overfitting due to its diversity in the feature space, while the latter is easier to implement as it does not require any additional training procedure.)
Thus, during training, model predictions are computed as

$$ $\hat{y}_{k}^{(j)}=\sigma_{k}(\mathbf{W}(\mathbf{h}(\mathbf{x}^{(j)}; \boldsymbol{\theta})+\mathbf{b}(\mathbf{x}^{(j)}))+\boldsymbol{\rho}),$ (5) $$

where $\mathbf{W}$ and $\boldsymbol{\rho}$ are the parameters of the last linear layer of $h(\cdot)$.
By incorporating the protected attribute features $\mathbf{b}$ into the training, we equip the model with the necessary information to consistently predict accurately the bias-aligned samples.
This means that the $\mathcal{L}_{\mathcal{A}}$ values are consistently close to 0, preventing the loss spikes (see Fig. 2), and thus enabling the $\mathbf{h}$ to encode the targets rather than the protected attributes, without having a negative impact on the loss of the bias-aligned samples. In terms of the training process implied by Eq. (4), the addition of $\mathbf{b}$ entails invariably large $\sigma_{\kappa}^{(i)}$ and small $A_{0}^{(i)}$, thus forcing the model updates to depend on the samples of $\mathcal{C}$ consequently eliminating the effect of bias-aligned samples. After training $h(\cdot)$, fine-tuning the classification layer with only the $\mathbf{h}$ features is required, as $\mathbf{b}$ should not be involved in inference, i.e. the predictions are derived using the following:

$$ $\hat{y}_{k}^{(j)}=\sigma_{k}(\mathbf{W}\mathbf{h}(\mathbf{x}^{(j)};\boldsymbol {\theta})+\boldsymbol{\rho}),$ (6) $$

## 4 Experimental setup

### 4.1 Datasets

#### Single-attribute bias.

Biased-MNIST [], derived from the original MNIST dataset [], serves as a benchmark for assessing bias mitigation methods. It features digits with colored backgrounds, introducing bias through the association of each digit with a specific color. The degree of bias, represented by the probability $q$, determines the strength of this association. We consider four variations of Biased-MNIST with $q$ values of 0.99, 0.995, 0.997, and 0.999, as commonly used in previous works.
Biased-CelebA [] is a subset of the well-known CelebA dataset consisting of facial images annotated with 40 binary attributes, which considers gender as the protected attribute, while HeavyMakeup and BlondHair serve as the target labels. Similarly, the Biased-UTKFace [] dataset is a subset of the UTKFace dataset comprising facial images annotated with gender, race, and age labels. Gender is the target label, with race or age considered as protected attributes. In both Biased-CelebA and Biased-UTKFace, the enforced correlation between the target and protected attributes is 90%.
As for the Corrupted-CIFAR10 dataset introduced in [], it consists of 10 classes with texture-related biases uniformly distributed in the training data. This dataset provides four correlation ratios: 0.95, 0.98, 0.99, and 0.995, offering a range of bias strengths for evaluation purposes. Finally, the Waterbirds dataset demonstrates a co-occurrence of 95% between waterbirds (or landbirds) and aquatic environments (or terrestrial environments) as background.

**Table 1: Fairness of a vanilla gender classifier trained on default CelebA w.r.t. potentially biased attributes.**
| Attribute | Accuracy |  |
| --- | --- | --- |
| Unbiased | Bias-conflicting |  |
| Smiling | 98.6 | 98.5 |
| WearingNecklace | 98.1 | 97.3 |
| WearingEarrings | 97.7 | 96.3 |
| BlondHair | 96.9 | 94.9 |
| Eyeglasses | 96.5 | 94.5 |
| WearingLipstick | 95.2 | 91.1 |
| HeavyMakeup | 93.0 | 86.7 |

#### Multi-attribute bias.

Similar to the Biased-MNIST, we create FB-Biased-MNIST, an extension that enhances the bias introduced by the background color in Biased-MNIST. Specifically, FB-Biased-MNIST introduces an additional layer of bias by injecting foreground color biases into the dataset. Considering the increased complexity of this dataset compared to the Biased-MNIST, we opt for lower $q$ values, namely 0.9, 0.95, and 0.99. Furthermore, the UrbanCars dataset is an artificially generated dataset that shows a 95% co-occurrence between car body types and both the background and certain objects relevant to urban or rural regions.
We also assess the performance of bias mitigation methods on established CV datasets devoid of injected biases, whether artificially generated or introduced through subset selection. To this end, we utilize the default CelebA [] dataset. To properly select the attributes introducing bias, we consider the performance disparities of a standard gender classifier trained on CelebA with respect to various potentially biased attributes. Subsequently, we identify the top two attributes with most significant impact on the model’s performance. As shown in Tab. 1, WearingLipstick and HeavyMakeup are those two attributes.

**Table 2: CelebA gender bias degree $q$ w.r.t. the attributes WearingLipstick and HeavyMakeup.**
| Attribute | $q$ |  |
| --- | --- | --- |
| Females | Males |  |
| WearingLipstick | 0.806 | 0.994 |
| HeavyMakeup | 0.663 | 0.997 |

### 4.2 Model architecture

In experiments involving the MNIST-based datasets, namely Biased-MNIST and FB-Biased-MNIST, we utilize a simple CNN architecture outlined in [], which comprises four convolutional layers with 7$\times$7 kernels and a classification head. For the experiments involving the Biased-UTKFace, Corrupted-CIFAR10, and CelebA datasets, we adopt the ResNet-18 architecture []. For Waterbirds and UrbanCars datasets, we use Resnet-50 networks.

### 4.3 Implementation details and evaluation protocol

We employ the Adam optimizer with a 0.001 initial learning rate, which is divided by 10 every 1/3 of the training epochs. Batch size is fixed at 128 and weight decay is set to $10^{-4}$.
Following previous works [, , ], we train the models on Biased-MNIST and FB-Biased-MNIST datasets for 80 epochs. For Biased-UTKFace and CelebA, training duration is set to 20 and 40 epochs, respectively. As for the Corrupted-CIFAR10 dataset, models are trained for 100 epochs using a cosine annealing scheduler. For the Waterbirds and UrbanCars datasets, we do not use a learning rate scheduler, and the models are trained for 300 and 100 epochs, respectively.
Following the initial training phase, the classification head of all models is fine-tuned for an additional 20 epochs.
All experiments were conducted on a single NVIDIA RTX-3090 Ti GPU and repeated for 5 different random seeds.
Regarding the evaluation protocol, the test sets used for Biased-MNIST and FB-Biased-MNIST have $q=0.1$ to ensure unbiased evaluation. For Biased-UTKFace and CelebA datasets, we utilize both bias-conflicting and unbiased accuracy as in [, ]. The official unbiased test set is utilized for Corrupted-CIFAR10. For the Waterbirds, we employ the average accuracy between different groups and the Worst-Group (WG) accuracy. Finally, for the UrbanCars dataset, we measure the In Distribution Accuracy (I.D. Acc) which is the
weighted average accuracy w.r.t. the different groups, where the correlation ratios are the weights. The I.D. Acc is used as a baseline to measure the accuracy drop with respect to the background (BG Gap), co-occurring objects (CoObj Gap), and both the background and co-occurring objects (BG+CoObj Gap).

Figure: (a) Mean activation values of the first convolutional layer on sample backgrounds.
Refer to caption: x3.png

## 5 Results

### 5.1 Single attribute biases

Table 3 showcases the performance of BAdd against nine competing methods. As one may observe, the proposed approach consistently surpasses state-of-the-art, demonstrating accuracy improvements ranging from 0.1% to 0.8% across different $q$ values. Furthermore, Fig. 3 illustrates the mean activations in image regions where bias occurs alongside the corresponding classification errors for both the Vanilla and BAdd approaches. As one may observe, the proposed method effectively reduces activations in areas where bias appears, leading to significant improvements in classification performance. This effect is particularly pronounced in experiments with $q=0.999$, where the Vanilla approach struggles to mitigate the impact of the biased attribute. Furthermore, the efficacy of BAdd to learn representations that are independent of the protected attribute is illustrated in Tab. 4. Specifically, Tab. 4 shows the mean pairwise cosine similarity between 10 variations of each Biased-MNIST test sample, where each sample variation has a different background color (i.e., protected attribute). As one may observe, BAdd showcases similarity values close to 1 for all correlation ratios, which is not the case for the vanilla model that cannot maintain high similarities when the correlation ratio increases (e.g., 0.416 similarity for $q=0.999$).

**Table 3: Evaluation on Biased-MNIST for different bias levels.**
| Method | $q$ |  |  |  |
| --- | --- | --- | --- | --- |
| 0.99 | 0.995 | 0.997 | 0.999 |  |
| Vanilla | 90.8$\pm$0.3 | 79.5$\pm$0.1 | 62.5$\pm$2.9 | 11.8$\pm$0.7 |
| LM [] | 91.5$\pm$0.4 | 80.9$\pm$0.9 | 56.0$\pm$4.3 | 10.5$\pm$0.6 |
| Rubi [] | 85.9$\pm$0.1 | 71.8$\pm$0.5 | 49.6$\pm$1.5 | 10.6$\pm$0.5 |
| ReBias [] | 88.4$\pm$0.6 | 75.4$\pm$1.0 | 65.8$\pm$0.3 | 26.5$\pm$1.4 |
| LfF [] | 95.1$\pm$0.1 | 90.3$\pm$1.4 | 63.7$\pm$20.3 | 15.3$\pm$2.9 |
| LNL [] | 86.0$\pm$0.2 | 72.5$\pm$0.9 | 57.2$\pm$2.2 | 18.2$\pm$1.2 |
| EnD [] | 94.8$\pm$0.3 | 94.0$\pm$0.6 | 82.7$\pm$0.3 | 59.5$\pm$2.3 |
| BC-BB [] | 95.0$\pm$0.9 | 88.2$\pm$2.3 | 82.8$\pm$4.2 | 30.3$\pm$11.1 |
| FairKL [] | 97.9$\pm$0.0 | 97.0$\pm$0.0 | 96.2$\pm$0.2 | 90.5$\pm$1.5 |
| FLAC [] | 97.9$\pm$0.1 | 96.8$\pm$0.0 | 95.8$\pm$0.2 | 89.4$\pm$0.8 |
| BAdd | 98.1$\pm$0.2 | 97.3$\pm$0.2 | 96.3$\pm$0.2 | 91.7$\pm$0.6 |

**Table 4: Mean pairwise cosine similarity between 10 variations of each Biased-MNIST test sample, where each sample variation has a different background color.**
| Method | $q$ |  |  |  |
| --- | --- | --- | --- | --- |
| 0.99 | 0.995 | 0.997 | 0.999 |  |
| Vanilla | 0.889 | 0.854 | 0.811 | 0.416 |
| BAdd | 0.985 | 0.985 | 0.980 | 0.973 |

Table 5 illustrates the performance comparison of BAdd against the competing methods on the Biased-UTKFace dataset, where race and age are considered as protected attributes. Across both protected attributes, the proposed approach outperforms the second best-performing competing methods on bias-conflicting samples, achieving improvements of +1.1% and +1.9%, respectively. For the unbiased performance, BAdd exhibits only marginal differences compared to the state-of-the-art methods, with increases of 0.2% and decreases of 0.3%, respectively.

**Table 5: Evaluation of the proposed method on Biased-UTKFace for two different protected attributes, namely race and age, with gender as the target attribute.**
|  | Bias |  |  |  |
| --- | --- | --- | --- | --- |
| Method | Race | Age |  |  |
|  | Unbiased | Bias-conflicting | Unbiased | Bias-conflicting |
| Vanilla | 87.4$\pm$0.3 | 79.1$\pm$0.3 | 72.3$\pm$0.3 | 46.5$\pm$0.2 |
| LNL [] | 87.3$\pm$0.3 | 78.8$\pm$0.6 | 72.9$\pm$0.1 | 47.0$\pm$0.1 |
| EnD [] | 88.4$\pm$0.3 | 81.6$\pm$0.3 | 73.2$\pm$0.3 | 47.9$\pm$0.6 |
| BC-BB [] | 91.0$\pm$0.2 | 89.2$\pm$0.1 | 79.1$\pm$0.3 | 71.7$\pm$0.8 |
| FairKL [] | 85.5$\pm$0.7 | 80.4$\pm$1.0 | 72.7$\pm$0.2 | 48.6$\pm$0.6 |
| FLAC [] | 92.0$\pm$0.2 | 92.2$\pm$0.7 | 80.6$\pm$0.7 | 71.6$\pm$2.6 |
| BAdd | 92.2$\pm$0.2 | 93.3$\pm$0.2 | 80.3$\pm$0.8 | 73.6$\pm$1.0 |

In the final single-attribute evaluation scenario, biases stemming from image background or textures are considered. As for the texture biases, the results obtained on the Corrupted-CIFAR10 dataset for four different bias ratios are summarized in Tab. 6. Given the complexity of training a bias-capturing classifier in this scenario, BAdd is implemented using a linear regressor to obtain feature vectors of the desired size from one-hot vectors representing the texture labels. Notably, even without utilizing a bias-capturing classifier, BAdd consistently outperforms state-of-the-art across all Corrupted-CIFAR10 variations. Specifically, it achieves improvements of 6.5%, 3.1%, 3.4%, and 1.6% for correlation ratios of 0.95, 0.98, 0.99, and 0.995, respectively. Table 7, demonstrates the performance of BAdd on the Waterbirds dataset compared to the state-of-the-art methods for distributionally robust optimization. Here, BAdd reaches the state-of-the-art WG accuracy, i.e., 92.9%, and demonstrates competitive average accuracy, i.e., 93.6%.

**Table 6: Evaluation on Corrupted-Cifar10.**
| Method | $q$ |  |  |  |
| --- | --- | --- | --- | --- |
| 0.95 | 0.98 | 0.99 | 0.995 |  |
| Vanilla | 39.4$\pm$0.6 | 30.1$\pm$0.7 | 25.8$\pm$0.3 | 23.1$\pm$1.2 |
| EnD [] | 36.6$\pm$4.0 | 34.1$\pm$4.8 | 23.1$\pm$1.1 | 19.4$\pm$1.4 |
| ReBias [] | 43.4$\pm$0.4 | 31.7$\pm$0.4 | 25.7$\pm$0.2 | 22.3$\pm$0.4 |
| LfF [] | 50.3$\pm$1.6 | 39.9$\pm$0.3 | 33.1$\pm$0.8 | 28.6$\pm$1.3 |
| FairKL [] | 50.7$\pm$0.9 | 41.5$\pm$0.4 | 36.5$\pm$0.4 | 33.3$\pm$0.4 |
| FLAC [] | 53.0$\pm$0.7 | 46.0$\pm$0.2 | 39.3$\pm$0.4 | 34.1$\pm$0.5 |
| BAdd | 59.5$\pm$0.5 | 49.1$\pm$0.3 | 42.7$\pm$0.2 | 35.7$\pm$0.6 |

**Table 7: Evaluation on Waterbirds.**
| Method | WG Acc. | Avg. Acc. |
| --- | --- | --- |
| JTT [] | 86.7$\pm$1.5 | 93.3$\pm$0.3 |
| DISC [] | 88.7$\pm$0.4 | 93.8$\pm$0.7 |
| GroupDro [] | 90.6$\pm$1.1 | 91.8$\pm$0.3 |
| DFR [] | 92.9$\pm$0.2 | 94.2$\pm$0.4 |
| BAdd | 92.9$\pm$0.3 | 93.6$\pm$0.2 |

### 5.2 Multi-attribute biases

As previously discussed, evaluating bias mitigation performance solely in single-attribute scenarios provides an initial assessment but fails to capture the complexities of real-world settings. In this section, we present the performance of BAdd in two multi-attribute bias evaluation setups, namely on FB-Biased-MNIST and CelebA datasets. As depicted in Tab. 8, competing methods struggle to effectively mitigate bias on the FB-Biased-MNIST dataset, while BAdd consistently outperforms the second-best performing methods by significant margins of 8%, 23%, and 27.5% for correlation ratios of 0.9, 0.95, and 0.99, respectively. Notably, even in an artificial dataset like FB-Biased-MNIST, existing approaches struggle to address multiple biases, underscoring the necessity for methodologies applicable to complex settings, such as BAdd.
Table 9 demonstrates the performance of BAdd on UrbanCars, a dataset with artificially injected bias that is much more challenging than FB-Biased-MNIST.
As observed, most compared methods struggle to address both the background and the co-occurring object biases. The only exception is LLE, which employs architectural modifications and specific bias-oriented augmentations to tackle each type of bias. However, it should be stressed that this approach requires extensive preprocessing, such as object segmentation, making its application to other CV datasets challenging or even infeasible.
Finally, an example of a real-world dataset - without artificially injected biases - is the default CelebA dataset, which exhibits multiple biases. As shown in Tab. 10, BAdd consistently improves fairness performance for both biased attributes, achieving absolute accuracy improvements of +3.5% and +5.5% for the bias-conflicting samples and +1.1% and +2.1% for the unbiased sets compared to the second-best performing methods.

**Table 8: Evaluation on FB-Biased-MNIST for different bias levels.**
| Method | $q$ |  |  |
| --- | --- | --- | --- |
| 0.9 | 0.95 | 0.99 |  |
| Vanilla | 82.5$\pm$0.8 | 57.9$\pm$1.7 | 25.5$\pm$0.6 |
| EnD [] | 82.5$\pm$1.0 | 57.5$\pm$2.0 | 25.7$\pm$0.8 |
| BC-BB [] | 80.9$\pm$2.4 | 66.0$\pm$2.4 | 40.9$\pm$3.4 |
| FairKL [] | 87.6$\pm$0.8 | 61.6$\pm$2.6 | 42.0$\pm$1.1 |
| FLAC [] | 84.4$\pm$0.8 | 63.1$\pm$1.7 | 32.4$\pm$1.1 |
| BAdd | 95.6$\pm$0.3 | 89.0$\pm$1.8 | 69.5$\pm$2.5 |

**Table 9: Evaluation on UrbanCars.**
| Method | I.D. Acc | BG Gap | CoObj Gap | BG+CoObj Gap |
| --- | --- | --- | --- | --- |
| LfF [] | 97.2 | -11.6 | -18.4 | -63.2 |
| JTT [] | 95.9 | -8.1 | -13.3 | -40.1 |
| Debian [] | 98.0 | -14.9 | -10.5 | -69.0 |
| GroupDro [] | 91.6 | -10.9 | -3.6 | -16.4 |
| DFR [] | 89.7 | -10.7 | -6.9 | -45.2 |
| LLE [] | 96.7 | -2.1 | -2.7 | -5.9 |
| BAdd | 91.0$\pm$0.7 | -4.3$\pm$0.4 | -1.6$\pm$1.0 | -3.9$\pm$0.4 |

**Table 10: Evaluation of the proposed method on CelebA for multiple attributes introducing bias, namely WearingLipstick and HeavyMakeup. Gender is the target attribute.**
|  | Biases |  |  |  |
| --- | --- | --- | --- | --- |
| Method | WearingLipstick | HeavyMakeup |  |  |
|  | Unbiased | Bias-conflicting | Unbiased | Bias-conflicting |
| Vanilla | 95.2$\pm$0.3 | 91.1$\pm$0.6 | 93.0$\pm$0.8 | 86.7$\pm$1.6 |
| EnD [] | 95.1$\pm$0.4 | 91.0$\pm$0.7 | 92.3$\pm$0.7 | 85.3$\pm$1.5 |
| BC-BB [] | 91.6$\pm$2.6 | 85.8$\pm$5.1 | 89.7$\pm$2.3 | 81.8$\pm$4.5 |
| FairKL [] | 82.7$\pm$0.4 | 74.7$\pm$0.3 | 84.4$\pm$0.9 | 77.9$\pm$1.2 |
| FLAC [] | 95.4$\pm$0.3 | 91.6$\pm$0.5 | 93.2$\pm$0.3 | 87.2$\pm$0.7 |
| BAdd | 96.5$\pm$0.2 | 95.1$\pm$0.4 | 95.3$\pm$0.5 | 92.7$\pm$1.1 |

## 6 Ablation Study

In this section, we explore the ways of integrating bias-capturing features into the training process. Table 11 presents a comparison of BAdd’s performance when the bias-capturing features are added to the main features versus when they are concatenated with them. As one may observe, the concatenation approach is much less effective than the addition. This is anticipated, as relying on $\mathbf{b}$, with non-zero corresponding weights, would perform poorly on balanced settings (random background color), while not relying on it, with $\sim 0$ corresponding weights, would be equivalent to the sub-optimal vanilla training.
Furthermore, Tab. 12 demonstrates how the selection of layer to incorporate the bias-capturing features affects the performance of BAdd. The penultimate layer yields the most favorable performance, as the shallower the selected layer, the fewer layers remain independent of the protected attributes. Consequently, the model’s capacity to learn fair features gradually diminishes.

**Table 11: Addition vs Concatenation: Biased-MNIST performance comparison between different approaches of integrating bias-capturing features.**
| Method | $q$ |  |  |  |
| --- | --- | --- | --- | --- |
| 0.99 | 0.995 | 0.997 | 0.999 |  |
| Concatenation | 91.5 | 81.7 | 70.3 | 36.5 |
| Addition | 98.1 | 97.3 | 96.3 | 91.7 |

**Table 12: BAdd performance on Biased-MNIST with $q=0.99$ when considering different layers for incorporating the bias capturing features.**
| Method | Layer |  |  |  |
| --- | --- | --- | --- | --- |
| 1st | 2nd | 3rd | 4th |  |
| BAdd | 74.6 | 85.8 | 97.7 | 98.1 |

## 7 Conclusion

In this work, we propose a method for bias mitigation in CV deep-learning models, termed BAdd. The proposed method essentially enriches the feature representations of a model with bias-capturing features in order to force the model parameters’ updates to rely only on unbiased samples, thus leading to fair representations. Through a comprehensive experimental evaluation, we show that the proposed approach surpasses the state-of-the-art in single- as well as multi-attribute bias scenarios. The main limitation of BAdd is the requirement to have access to protected attribute labels, either directly within the dataset or through another dataset where these labels are available.

## Acknowledgments

This research was supported by the EU Horizon Europe project
MAMMOth (Grant Agreement 101070285).

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: Please see the questions in the weakness section.
soundness: 2
strengths: - The paper is clear and well-written.

- The core idea seems interesting and is supported by various experiments.
summary: The document introduces BAdd, a method for mitigating bias in deep learning models by injecting bias-capturing features into the training process. This approach aims to create bias-neutral representations by incorporating features that encode bias-inducing attributes, thus preventing the model from relying on these biases during training. 


The core idea is to divide the training loss between bias-aligned and bias-conflicting samples and study their behavior. To mitigate the loss spike due to underfitting on bias-aligned samples, the authors introduce the attributes themselves as features in the final layer, which mitigates the loss spikes and leads to proper training.
weaknesses: The idea of Bias Injection to Mitigate Bias has been used in the following work [1].

Although the idea is that a bias injection module, can prevent the loss spike. However, when the loss is forced to be zero, it needs to overcorrect the bias injection module, does it lead to correct features?

For the bias-aligned examples, the network can probably take the shortcut. Hence, the learning needs to happen just which Bias Corrected samples. In case B_c >> B_a won’t it affect the learning of diverse features? It would be great if the authors could clarify this aspect more.


The introduction section mentions that the method is more suitable for real-world. However, on closely examining the method, I found that BAdd also requires knowing the bias attributes. Could the authors please clarify on this aspect?

---

confidence: 4
contribution: 4
initial_rating: 6
presentation: 3
questions: please check the weakness part, and give some explanations.
soundness: 3
strengths: 1.BAdd reduces bias effectively by adding bias-related features into the training, helping the model avoid being influenced by biased data.
2.The authors showed that BAdd works well on different datasets, with consistent improvements in various bias situations, proving that the method is scalable and works in different applications.
summary: The model proposed in this paper learns unbiased features by adding specific elements that capture potential biases during training. This ensures that the model doesn’t rely on biased information when making predictions. BAdd was tested on various datasets, including those with single and multiple biases, and showed strong improvements in reducing bias and overall model performance.
So,  this paper:
1.Introduced BAdd, an easy and effective method for learning unbiased features by incorporating bias-detecting elements into training.
2.Evaluated BAdd on several benchmarks, including different types of biased datasets, showing that it outperforms other state-of-the-art methods.
weaknesses: 1.BAdd requires a classifier or labels that identify the bias, which may not always be available. The authors could consider ways to detect and handle biases automatically without needing predefined labels.
2.The method mainly addresses visual biases, and it’s unclear if it works for other types of biases, like those in text, limiting its use beyond visual data.
3.The paper is clear, but lacks a detailed comparison to standard deep learning training. It’s not explained how the bias-detecting classifier fits into the training process.
4.In line 728, the tense is inconsistent—“were” should be changed to “are.”

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1-Describe the key differences between the BAdd and  FLAC [1].

2-Conduct experiments using different architectures, such as ViT with BAdd, to prove that the approach works properly with different architectures. Also, use the ImageNet dataset to show the approach performance with the balanced dataset.

3-Discuss the computational complexity of BAdd approach.

4-We encourage the author to do more ablation studies on the proposed approach to show consistency, such as changing the batch size.
soundness: 3
strengths: 1-	The problem is described clearly.

2-	The investigated problem is important.
summary: The paper introduces a BAdd approach to mitigate the effects of biased data during model training. The idea is to incorporate captured bias features into the final layer of the model, which helps the model be invariant for these features and create a bias-neutral feature representation.
weaknesses: 1-	The BAdd approach seems similar to FLAC [1]. 

2-	On page 3, The author claims that BAdd is easily applied to any network architecture and to any CV dataset. This claim needs to be proven.

3-	The paper does not discuss the computational complexity or scalability of the proposed approach in detail, which could be a concern for large-scale applications.

4-	Limited Ablation Studies: While the paper includes some ablation studies, more extensive ablations could strengthen the claims about the individual contributions of each component of the proposed method.

[1] Ioannis Sarridis, Christos Koutlis, Symeon Papadopoulos, and Christos Diou. Flac: Fairnessaware representation learning by suppressing attribute-class associations. arXiv preprint arXiv:2304.14252, 2023a.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: Availability of Protected Attributes: How does BAdd perform when protected attribute labels are unavailable or unreliable?

Handling Unknown Biases: In real-world applications where biases may be unknown or multifaceted, how can BAdd be adapted to mitigate biases that are not explicitly identified?
soundness: 2
strengths: Versatility: The approach is architecture-agnostic and can be easily integrated into any deep learning model without extensive modifications or pre-processing.
Visualization of Results: Use of activation maps and GradCam visualizations helps to intuitively demonstrate how BAdd shifts the model's focus away from bias-inducing features.
summary: The paper introduces BAdd (Bias Addition), a method for mitigating bias in deep learning models for computer vision tasks. Bias in datasets often arises from spurious correlations between certain attributes (e.g., gender, race, background) and target variables, leading models to make decisions based on these irrelevant features. Existing bias mitigation methods often struggle in complex, real-world scenarios, especially when multiple biases are present.

BAdd addresses this issue by injecting bias-capturing features into the model's training process. Specifically, it involves adding features that encode the protected (bias-inducing) attributes into the penultimate layer of the model during training. This approach aims to decouple the learning of biased features from the optimization process, allowing the model to learn bias-neutral representations of the data.

The method is evaluated on seven benchmarks, including both artificially biased datasets (e.g., Biased-MNIST, Corrupted-CIFAR10) and more complex, multi-attribute biased datasets (e.g., FB-Biased-MNIST, CelebA, UrbanCars). Notably, it achieves accuracy improvements of +27.5% on FB-Biased-MNIST and +5.5% on CelebA.
weaknesses: Assumption of Bias Representability: The method assumes that biases can be captured and represented explicitly. In cases where biases are subtle or unknown, this approach may be less effective.
Potential Increase in Model Complexity: Adding bias-capturing features might increase computational overhead during training and potentially during inference if not properly managed.
Fine-Tuning Requirement: The necessity of a fine-tuning step adds an extra phase to the training process, which might not be ideal in time-constrained or resource-limited applications.
Limited Analysis on Real-World Biases: While the paper includes evaluations on datasets like CelebA, further analysis on real-world datasets with complex, less-defined biases would strengthen the applicability of BAdd.

Meta-review: This paper received mixed reviews. The reviewers appreciated the interesting and model-agnostic approach to debiased training, its consistent improvement across multiple datasets, and extensive experiments and analyses. However, they at the same time raised concerns with the demand for manually annotated (well-represented) bias attributes for training data (iTN8, JPua, cPAJ), incremental novelty (iTN8, e7ck), a potential over-correction issue caused by the proposed method (iTN8), lack of empirical validation for the versatility of the method (e7ck), additional space-time complexity imposed by the method (e7ck, cPAJ), limited ablation study (e7ck), limited analysis on real-world biased data (cPAJ), and some clarity issues (JPua). 

The authors' rebuttal and subsequent responses in the discussion period address some of these concerns but failed to fully assuage all of them. After the discussion period, Reviewer iTN8 still had concerns about the manual annotation of multiple bias attributes for training data and the potential technical issue of the proposed method; the former was pointed out by other two reviewers too, they did not come back for discussion though. The AC agrees with the authors that the manual multi-bias labeling can be execused regarding the fact that the task of mitigating multiple biases is extremely challenging. However, the authors are strongly encouraged to at least report results with existing bias discovery methods (e.g., [M2]) to demonstrate that the proposed method is ready to be applied to real-world complex scenarios as the authors argued in the paper; this is also essential since a few prior methods mitigate multiple biases with no manual bias labels [M4, M5], and because these methods exist, someone might think that the authors' claim in the rebuttal are overstated. Moreover, considering that the paper strongly emphasizes the ability of the proposed method to mitigate multiple biases as its contribution, experiments with multiple biases need to be strengthened. Currently, the datasets used for the purpose, except the CelebA variant, are all artifical (relevant to the concern of Reviewer cPAJ on limited analysis on real-world biased data), and details of the CelebA variant (e.g., the number of bias attributes and their populations) have not been reported in either the main paper or the appendix although such details are crucial to examine if the benchmark is valid. In this context, the AC strongly encourages the authors to adopt existing benchmarks and/or evaluation protocols of previous work on mitigating multiple biases [M1, M2, M3] in addition to the UrbanCar dataset; of course, the novelty and contribution of the proposed method compared to the all aforementioned papers [M1, M2, M3, M4, M5] should be clearly discussed in the paper, while the current version does not.

Putting these together, the AC considers that the remaining concerns outweigh the positive comments and the rebuttal, and thus regrets to recommend rejection. The authors are encouraged to revise the paper with the comments by the reviewers and the AC, and submit to an upcoming conference.


[M1] Men Also Do Laundry: Multi-Attribute Bias Amplification, ICML 2023

[M2] Discover and Mitigate Multiple Biased Subgroups in Image Classifiers, CVPR 2024

[M3] Improving Robustness to Multiple Spurious Correlations by Multi-Objective Optimization, ICML 2024

[M4] ExMap: Leveraging Explainability Heatmaps for Unsupervised Group Robustness to Spurious Correlations, CVPR 2024

[M5] Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias, AISTATS 2024

Decision: Reject