Identifying Spurious Correlations using Counterfactual Alignment
Authors: Joseph Paul Cohen
Louis Blankemeier
Akshay S Chaudhari
Venue: Accepted by TMLR ()

Abstract: Models driven by spurious correlations often yield poor generalization performance. We propose the counterfactual (CF) alignment method to detect and quantify spurious correlations of black box classifiers. Our methodology is based on counterfactual images generated with respect to one classifier being input into other classifiers to see if they also induce changes in the outputs of these classifiers. The relationship between these responses can be quantified and used to identify specific instances where a spurious correlation exists. This is validated by observing intuitive trends in face-attribute and waterbird classifiers, as well as by fabricating spurious correlations and detecting their presence, both visually and quantitatively. Furthermore, utilizing the CF alignment method, we demonstrate that we can evaluate robust optimization methods (GroupDRO, JTT, and FLAC) by detecting a reduction in spurious correlations.

Published in Transactions on Machine Learning Research (01/2025)
Identifying Spurious Correlations using Counterfactual Alignment
Joseph Paul Cohen
jusephh@josphpoohen.com
Stanford University
Louis Blankemeier Stanford University
Akshay Chaudhari
Stanford University
Reviewed on OpenReview: https: openreview. net/ forum? id= jtjuziaall
Abstract
Models driven by spurious correlations often yield poor generalization performance. We propose the counterfactual (CF) alignment method to detect and quantify spurious correla- tions of black box classifiers. Our methodology is based on counterfactual images generated with respect to one classifier being input into other classifiers to see if they also induce changes in the outputs of these classifiers. The relationship between these responses can be quantified and used identify specific instances where a spurious correlation exists. This is validated by observing intuitive trends in face-attribute and waterbird classifiers, as well as by fabricating spurious correlations and detecting their presence, both visually and quantitatively, Furthermore, utilizing the CF aligmment method, we demonstrate that we can evaluate robust optimization methods (GroupDRO, JTT, and FLAC) by detecting a reduction in spurious correlations.
1 Introduction
Challenges related to neural network generalization and fairness often arise due to covariate shift (Moreno- Torres et al. 2012) and shortcut learning (Ross et al. 2017 Geirhos et al. 2020). Shortcut learning can lead to models making decisions based on factors not aligned with expectations of the human that created the model. These powerful models (vision models in this work) leverage wide array of features and relationships, which may inadvertently incorporate unwanted spurious correlations. These spurious relationships may stem from sample bias (e.g., predicting cows when cows are observed grass but not inherent to the class definition (e.g., predicting cows when an animal has four legs) (Beery et al. 2018).
beach) or may be
In this work our objective is to understand black-box classifiers, without access to thoir training data, as this is a common use case encountered by practitioners. In this analysis, we utilize counterfactual (CF) images which are synthetic images simulating a change in the class label of an image (Wachter et al., 2017). These synthetic images have fcatures modified such that the prediction of the classifier changes. We can then view the synthetic images to understand the reasons that prediction was made.
Specifically, we are interested in CF images that are directly generated using the gradients of a classifier Cohen et al.] 2021 Joshi et al. 2018 Balasubramanian et al. 2020). Generating CF images with respect to a classifier is rooted in similar logic to that of crafting adversarial examples. However, the key distinction lies the constraint that CF images remain within the data manifold of plausible images. The latent space of an autoencoder provides such data manifold. This approach enables us to study the specific features used
Work not related position Amazon.

---

Published in Transactions on Machine Learning Research (01/2025)
fb
(xr)
A
D
Plotting relative to base classifier.
f1
1.2
fb
0.3
D
f1
Change:
fb
04
0.5
C.G
0.
6/
1.0
2- D
Model Fredd
f1
Compute relative change between the predictions:
Images generated different lambdas w.r.t. the base classifier.
Images processed with multiple classifiers.
f1(((z)) - f((((zo) Jb(D(z))) fo(D(z)))
Figure Overview of the alignment methodology. An image is encoded, reconstructed, and then processed by a classifier. The counterfactual generated by subtracting the gradient of the classifier output w.r.t. the latent representation. The resulting representation is reconstructed back into image. The reconstructed images processed with multiple classifiers and the classifier outputs can be plotted side by side to study their alignment. The base model value can be used as the x-axis to more casily compare it to the predictions of another classifier. The output changes can be quantified and compared using relative change.
by a given classifier on a particular input. By keeping the classifier and autoencoder independent, different classifiers can be analysed using single auloencoder reference point.
The task of interpreting these CF images presents new challenges of scale. Spurious correlations may only cxist in handful of samples where rare features occur together. Locating these samples requires investigating the features used for each prediction, which can be automated using the approaches we present in this study,
The approach we take is to generate CF images for the positive predictions a classifier over an entire dataset and study the resulting CF images using collection of different classifiers, which we refer to as downstream classifiers, observe what outputs are impacted. If the CF images generated for one classifier also change the predictions of other classifiers, we conclude that there is shared feature usage. This shared feature usage prompt us to investigate if this relationship is unexpected. Such as if classifier that predicts big noses should share features with classifier that predicts arched eyebrows.
The relative change metric (Eq. 3 can be used to quantify the impact of counterfactual perturbations and identify spurious correlations that may occur for a specific example or to identify trends across an entire dataset. We study face-attribute classification, recognizing advantage in allowing visual verification by readers.
Overall, given the problem setting of spurious correlations, the contributions of this work include:
We propose CF alignment to reason about the feature usage relationships between classifiers, quan- tifidd using the relative change metric. This approach allows for both aggregate quantification and targeted querying of models to locate specific examples where the spurious correlations are used to make prodictions.
Our work demonstrates the ability to detoct spurious corrclations on existing face-attribute and wa- terbird classifiers. This is validated by observing intuitive trends in a faco-attribute classifier as well as inducing spurious correlations and then detecting their presence, both visually and quantitatively.
2

---

Published in Transactions on Machine Learning Research (01/2025)
We demonstrate that CF Alignment can evaluate robust optimization methods (GroupDRO, JTT, and FLACC) by detecting reduction in spurious correlations. We also observe improved generaliza- tion performance when a spurious correlation is reduced.
2 Related Work
Counterfactual (CF) generalion can be done in a variely of ways (Verma el al. 2020 Classifier specific approaches generally perturb latent space representation guided by classifier. Some methods use the gradient of classifier to guide movement in the latent space by computing the gradient directly (Cohen Balasubramanian et al.) 2020). Another classifier based approach is to train model that predicts, using training examples, where to walk in the latent space to change the classifier's prediction (Schutte et al.) 2020). This has the benefit of working with classifiers that are not differentiable. Another approach is conditional generation which doesn't use a classifier and instead generates images based on a conditioning variable. This conditioning can be a label
et al. 2021 or defining loss that is optimized (Joshi et al., 2018
MMirza& & Osindero 2014| Baumgartner et al. 2018: Samangouei et al., 2018 Schutte et al. 2020 Hasenstab et al. 2023 [arredo- Arrieta & Del Ser| 2020; Singla et al. 2023) or text (Chambon al. 2022 , or can be provided by manually adjusting the latent representation (Seah et al.
2019). Conditional generation learns representation from the data and does not capture the exact features utilized by a classifier, as the gradient based methods do.
Several methods exist for studying neural network based classifiers to understand the features they use. Early approaches include gradient based attribute maps (aka saliency heatmaps) that could be overlaid on the image (Simonyan et al. 2014). This was extended capture more caveats of neural network reasoming (Sundararajan al.) 2017 Springenberg al.] 2015)) but these methods st.ill only generate healmap expla- nations. Other work focuses on leveraging occlusions to identify discrete image regions (Ribeiro et al. 2016) that are relevant to a prediction. This is useful when predictions are based on discrete features that are smaller than the occlusion size. Other work inspects individual neural network neurons to identify where class information is propagating using linear probing (Alain & Bengio 2016 and manual identification (Olah let al. 2020 2017). This useful for understanding the internal representations of neural networks but does not explain individual prodictions.
Along these lines, work by Kim et al. (018)) focuses on concepts represented by layer activations forming called "Concept Activation Vectors". These vectors allow us to identify inputs which result in similar internal stale and therefore have similar reason for being predicted. A limitation of this approach is that it is difficult to compare vectors between classifiers because they have different weights.
An approach by Balakrishnan et al. (2021] generates synthetic images using human annotators such that the images have known feature similarity (e.g. a face with all features held constant except for skin color). These images can then be used to identify spurious correlations between attributes. To contrast this approach ours, this work is creating CFs manually using humans and then looking for unexpected changes in classifier predictions, similar to our approach. A limitation of this method is the required manual effort and the reliance on human knowledge of concepts. The automatic generation of CFs in our work overcomes this issue and allows us to identify features that are unknown initially to humans. And the relative change metric makes easy to identify potential spurious correlations.
3 CF Alignment Methodology
In this work, we introduce the CF alignment approach, outlined in Figure D The CF alignment approach tests if a base classifier fo utilizes features that are also used by a downstream classifier (e.g. fi). If they do, the generated CF would have fcatures modulated which would cause the other classifier (e.g. fi) to have a different output. This aligument can then be measured quantitatively by comparing how aligned the changes in prediction are. Here we consider a classifier as predieting a single scalar value f(x) E [0, 1]. A multiclass classifier can also work if we consider one class vs all other classes to create a single output value.
We say that two classifiers are aligned if both their predictions change negatively with a counterfactual generated for one of them, implying the features removed were used for both predictions. We say they are
3

---

Published in Transactions on Machine Learning Research (01/2025)
inverse aligned when one classifier's prediction is reduced and the other's prediction increased. We discuss how to quantity this in $3. but will continue on how we generate CFs.
To generate CF samples a technique known as Latent Shift (Cohen et al. 2021 is used. The implementation of Latent Shift requires the integration of an encoder ponder//ecoone /ecoder model, denoted as D(E(z)), where E represents the encoder and D the decoder. Additionally, a classifier f is incorporated, responsible for predicting the target variable y, expressed as y = (1)) is important note that both the autoencoder and the classifier are trained independently, with the only specified requirements being differentiability and operation on the same data domain.
To compute a counterfactual, the process begins with encoding an input image x using the encoder E(x), resulting in a latent representation 2. The next step involves perturbing the latent space lo generate coun- terfactual samples. This perturbation is performed using a base classifier for as illustrated in Equation [] The resulting perturbed latent representation is denoted as 2y. Subsoquently, the decoder is employed
reconstruct the image, resulting in a counterfactual image ' as depicted in Equation 2
The perturbation of the latent space, represented by 2λ, is computed by subtracting À times the gradient of the base classifier fo with respect to the latent representation 2. This operation is conducted to induce changes in the latent space that influence the predictions of the classifier. The parameter À is determined through an iterative search process, where its value is systematically adjusted in steps. The objective is to find a suitable À such that the classifier's prediction is either reduced by 0.6 or starts to increase. 0.6 is chosen as difference that should cross the decision boundary. Just crossing the 0.5 mark not always large enough to generate reasonable counterfactual.
Ozo(D(z)))
ス = 50
(1)
x = D(xx)
(2)
Oz
Equation expresses the computation of 2x, and Equation 2 outlines the generation of the counterfactual image o¹x by decoding the perturbed latent representation.
This process allows to systematically explore and manipulate the latent space to generate counterfactual samples that reveal insights into the decision-makin process of the classifier and its sensitivity to changes in the input data.
3.1 Relative Change
Having generated counterfactual samples using the Latent Shift approach, we proceed to algorithmically assess the impact of these samples on various downstream classifiers responsible for predicting the probability of different attributes, denoted as f1, and so on. These classifiers are distinct from the base classifier, denoted as fh, which was utilized in the counterfactual generation process.
To quantify the relationship between the base classifier and each downstream classifier, we employ the "relative change metric", as defined in Equation g This metric is akin to correlation but takes into account not only the direction of change in predictions but also the magnitude of that change. The formula for relative change is expressed as follows:
-(D(()))- fi((2o))
RelativeChannee(1
Fo(D(zx)) (z(o))
Here, D(2x) and D(=0) represent the reconstructions of the latent representations 2A and 2g (perturbed and original, respectively) by the decoder D. The numerator captures the change in prediction made by the downstream classifier fi in response to the counterfactual perturbation, while the denominator corresponds to the change in prediction made by the base classifier fo due to the same perturbation.
In our experiments, we opted for relative change over traditional correlation mcasures. This decision was motivated by the observation that correlation metrics occasionally yielded false positives when only slight change in the prediction of fi occurred compared to the base classifier fo. Relative change provides a more nuanced understanding by considering not just the direction but also the magnitude of the change, offering a more robust asscssment of the impact of counterfactual perturbations on downstream classifiers. The relative change metric value can be interpreted follows, value of 1 indicates very high alignment, 0 indicates
4

---

Published in Transactions on Machine Learning Research (01/202))
V22
EST
HC3
-275
-0:0
1:00
0:50
300
400
166
10C
1364
E00
31%
1000
-280
0.
645
104
100
0:31
6.3
+022
Hat. cifesticii
341
310
.008
400
-37
EX1
per
6008
141
4:10
319
100
UM
305
visr NJ, Jevika
6R
+139
404
311
3.0
3.6
47%
OM
301
220
enti/tee
44.9
000
Relattive change using CF aligmment
Correiatio batweon predictions of ciassifer.
CJ Comreition between labels training data.
Figure 2: Relationships between face-attribute classifiers as measured by CF-alignment relative changes (left), classifier predictions (middle) and training data labels (right). In (a), base classifiers are along the rows and downstream classifiers are along the columns. Comparing (a) to (b) and (c), shows that many relationships reffected in the CF outputs are preserved from corrciations in the training data. We draw the readers allention to some unique differences. The relationship between male and big nose, highlighted in rod, is strong in both the classifier predictions and ground truth labels but low in CF alignment, indicating that although correlated, these features are not exploited by the classifier. In contrast to this, the relationship between pointy_ nose and smiling, highlighted in green, is weak in both the classifier predictions and ground truth labels but high in CF alignment, indicating that this relationship was introduced by the classifier.
no alignment//ie Illgnmment/ideepeddeene /nndepeeeeenee, and -1 indicates a strong inverse aligument. The relative change metrie is key contribution that allows us to make sense of the CF alignment results at scale over large datasets.
4 Experiments
The experiments in this work are performed the CelebA HQ dataset (Karras et al. 2018) that contains over 200k celebrity images with 40 facial altribule labels per image. The resolution of the images is (024x1024) Experiments are also performed on the lower resolution (178 x 178) CelebA dataset (Liu et al. 2015).
The pre-trained face classifiers used in this work are sourced from (Vandenhende et al. 2020 They were trained on the CelebA dataset (Liu el al. 2015) to predict 40 different faacial altributes on images of dimen- sions 224. These classifiers all have the same architecture, the only difference between them is the last classifier layer weights. We chose this model because it has good performance, publicly available, and was implemented in PyTorch.
We leverage the VQ-GAN autocncoder from [Eseer et al. 2021 trained on the FacesHQ dataset, which combines the CelebA HQ dataset (Karras et al.] 2018) and the Flickr-Paces-HO (FFHQ) dataset (Karras ett al.) 2019). The resolution of this model is 256x256. In order to make this work for existing classifiers, each classifier normalizes the input image dynamically to match its training domain.
The Captum library (Kokhlikyan et al., 2020 is utilized for baseline attribution methods. PyTorch (aaske et al. 2019 is used for efficient tensor computation and automatic differentiation. The source code and model weights for all experiments will be released publicly online"] The CF aligument algorithm requires few seconds to run for each image on a NVIDIA V100 16GB GPU. The CF generation step is the most expensive and is variable due to the automated search for the optimal lambda. The resulting CF images are then processed by each classifier which scales by the number of classifiers studied.
4.1 Aggregate statistics over dataset
Viewing aggregate CF alignment statistics over dataset can be useful when investigating a model for bias spurious correlations. To achieve this we compute the average relative change between pairs of classifiers (N=400 images per class) is shown in Figure 2h Figures 2 and 2 show correlations between classifier
hhttp:://ggiihbb..c///eeeeess
5

---

Published in Transactions on Machine Learning Research (01/2025)
Alignad
Alignad
"parnny pree (1
ancher, eyehhoww 23)
04
bushy, byrb mes ft.
beokss (0
staighiCha (0.12)
(1.00
(8,42)
(0.40)
Prediction
Danm Hooid Pradiction
(a)
Aligred
Inverse Aligned
0.5
fpuinlyy (1.0)
0.7
maaunoos 18.351
02-
0.5
0.1
0.4
high_dheekzzoes (Q.46)
02-
smrlling (..r))
0.4
0.5
02
04
0.8
8:ce Hodel Prediction
Bood Medel Frediction
(b)
Figure 3: CF alignment. examples for the fo=poinly nose with the highest aligned and inverse aligned clas- sifiers. We observe an inverse alignment with big nose and potential spurious relationships with eyebrows, eyes, hair, and smiling. The relative change is shown next to cach classifier name.
prodictions and the ground truth training labels. subset of classifiers studied are shown in this figure for clarity, with the full CF alignment matrix in Appendix Fig. These classífiers were chosen because they have intuitive and unintuitive relationships with large relative change that help illustrate the contribution of this method. The positive relationships between smiling and mouth_ seatthhsligghhy__o open and high high__chehkone cheekbones seems intuitive while the positive relationship between poinly and likely due to spurious correlations. This relationship is explored on a single example in 44.2
and arched eyebrows seems unintuitive
The caption of Figure 2 dctails obscrvations where correlation in the training data and classifier outputs are not the same as the relationships uncovered using CF alignment. These observations indicate that even if correlated attributes are presented to the classifier it does not cause feature usage to be correlated. These observations also imply that even if uncorrelated attributes are presented to the classifier it may still construct spurious feature relationships.
4.2 Studying specific examples
Using the aggregate analysis from $4.1 a guide, classifiers can be selected to investigate undesired rela- tionships on specific images. Inspecting the poinly nose classifier in Figure using two images with positive predictions for pointy_noos nose observe similar and contrasting relationships between classifiers. A com- mon theme is the inverse alignment with big_nose nose which is intuitive as it is the opposite of a pointy nose. rosy cheeks is common aligned classifier which does not appear to have an intuitive reason (to the authors) and is likely a spurious correlation. An alignment with smiling only observed in one of the examples.
A takcaway from these examples is the uniqueness of a spurious correlations to specific images with specific features, thus demonstrating the need for the method we present to mine for examples where these spurious correlations be observed.
4.3 Validation by inducing spurious correlations
In order further verify the CF alignment approach, we construct a classifier with known spurious correlation and then demonstrate that this bias is observable in the CF alignment plot. spurious correlation

---

Published in Transactions on Machine Learning Research (01/2025)
x
X aniiing
X X
0.6
0.4
*smiling 11.0)
archod eyebrows (0.01)
0.2
0.4
0.6 Model Prediction
0.8
10
(a) CF for the base classifier smiling showing eyebrows are unchanged. The horizontal line indicates the prediction of arched egebrows is not influenced by the features used for smiling in this image.
Speningt syebross))
Jemingg Sarched systormwa)
3.2
"uriling-0.aacheed Barcheeeeyeerrors (1.9)
eyebrows
vicheed_eyebroos (0.97)
0.8
0.4
0.8
Hodel Predictioon
(b) CF for a modified smiling classifier which has been combined with an arched eyebrows classifier.
Figure 4: Example of detecting a spurious correlation in a biased base classifier. The classifier is biased with arched eyebrows and this observed in the alignment plot as well as in the counterfactual image. The relative change is now 0.97 compared to 0.01 for the unchanged smiling classifier.
can induced the classifier by composing classifiers:
fviosea(x) = famstiing()) + O.3farched_ eyebrowe(
(4)
The CF alignment plot for the base smiling classifier predictions in Fig. 4a show that arched _etebrows does not change and we can confirm this the CF image. The resulting biased classifier can be observed using the arched eyebrows features in Fig. 4b both visually as well as in the CF alignment plot. The CF arched_eyebrouss
alignment plot shows that the prediction of arched eyebrows changes and is aligned with smiling.
4.4 Analysis of robust optimization methods
An application of CF alignment to assess the effectiveness of bias mitigation techniques. Various biased training settings have been developed to induce spurious correlations with sensitive attributes in models and then employ bias mitigation strategies to counter them. The experiments we present are designed to demonstrate that measuring the CF aligument with sensitive attributes can measure the impact of these methods.
To properly compare these methods, experiments are grouped based on the specific training configuration used each baseline is unique that setting. For this reason, these models cannot be compared side by side for the same dataset.
The bias mitigation methods we use are selected because their authors make their code available to generate pre-trained models that can be studied post-hoc. Using only pre-trained models further demonstrates the utility of our proposed approach where our method versatile enough to evaluate these existing pre-trained models.
Experiments in this section are performed on the held out test datasets and selected to be balanced such that there is balanced distribution of positive and negative examples with the sensitive attribute (e.g. in

---

Published in Transactions on Machine Learning Research (01/2025)
the CelebA dataset, 1/4 of the samples are labeled to have blond hair and be male, 1/4 blond_ and be not male, 1/4 not blond hair and be not male, and 1/4 not blond hair and be male).
During classification experiments, the samples evaluated are selected to be inversely correlated with the sensitive attribute to causse the most discrepancy in performance (e.g. Samples are selected that are labeled blond hair and male as well as not blond hair and not male).
4.4.1 GroupDRO on Waterbirds
Work GroupDRO Sagawa et al. (2020) constructed the Waterbirds task such that there is spurious correlation between birds (Irom the CUB dalasel (Wah et al. 2011 with backgrounds that contain water or land.
Three models are evaluated for this experiment, a bascline model which was induced to rely on spurious correlations, a model trained with Group Distributionally Robust Optimization (DRO) (Sagawa et al., 2020 which aims minimize error over groups (which are known during training, in this case the background class), a model using Just Train Twice (JTT) (Liu al., 2021) which uses two stage approach that boosts misclassified training examples in a second training cycle. To perform CF alignment, classifiers are trained to predict land and water using the samples from the waterbirds dataset labeled as those respective classes.
A limitation of this analysis is the limited latent variable model used (A VQ-GAN trained on Open Images Krasin et al. 2017)) that has difficulty modulating features (likely because its training domain broad). A model that is better at representing birds would be able to generate better counterfactuals that are easier interpret.
In Figure two models are evaluated the same image illustrating a difference in CF alignment results to classifier that predicts the background. The relative change with the background classifier is much higher for the baseline model than the model trained with DRO (0.35 vs 0.25) indicating that training with GroupDRO prevented the models reliance on Tealures associated with the background classifier. A magnified view of the counterfactuals of the bird's head indicate nostril size being reduced and colors becoming brighter in the model trained with DRO. This may indicate larger nostril sizes are associated with water birds which aligns with the family of waterbirds, Procellarifforees, having an enlarged nasal gland at the base of the beak for secreting salt water (Ehrlich et al., 1988). While the mallard in the image is not Procellariiforme, nostril size may be feature used by the classifier in general many in the waterbirds are seabirds.
Aggregate results over 4097 samples from the test set of the waterbirds dataset reveals reduction in mean relative change using DRO shown in Table[] This indicates that the features used by the baseline classifier overlap the land and water classifier and these spurious correlations were reduced when utilizing DRO. We observe that this improved model, with a lower relative change, achieves a higher classification accuracy.
Relative change +
Relative change + w w/land classifier
Classification AUC
Target
w/water classifier
Waterbird (baseline)
0.35+0002
-0.36+0.03
0.69
Waterbird (groupdro) Waterbird (jtt)
0.25+10.00
-0.22:10.03
0.92
0.42-10.03
-0.43-10.00
0.65
Table 1: Aggregate relative change metrics for models trained on the waterbirds test dataset (N=4097). A reduction relative change with the land and water background classifiers indicates DRO has reduced the spurious correlation.
4.4.2 GroupDRO on CelebA
GroupDRO models are also evaluated on the CelebA datasct on the task of Blond Hair and the sensitive attribute Male. JTT not evaluated on this task because we were unable generate pre-trained weights using the provided code due to memory issues that we were unable to resolve.
8

---

Published in Transactions on Machine Learning Research (01/2025)
wolorbird
eolobird
1.00
*Baseline (Waterbird)
Place (Water)
Model Prediction
0.98
0.97
Relative Change:
0.36
0.96
0.97
0.98
0.99
1.00
Base Model Prediction
(a) Evaluating baseline model trained that is expected to use spurious correlations.
X
X X aatutind
1.0
"WithDRO (Waterbird)
Model Prediction
Place (Water)
0.8
Relative Change: 0.03
0.6
0.7
0.8
(b) Evaluating model trained using Distributionally Robust Optimization (DRO) to reduce the spurious correlation with the background.
X
x
swalerrbird (Baseline)
X
ealerbird (GroupDRO)
magnified view of the birds head which shows differences in nostril size and head color.
Figure 5: Counterfactuals generated for models trained the waterbirds dataset together with CF alignment plots. (a) and (b) The relative change between the waterbird classifier and the background classifier is shown in the lower right of the plot. (c) The nostril size is reduced in the counterfactual for the DRO model indicating that larger nostril size is associated with waterbirds.
In Table can the relationship between Blond Hair and Male negative in the baseline and then is reduced by the GroupDRO training and becomes slightly positive. This reduction results in higher classification accuracy on the CclebA test set.
Relative change 1 w/male classifier
Classification 1 AUC
Target
Blond Hair (baseline) Blond Hair (groupdro)
-0.05240.01 0.005+0.000
0.91
0.95
Table 2: Methods evaluated on models that predict Blond Hair trained on the CelebA dataset. Evaluations are performed on 1024 samples from the test set.

---

Published in Transactions on Machine Learning Research (01/2025)
4.4.3 FLAC on CelebA
The work Pairness-Aware Representation Learning by Suppressing Attribute-Class Associations (FLAC) (Sarridis et al. 2023) uses pairwise similarity between the between the model representation and the rep- resentation of a bias-capturing classifier that predicts the sensitive attribute. The bias capturing classifier acts as proxy for labels and provides embeddings that can be used to regularize the feature space of the classifier we are training. Their approach is to minimize the difference, in feature space, between samples with different sensitive attributes and the same target label as well as increase the difference between samples with the same sensitive attributes and the different target label. In Table 3 the relative change is shown to reduced and the classification performance improves.
Relative change 4
Classification f AUC
Target
Alpha
w/male classifier
Blond Hair
0
-0.081#0.016
0.77
Blond Hair 30000
-0039=00.009
0.93
Blond Hair 60000
-0.051+00010
0.92
Table 3: Aggregate relative change of models trained with FLAC on 1024 samples from the CelebA test set. The strength of this regularization is rolled with an a parameter.
5
Limitations
There are limitations and challenges CF generation (Verma et al. 2020) where bias can also exist in the CF generation method which can limit the features able to be modulated and prevent spurious features from being detected.
Spurions alignment: This situation can occur if the downstream classifier (fi) also has the same spurious features as the base classifier (fb). Tf this is the case, then both of their output would change if the spurious features were modulated causing these classifiers to appear aligned. For example, a base classifier predicts cow but spuriously uses the presence of a beach to produce low prediction. A CF generated for this classifier may change the background of a farm to beach. If we check the alignment with a classifier that predicts chicken (but this chicken classifier also reduces its prediction if the image contains beach) then these classifiers would appear aligned and the relative change would be positive. These classifiers are in fact aligned based on feature usage but just because they just use the same spurious features. This may be similar to a false positive but it isn't false, the classifiers are aligned.
For this reason, in this work we reference the classifier target names (e.g. mouth slightly open) instead of the concept that it assumed represent. One solution to avoid these edge cases is to utilize multiple classifiers that are trained to predict the same concept using multiple training datasets so on average they predict the correct concept.
False negative alignment: This situation can occur if the downstream classifier (fi) predicts using features that are 'opposite" of the base classifier (fa). For example 'opposite" can mean the base classifier (fi) predicts cow using only the presence of absence of a beach while the downstream classifier (fi) predicts beach using only the presence of absence of cow. The CF for the base classifier changes the background from a beach to something else but this docsn't change the prediction of the downstream classifier (fi) so there is no alignment, the relative change would be 0. This case could be considered false negative because we didn't 't find alignment with beach classifier but the beach classifier was flawed. To address an issue like this we could simply inspect the CFs that are being generated with domain experts as well as vary the downstream classifier's ' training data.
Furthermore, although we are optimistic that our method will generalize to additional domains, we focus our current analysis on faco-attribute classification and waterbirds. Although we do not demonstrato the generalizability of our method to additional domains, using face-attribute classification is a deliberate choice enabling qualitative evaluation, in addition to our quantitative evaluation. Additionally, since CF generation
10

---

Published in Transactions on Machine Learning Research (01/2025)
relics separate autoencoder, improving the representational capability of the autoencoder may improve the fidelity of the CF generation.
6 Conclusion
this work we propose counterfactual (CF) aligument along with the relative change metric $3 strate that this method enables us to reason about the feature relationships between classifiers in aggregate and locate specific examples where the spurious correlations are used. These claims are supported with analysis of faco-attribute classifiers that identify expected and unexpected spurious correlations.
We demon-
We observe that if correlated altributes are presented to the classifier, this does not cause feature usage to be correlated $4.1 We also observe that if uncorrelated attributes are presented to the classifier, the classifier may still construct spurious feature relationships.
The validity of the CF alignment method is confirmed by inducing and quantifying spurious correlations via additive composition $4.3 Classifiers are composed together to create a classifier with known spurious correlation and then this is observed using CF alignment.
We then explore classifiers trained using robust optimization methods to demonstrate the applicability to black box classifiers and to provide more visibility into what these methods achieve $4.
Overall, the proposed approach may serve as an end-to-ena end or Haman-in-the-loop system to automatically detect, quantify, and correct spurious correlations for image classification tasks that lead to biased classifier outputs.
7 Acknowledgments
We would like to thank Stanford University and the Stanford Research Computing Center for providing computational resources and support that contributed to these rescarch results.
References
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In- lernational Conference on Learning Representations, 2016. URL
01644
Guha Balakrishnan, Yuanjun Xiong, Wei Xia, and Pietro Perona. Towards Causal Benchmarking of Bias Face Analysis Algorithms. In Computer Vision and Pattern Recognition, 2021. URL httt::/arxxxii
abs/2007.06577
Rachana Balasubramanian, Samuel Sharpe, Brian Barr, Jason Wittenbach, and C. Bayan Bruss. Latent-CH A Simple Bascline for Reverse Counterfactual Explanations. In Neural Information Processing Systems (NeurIPS) Fair AI in Finance Workshop, 2020. doi: /ttp:/aaaxxv..2..990111 URL https://arxiv
org/ abs/ 20122093111001 09301
Alejandro Barredo- Barredo-Arrieta Arrieta and Javier Del Ser. Plausible Counterfactuals: Auditing Deep Learning Classi- fiers with Realistic Adversarial Examples. In International Joint Conference on Newral Networks, 2020. ISBN 9781728169228 doi: 10.1109 http://NNNN800..2002...2 document/9206772/
URL htttps://ieeexploree org/
Christian F. Baumgartner, Lisa M. Koch, Kerem Can Tezcan, Jia Xi Ang, and Ender Konukoglu. Visual Feature Attribution Using Wasserstein GANs. In Computer Vision and Pullern Recognilion, 2018. ISBN
0781538664209.. doi: 10.1109/CVPR.2010 000667. URL http:://rxii.. owg/absg////ooooooooo...oo.. http//bb/1711.... 1711 08998
Sara Beery, Grant Van Horn, and Pictro Perona. Recognition in Terra Incognita. In European Conference on Computer Viston, 2018. ISBN 9783030012694.
doi: 10.1007/978-3-00 3-030-0-0002-22 03000127- )28. URL http:
Ftexxiv.org/bbs/1800 04975
11

---

Published in Transactions on Machine Learning Research (01/2025)
Pierre Chambon, Christian Bluethgen, Jean-Benoit Delbrouck, Rogier der Sluijs, Malgorzata Polacin, Juan Manuel Zambrano Chaves, Tanishq Mathew Abraham, Shivanshu Purohit, Curtis P Langlotz, and Akshay Chaudhari. Gen: Vision-Language Foundation Model for Chest X-ray Generation, 2022. URL
betps://arxiv.on wttpo/////....////////
Joseph Paul Cohen, Rupert Brooks, Sovann En, Evan Zucker, Anuj Pareek, Matthew P. Lungren, and Akshay Chaudhari. Gifsplanation via Latent Shift: A Simple Autoencoder Approach to Counterfactual Generation for Chest X-rays. Medical Imaging with Deep Learning, 2021. URL httpp:://pennreviev.
net/fforumiid-nnnjyyelaal
Paul R. Ehrlich, David Dobkin, and Darryl Wheye. The Birder's Handbook. Simon & Schuster, New York, NY, 1988. ISBN 978-0-671-65989-9,
Patrick Esser, Robin Rombach, and Björn Ommer. Taming transformers for high-resolution image synthesis. In Computer Vision and Pattern Recognition, 2021. ISBN 9781665445092. doi: 10.1109 ALL10//CCVRAA43..22000 01268. URL http://axxiv. wwtp:////////aa..eee/ee wttp://2......9... 09841
Robert Geirhos, Jörn-Henriz Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intel- ligence, 11 2020. ISSN 2522-5839.
doi: 10.1088//4225--000000277 URL http://///e www.:nature.com
articles/s42256 020-6-00000000222222
Kyle A. Hasenstab, Justin Huynh, Samira Masoudi, Guilherme M. Cunha, Michael Pazzani, and Albert Hsiao. Feature Interpretation Using Generative Adversarial Networks (FIGAN): A Framework for Visualiz- ing a CNN's Learned Features. IEEE Access, 2023. ISSN 2169-3536. doi: 10. RA2109/AACESS.2023....
URL https://ieeexplors ieee. org//oocment//1015770
Shalmali Joshi, Oluwasanmi Koyejo, Been Kim, and Joydeep Ghosh. xGEMs: Generating Examplars Explain Black-Box Models, 2018. URL http::/arxiv org/ abs/ 1806 08867
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive Growing of GANs for Improved Quality, Stability, and Variation. In International Conference on Learning Representations, 2018. URL
htttps///axiis http:////axwv.../aa//////// (aarg/aba/177an 10196v3
Tero Karras, Samuli Laine, and Timo Aila. A Style-Based Generator Architecture for Generative Adversarial Networks. In Computer Vision and Pattern Recognition, 2019. URL h 04948
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. Interpretability beyond feature attribution: Quantitative Testing with Concept Activation Voc- tors (TCAV). In International Conference on Machine Learning, 2018. ISBN 9781510867963. URL http://arxiv
http:///axxv.orrg///// absaaaaaaaa 1711. 11279
Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz--Richardson. Captum: A unified and generic model interpretability library for Torch,, 2020.
Ivan Krasin, Tom Duerig, Neil Alldrin, Vittorio Ferrari, Sami Abu-EI-Haije Alina Kuznetsova, Hassan Rom, Jasper Uijlings, Stefan Popov, Shahab Kamali, Matteo Malloci, Jordi Pont- Tust-, Andreas Veit, Serge Belongie, Victor Gomes, Abhinav Gupta, Chen Sun, Gal Chechik, David Cai, Zheyun Feng, Dhyanesh Narayanan, and Kevin Murphy. Openimages: A public dataset for large-scale multi-label and multi-class image classification. Dalasel available from 2017.
David Krueger, Ethan Caballero, Joern Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-Distribution Generalization via Risk Extrapolation. In
International Conference of Machine Learning, 2021. ISBN 9781718845065. URL http://aaxxv.org abs/ 00688
12

---

00.016/..PArCOG.2211....6. Published in Transactions on Machine Learning Research (01/2025)
Evan Zheran Liu, Behzad Haghgoo, Annic S. Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just Train Twice: Improving Group Robustness without Training Group Information. In International Conference on Machine Learning, 2021. URL http://arxiv. org/ abs//
2107.09044
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. International Conference on Compuler Vision, 2015. ISBN 9781467383912. doi: 10.1109/TCCV.200 201...22221422.55 URL http://aaaaiv..org/////11.......
Mehdi Mirza and Simon Osindero. Conditional Generative Adversarial Nets, 2014. URL http://arxiy
org/ abaaabs/aa 1781
Jose G. Moreno-Torres, Troy Raeder, Rocio Aldiz-Rodriguez, Nitesh V. Chawla, and Francisco Herrera. A unifying view on dataset shift in classification. Pattern Recognation, 2012. ISSN 0031-3203. doi:
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature Visualization. Distill, 2017. ISSN 2476-0757. doi: 10.23915 10.23915/distill.00007 distill..00000. URL http://ai....pp///////feeeeeeeed
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom In: An Introduction to Circuits. Distill, 2020. ISSN 2476-0757. doi: 10.23915/diste Misstil..0002..000 URL
https://aistll..bp200000 wwtp:itssirrruttaaaoaaaaaaaaaatt
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeminng Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjic Bai, and Soumith Chintala. Py'Torch: An Imperative Style, Perfoomefoee Deep Learning Library. Neural Information Processing Systems, 12 2019. URL httph:htts///att /tabxi://rrxiv.... argg/bs//912./00
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guostrin. "Why Should Trust You?": Explaining the Predictions of Any Classifier. In Knowledge Discovery and Data Mining, 2016. doi: 10.1145/2939677 2939778. URL httpps//arxxv..rrtt org/ htt///bb//1110/ (abbb//662004999 04938
Laura Rieger, Chandan Singh, W. James Murdoch, and Bin Yu. Interpretations are useful: Penalizing explanations to align neural networks with prior knowlodge. In International Conference on Machine Learning, 9 2020. ISBN 9781713821120. URL httt:://///::i..ooo
13584
Andrew Ross, Michael C Hughes, and Finale Velei- Right for the Right Reasons: Training Dif- ferentiable Models by Constraining their Explanations. In International Joint Conference on Artificial Intelligence, 2017. URL https://github. com/dtak/rrr
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization. Interna- tional Conference on Learning Representations, 2020. URL
Pouya Samangouei, Ardavan Saeedi, Liam Nakagawa, and Nathan Silberman. ExplainGAN: Model Ex- planation via Decision Boundary Crossing Transformations. In European Conference on Computer Vision, 2018. URL https:: ExplainGAN Las__Moeee_________ Bet_al__xppaaaaiinnmmmoo ECCV _012 1ai8_paperr... httml
thecvf content ECCV 2011/httm//athaaaa Saciiaannniilll Silberman
html
Ioannis Sarridis, Christos Koutlis, Symeon Papadopoulos, and Christos Diou. FLAC: Fairness- Bairness-Aware Rep- resentation Learning by Suppressing Attribute-Clas: Associations. X3i:2204.14222, 4 2023. URL
http://arxiv wrt.oog///aaao ab2//04../30..... 14252
Kathryn Schutte, Olivier Moindrot, Paul Hérent, Captistt Schiratti, and Simon Jégou. Using StyleGAN for Visual Interpretability of Deep Learning Models on Medical Images. In Medical Imaging meets NeurIPS,
2020. URL TtRpp:t:::: Ssttss//aarav.oorr//////11.....1
13

---

Published in Transactions on Machine Learning Research (01/2025)
Jarrel C.Y. Seah, Jennifer S.N. Tang, Andy Kitchen, Frank Gaillard, and Andrew F. Dixon. Chest radio-
graphs in congestive heart failure: Visualizing neural network learning. Radiology, 2019. ISSN 15271315. doi: (0.1144//adiol221888883333
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. In International Conference Learning Representations, 2014.
6034
Chandan Singh, Guha Balakrishnan, and Pictro Perona. Matched s with GANs for mitigating attribute confounding. arXiu:2103.13455 2021. URL http:///aaxiv..r/abbbb////ss 13455
Sumedha Singla, Motahhare Eslami, Brian Pollack, Stephen Wallace, and Kayhan Batmanghelich. Explain- ing the black-box smoothly A counterfactual approach. Medical Image Analysis, 2023. ISSN 13618423. doi: 10.1001/j.modiaa20000 w02.mmmdaa..22.1227...... 102.21 URL https://arxii: http:/ivvwrgg/////////// http//b///111.... 04230
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for Simplicity: The All Convolutional Net. In International Conference on Learning Representations Workshop, 2015.
URL http://arxiv..rg// abd/ 6806.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic Autibuttion for Deep Networks. In Interna- tional Conference on Machine Learning, 2017. URL
wttp://wwww...aa/a/////...0.0.. 01365
Simon Vandenhende, Stamatios Georgoulis, Bert De Brabandere, and Luc Van Gool. Branched Multi- Task Networks: Deciding What Layers To Share. British Machine Vision Conference, 4 2020. URL
http://arxiv.orr wwwexiv..orgaaae aaaaabaa1999 1904. 02920
Sahil Verma, Varich Boonsanong, Minh Hoang, Keegan E. Hines, John Dickerson, and Chirag Shah. Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review. In Neural Information Processing Systems (NeurIPS) Retrospeclives Workshop, 2020. doi: 10.48550//rxivv20010 10596. URL http://arxiv. httg/abs/20110 10596
Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR. SSRN Electronic Journal, 2017. doi: 10.2139/ssmm. 3063289. URL https://arxiv.. org/ abs/1711.00399
Catherine Wah, Steve Branson, Peter Welinder, Pictro Perona, and Serge Belongic. The Caltech-UCSD Birds-200-2011 Datasel, 2011.
Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Contijo-Lopes, Ari S.
Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt Model soups: averaging weights multiple fine-tuned models improves accuracy without increasing inference time. In International Conference on Machine Leurning, 2022. URL httt::////artt wwwpr/iww.rrg//////00000 05482.
Huimin Zeng, Zhenrui Yue, Ziyi Kou, Yang Zhang, Lanyu Shang, and Dong Wang. Fairness-aware training of face attribute classifiers via adversarial robustness. Knowledge Based Systems, 2023. ISSN 09507051. doi: 10.1016 ho0.6//.noosss...... 20l.oosy...22..1.. 110556. URL. https: http://b...rr/1..01//////// knosys 2023. 110356
14

---

Published in Transactions on Machine Learning Research (01/2023)
A Appendix
140
0.31
100
SAX
321
4000
4008
-000
-021
-000
-331
-300
-281
406
861
4.6
-0.0
B004
1.6
1.00
1KC
612
0M
0.0
400
400
400
400
400
30
081
431
140
-0
60
400
1601
-66
036
634
0:31
1869
AM
631
1330
1130
541
-666
-MA
031
-502
(225
-021
1005
6:00
1849
031
1823
1843
4311
-380
-380
300
1381
SET
541
-000
-091
-000
0F
00
60
0M
0M
OM
C00
KCB
031
-001
-865
401
340
-012
-380
-U0
406
4004
0M
036
540
945
344
531
033
6:
-021
-M0
6CC
-021
021
320
40M
0H
undirg
..09
031
1981
1849
1M6
10M
0:0
CM
011
0000
1031
1001
4:34
560
860
-060
360
-380
-330
-601
1030
031
323
1921
4641
0:3
4.09
340
69
0.0
031
010
3M
par FLAA
660
580
1627
AMc
4:00
218
-300
X2N
32V
-MM
6.09
1863
-081
-031
-010
100
40
(000
4:30
-381
CCB
400
KG5
081
K05
015
140
09
02
3.0
6.0
0.0
601
030
-441
Lange
-665
0331
1860
10:00
360
0MM
0:0
0:00
CON
011
018
330
OFF
0:30
0.34
0.30
-069
-000
031
0211
631
-331
3.0
380
60
934
0:00
0:11
030
C.54
201
S21
346
1.47
634
541
PRISSTION
660
-660
-131
-860
-38
380
-3K
-300
4.13
4.11
0.6
-030
4031
4:30
301
-281
-301
319
-331
+E.3
(013
545
019
165
332
6%
1869
-880
1849
138
030
337
4F1
0/0
1811
COR
K10
1840
1330
541
-600
-663
349
MG
012
-28
400
4660
421
849
4021
+282
300
668 060
531
340
68
382
382
0M
0
541
633
03
6C0
000
616
025
340
336
EXT
433
SET
Figure 6: The complete matrix of relative changes for all classifiers. Ordering is determined by clustering to group similar classifiers.
15

---

Published in Transactions on Machine Learning Research (01/2025)
A.1 Comparison with saliency maps
Counterfactual
Input Gradients
Integrated Gradients
Guided Backprop
pointy nose
Prediction explanations for this image
big. nose
Figure 7: Example of saliency maps failing to provide meaningful localization when the concepts overlap with cach other. Here, saliency map methods localize around the nose which doesn't provide the ability to distinguish between big or poinly nose prediction.
Saliency maps for feature attribution could potentially be used perform similar analysis by looking at relationships between their generated heatmaps. Here, present example that demonstrates their limitation. The attribution methods input gradients, integrated gradients (Sundararajan et al. 2017) and guided backprop (Springenberg et al. 2015) are used to explain the classifier predictions of pointy nose and big nose.
In Fig. the salient areas for pointy_ nose and big nose big_nose are the same and cannot be disambiguated. The saliency maps only present information as pixel importance which overlaps because both classifiers use features the Due to these very similar heatmaps, comparing them would not allow to conclude that the classifiers are using different features. We observe in Fig, 2 that these two classifiers have inverse relationship. Using CF alignment, we can gain deeper understanding of what features the model is using, and we can better reason about why the decision was made.
16

---

Published in Transactions on Machine Learning Research (01/2025)
A.2 Comparison with other CF generation methods
Using the biased classifier from $43 can capture its bias. As a baseline, we use the Wachter et al. (2017) method, which generates a new x' that minimizes the classificr's prediction as well as the proximity to the input z. The objective is as follows:
evaluate whether another counterfactual (CF) generation methods
argmin X (xx,x' + ((())-)(
(5)
The distance function d is chosen to the Frobenius norm in order to match pixels in the image. While this approach was not designed for images still able to change the base classifiers prediction our experiments. The CF generated with this method shown in Figure S and does not appear generate features that capture aligmment with the classifier's bias. This method operates in similar way to adversarial examples which introduce imperceptible changes.
X (omlling0.. suuched_eeyebror
domiingg0..eccheee eyebrows)
*smiing 0.3arched eyebrows (1.3)
arthed eyeerows arshec__yeebruus (0.07)
1.1
Modal inadiction
0.9
0.8
0.7
0.5
0.0
1.0
1.2
Nodel Prediction
Figure 8: Comparison with the Wachter counterfactual generation method, smiling classifier with a known bias of arched eyebrows is used to observe if this bias can be detected. The Wachter counterfactual doesn't 't appear to modulate the imput image in human interpretable way, instead many small swirls appear on the face which change the prediction for smiling but not for arched eyebrows.
17

---

Published in Transactions on Machine Learning Research (01/2025)
A.3 Rectifying bias by composing classifiers
This section demonstrates the use of CF alignment to fix model bias in classifiers. We reuse the relative change between model predictions loss function that be minimized.
This section just serves as an example to better understand a use of the CF alignment idea. We don't claim this method is competitive to other methods which correct model bias and spurious correlations. Related approaches include averaging model weights Wortsman et al. (2022) or methods of invariant optimization Sagawa et al. (2020); Krueger et al. (2021): Rieger et al. (2020); Zeng al. (2023) and approaches for change their response to specific samples that contain relevant features as shown in 44.3 By using this additive composition approach we avoid the variance of model training which makes it easy and reproducible to perform experiments.
balanced dataset sampling Singh et. al. (2021). Composing classifiers
First, we retonstrate a single example of correcting bias. In Fig. g the CF image and CF aligmment, plot for the heavy_makeup base classifier show unexpected reductions in lip size. Composing the model with the big lips classifier using a negative coefficient causes the classifier to lower any feature change focused on the lips as this would increase the prediction of the composed classifier (where the goal of the CF is to decrease it). We can also visually observe other features that change, such as skin color, bushy eyebrows, and the amount that the mouth is open.
1.-
Phatuy systrous
0.-
big (2.29) bo2ny__naaeaa wak.oon (1.)
0.4
(a) As makeup is removed from the face, the lip size also shrinks.
0.0
0.
arthes, cysbrows
0.6
03
(b) The lip size remains the same as the makeup is removed.
Figure 9: An example of a big__ip lips spurions correlation being corrected for the feavy__makee classifier.
Next, this unbiasing scaled up to a larger number of samples and classifiers. A collection of 12 classifiers that contain varying spurious relationships to big nose base classifier (shown in Fig. 10p) are modified to remove their bias until their relative change is close to 0 as possible. This experiment is performed with train, validation, and hold out test set in order to demonstrate the generalization of this unbiasing process to unseen data.
The classifiers are modified using a single coefficient (3) such that the resulting classifier is
(arget( ftargec (x) Bfbsg nose
(6)
Optimization to compute each performed using a pscudo gradient descent where the gradient is approx- imated by the mean relative change between the target classifier and the big the relative change from 3, scaled by learning rate, the relative change (1)) with that classifier will be
big_nose nose classifier. By subtracting
18

---

Published in Transactions on Machine Learning Research (01/2025)
train
Mean Relabive Change (big, nosee
volid
0.0
Mmmm
-1.0
Iteration
Figure 10: Training curves showing mean relative change during the optimization of a classifier with respect to the big_nos nose classifier during each iteration. The red vertical line the early stopping point based on the validation set. This demonstrates how bias can be quantified and corrected using CF alignment.
reduced. All together our training objective (including momentum) is
pn 10.0011p(rarga Jbig_ nose) + 0.1
We find that using small minibatches of 10 samples works well because the computation time for cach CF can take over 1 second on GPU. Additionally, training with samples which induce small change in the base classifier during the CF generation process can be challenging. As the classifiers are modified, this base change is reduced, which causes the relative change to become more erratic and prevents the optimization from converging. To prevent this, we use samples with a base change 0.6.
The resulting training metrics in Fig. 10 show model biases being minimized similar to differentiable training Joss. Looking the mean relative change allows summarize the bias of the model with respect to the big_nose nose classifier who's relationship the train, valid, and test sets, indicating some degree of overfitting. Farly stopping (red vertical line) is used on the validation set to determing the optimal parameters. We use momentum during training to average over the noise.
aim remove. We observe bias generalization gap between
The resulting reduction in bias is shown per classifier before and after training Fig I1 10 of the 12 classifiers being optimized have the bias considcrably reduced (closer to 0) except for attractive and sig_nose, which could be the result of overlapping feat ures used by both classifiers that cannot be changed.
19

---

Published in Transactions on Machine Learning Research (01/2025)
mouth, Beoutt__slgghtt w.slllggllll....... .open
A) Baseline
eyebrows
arched eyebrows
high. ww..cheeebnesssss
wearing, Jipstick
oval, face
pointy,
young
male
mouth_ saahh_slightly.pen
1.00
-0.20
-0.00
-0.04
-0.16
0.52
0.21
0.04
0.03
0.03
0.24
0.04
-0.07
big_nose
0.08
1.00
0.00
0.82
-0.01
0.61
0.55
-0.42
-0.06
-0.34
-0.36
-0.56
-0.02
male
-0.01
0.24
1.00
0.11
-0.02
-0.02
-0.09
-0.09
-0.39
-0.20
-0.03
-020
-0.01
eyebrosss
0.07
-0.05
0.00
1.00
0.18
-0.01
-0.06
0.06
0.00
0.00
-0.03
0.12
0.19
arched_eyebrrow
-0.01
0.25
0.00
0.02
1.00
-0.00
0.00
0.24
0.00
0.02
-0.33
-0.06
-0.01
smiling
0.33
0.25
-0.00
-0.00
0.05
1.00
0.90
0.02
0.00
0.01
0.12
0.13
-0.27
nigh_cheebbones
0.02
0.05
-0.01
-0.10
0.38
0.04
1.00
0.20
0.03
0.11
-0.06
-0.01
-0.16
pointy_nose
-0.05
-0.86
-0.00
0.08
-0.01
0.29
0.28
1.00
0.00
0.03
0.10
0.49
-0.00
lipstik
-0.05
-0.06
-0.41
-0.02
0.41
-0.01
0.02
0.14
1.00
1.26
0.06
0.52
-0.03
heavy_makeup
0.27
-0.03
-0.01
0.05
0.14
0.03
0.24
0.11
0.39
1.00
0.31
0.12
0.00
oval_face
0.11
-0.41
-0.00
0.16
-0.20
0.36
0.22
0.24
0.00
0.01
1.00
0.56
0.01
attractive
-0.04
-0.49
-0.08
0.20
0.08
0.09
-0.09
0.22
0.29
1.04
0.67
1.00
0.24
young
-0.02
-0.18
0.38
0.01
-0.02
-0.02
-0.19
-0.03
-0.25
-0.05
0.05
0.56
1.00
mouth, slightly. open
bushy, eyebrows
arched eyebrows
high. ch..ekbeeeessss
wearing Jipstick
makeup
Optimized
big_ nose
oval face
smiling
pointy_
young
male
mouth_sliee sligggttttytopnnn
1.00
-0.01
0.00
-0.02
-0.13
0.56
0.23
0.02
0.01
-0.01
-0.02
-0.03
-0.09
big_nose
0.08
1.00
0.00
0.82
-0.01
0.61
0.55
-0.42
-0.06
-0.34
-0.36
-0.66
-0.02
male
-0.01
-0.00
1.00
0.09
-0.04
-0.02
-0.10
-0.05
-0.34
-0.16
0.00
-0.12
-0.00
eyebroes
0.07
-0.05
0.00
1.00
0.18
-0.01
-0.06
0.06
0.00
0.00
-0.03
0.12
0.19
arched_eyebrows
-0.01
0.00
-0.00
0.01
1.00
-0.01
0.00
0.31
0.01
0.06
-016
0.08
-0.00
smiling
0.44
-0.01
-0.00
0.00
-0.08
1.00
0.79
0.02
0.01
0.02
0.19
0.35
-0.13
high_cheekben cheekboneeesss
0.02
0.01
-0.02
-0.12
0.40
0.04
1.00
0.20
0.04
0.14
-0.03
-0.01
-0.16
pointy_nose
-0.07
0.11
0.00
0.09
0.01
0.38
0.69
0.94
-0.00
-0.01
-0.07
-0.13
-0.12
Jipstic
-0.05
-0.00
-0.40
0.00
0.43
-0.01
0.03
0.11
1.00
1.26
0.06
0.48
-0.03
heavy_makoup
0,27
-0.03
-0.01
0.05
0.14
0.03
0.24
0.11
0.39
1.00
0.31
0.12
0.00
oval_face
-0.06
0.04
-0.00
0.17
-0.14
0.41
0.43
-0.22
0.00
0.01
0.99
0.45
0.01
attractive
-0.04
-0.49
-0.08
0.20
0.08
0.09
-0.09
0.22
0.29
1.04
0.67
1.00
0.24
young -0.02
-0.02
0.75
0.01
-0.01
-0.02
-0.20
-0.07
-0.49
-0.13
0.04
0.54
1.00
Figure 11: CF alignment matrices before and after optimization to remove the bias for big
hold
out test set. The figure shows a dramatic reduction in rolative change for most classifiers and big_nose nose with limited residual impact on other relationships.
20

---

Published in Transactions on Machine Learning Research (01/2025)
-A00
2:11
-912
0.54
0:34
0.31
941
C:00
120
8400
0.31
-150
840
-341
+820
524
1:20
-0.0
105
3.50
0221
340
0.11
340
-180
360
340
160
3/9
6.6
-008
5.66
031
-080
1800
850
981
941
1861
0000
948
600
0.21
-M6
10.4
-XX
660
-924
-244
-028
-C.
300
SAPU
JAKS
1.00
0.00
1.20
1:30
3.00
4:1
-0.0
51.
-420
2,24
3,29
0:30
6:28
-0:00
-6.08
2:08
5:22
5:09
3.11
0.00
4030
4.30
-0.00
3.23
4.30
+000
820
0.30
3.2
$1.
0.00
026
0.01
202
4.34
330
05A
0.0
124
5.00
138
3.6
5.24
-629
-081
522
821
07
0.2
4600
6:3
608
-6:8
-08
+30
621
032
3.4
107
230
002
088
3.0
89
0:34
608
-608
6.6
-204
0.80
842
301
301
6.1
0:21
-8.0
-600
06
4036
4.4
800
0/3
3:8
-8.4
0.6
1407
A3C
958
335
140
340
03
342
1330
014
33
0.44
4608
1008
502
331
0:0
0:0
-60
4.3
411
-848
0.2
802
331
600
4.2
1.2
280
111
(a) pointy nose
(b) smiling
(c) aftractive
Figure 12: Visualization of the training from AA.3 reducing the bias of big nose. The model being evaluated is flarget( (x) = fturget(z) + 3fbig nose(x) whore specified by the cocfficient column. The metric here is relative change computed between the ftarget (I:); where largel is specified in the caption, and the classifier specified by the column header. The top row is from the training set performed on batches (which explains the variance) and the second row is the results on the entire validation set. The row of cach table is for a specific iteration. The iterations proceed downward. The training of allractive shown here does not result in a debiased model, the relative change diverges from O indicating a failed training.
21


Reviews:
[{"review_id": "dnxlGN1Jq5", "reviewer": "Reviewer_93D4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The method itself is easy to understand. \n- As mentioned in the related work, the idea of modifying latent representations to produce counterfactual samples already has a basis in the literature. \n- The results on the CelebA dataset are intuitively easy to understand.\n\nWeaknesses:\n- It would seem that if the base classifier uses spurious correlations, you may also end up creating counterfactual samples which change other features of the sample. Put another way, given I find a high relative change metric for a classifier $f_1$ and $f_b$, why can I conclude this is a problem with spurious correlations in $f_1$ alone and not $f_b$ also? To draw a conclusion like this, I would have to know $f_b$ does not rely on spurious correlations, which if assessed by relative change metric would start to feel like a circular argument. \n- I find the experimental comparisons to be slightly lacking in that whilst the authors compare against saliency maps and correlational metrics, they do not compare against other counterfactual explanation methods. The relative change metric could be defined for any counterfactual sample and it would be informative to see how their method compares against such alternatives. \n- It seems unclear to me how you would asses what a large or small value for the relative change metric is apart from relatively to other evaluations of the same metric. This can be seen in the fact that between experiments there is a large difference in the magnitude of the relative change metric.\n- The definitions in section 3 could be clearer. The relative change metric should be brought out as an actual definition given its importance to the paper. Also at later points in the paper the authors seem to refer to the same metric as counterfactual alignment. If this is in fact the same this should be included in the definition.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors use an adapted version of the latent shift technique from Cohen 2021 to asses the impact of spurious features on classifier performance. In order to do this, the authors propose a method which uses an encoder/decoder model and a classifier which predicts a base feature. To the form counterfactual sample, they adjust the latent representation by the gradient of the classifier composed with the decoder, and then feeding the adjusted representation back into the decoder. They use this to form a metric which measures how much a new classifier of a different features uses the base feature by seeing the new classifiers prediction changes on the counterfactual sample.\n\nThe authors then give an experimental evaluation of their method. First they look at aggregate statistics for their metric over a dataset, demonstrating that their metric can go further than correlational based methods. They then examine specific examples, comparing against saliency maps. Finally, they do a series of experiments on group distributional robustness tasks, showing that state of the art models in the field are less susceptible to spurious correlations as measured by their metric.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Can the authors comment on what issues would occur if $f_b$ used spurious features?\n- Can the authors comment on if the size of the metric can only be assessed relatively? \n- Can the authors include comparisons with other counterfactual explanation methods? Comparing against Wachter 2018 would be good for a simple baseline but ideally also against some of the counterfactual explanation methods listed in the related work.\n- Improve the clarity of definitions in section 3.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7r0HrM6BN1", "reviewer": "Reviewer_vier", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- CF Alignment methodology is elegant, rigorous, and can be reused for a number of tasks - also shows promising results\n- Likewise, Relative Change metric is a flexible a reusable metric when comparing classifiers\n- Extensive study of specific examples and comparison with other methods (e.g. Saliency maps) gives interesting insights on the behavior of classifiers when encountering CFs\n- Link with Robust Optimization is interesting and unifies the whole framework \n\nWeaknesses:\n- I would have appreciated a further study and insights on the aggregated statistics and performance under them - i.e. section 4.1 and Figure 2\n- No code for reproducibility", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a novel method of Counterfactual generation based on perturbing the encoding of images using the gradient of a base classifier. This methodology is then used coupled with a new Relative Change metric to find specific instances of classifiers using spurious correlations to output their predictions. It also provides an analysis of specific examples of these spurious correlations. Finally, this methodology is also used with Robust Optimization methods to demonstrate their better generalization capabilities as spurious correlations are reduced.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weaknesses section.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "et9fs0xpUZ", "reviewer": "Reviewer_zg5u", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n1) The study considers a relevant and long-standing problem (spurious features).\n\n2) Under a certain assumption (see weakness 1), the proposed approach seems reasonable. The experimental results are convincing.\n\n3) Plenty of experiments on proper computer vision data sets, including future release of the source code for reproducibility.\n\n4) The manuscript is written in simple language and notation, making it accessible to a broad audience with diverse backgrounds in machine learning.\n\n**Weaknesses:**\n\n1) It seems the work hinges on the strong assumption that this *base classifier* is not affected by spurious features. This assumption should be made explicit and discussed critically. For instance, if both the *base classifier* and the *downstream classifier* use the presence of a beach to not predict the *cow* class, the relative change metric will not detect this spurious feature.\n\n2) The writing must be improved. Major parts of the paper are not clear to me (see weakness 3 and requested changes) and it may well be that my understanding of the proposed method is incorrect. Specifically, it would be great if the authors could provide more concrete examples and clear definitions.\n\n3) It is still not clear to me what the advantage of this *relative change metric* is in comparison to other metrics such as correlation of the change between base and downstream classifier. The *relative change metric* is stated as the main contribution of the study, so there should be a greater discussion around this aspect.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a method called *counterfactual alignment* for detecting spurious features in image classification via a novel *relative change metric* . This method considers a pair of classifiers: a base classifier and a downstream classifier. In broad strokes, a counterfactual explanation in the latent representation of an auto-encoder is run on the base classifier and compared to the change in the prediction of the downstream classifier on the output counterfactual image. A spurious feature is detected if this relative change in the prediction of the classifiers deviates far from $1$. The method is assessed empirically on CelebA and Waterbird data sets, where the method is demonstrated to detect spurious features reliably and in a localized way. This lies in contrast to other explainable AI methods such as saliency maps, which fail to localize spurious features. In addition, the proposed method indicates that robust methods like GroupDRO, JTT, and FLAC are indeed less reliant on spurious features.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Critical requested changes are highlighted by the comment *(critical)*.\n\n**Abstract**:\n\n1) *face-attribute face-attribute*. This seems to be a typo.\n\n**1. Introduction**:\n\n1) *Shortcut learning can\nlead to models making decisions based on factors not aligned with expectations of the model creator.* What is a *model creator*? Please clarify.\n\n2) *of an image (Pearl, 2009)* I believe that this is not the right reference. Pearl's construction of counterfactuals is interventional. It seems the authors instead use *counterfactual explanations* [1] , which can be derived from (non-interventional) *backtracking counterfactuals* (see [2]). Please clarify or change reference.\n\n3) *using a singular fixed autoencoder as a reference point.* What is a *singular autoencoder*? Please clarify.\n\n4) *This shared feature usage can prompt us to investigate unexpected relationships*. So if a feature is used by multiple models, that means there are unexpected relationships? I cannot follow. What is even meant by *unexpected relationships*? And why is that the case? Please clarify, maybe by providing a concrete example.\n\n5) *We propose CF alignment to reason about the feature relationships between classifiers* It is not clear to me what is meant by *feature relationships between classifiers*. Please clarify.\n\n**3.1 Relative Change**:\n\n1) (critical) *This decision was motivated by the observation that correlation metrics occasionally yielded false positives when only a slight change in the prediction of $f_1$ occurred compared to the base classifier $f_b$*. I do not understand this explanation. It seems to me that correlation measures differences between classifiers on a population level, in contrast to this relative change metric, which is instance-specific. However, a population-level variant of the proposed metric is considered in section 4.1. It would be great if this could be discussed in greater detail.\n\n**2. Experiments**:\n\n1) (critical) *The pre-trained face classifiers used in this work are sourced from...* How do these classifiers differ from one another? By random seed or by architecture? I believe that this is quite crucial.\n\n2) (critical) What I am missing in all experiments is a clear description of which model is used as base classifier and which one(s) is/are used as downstream classifier(s).\n\n**Figure 2**: \n\nI am not sure if I understand this figure correctly. Does that mean that the target label of the base classifier differs from the target label of the downstream classifier? How would this help in finding spurious features?\n\n**4.1 Aggregate statistics over a dataset**\n\n1) *N=400 per class*. What is $N$? The number of different classifiers?\n\n**4.2 Studying specific examples**\n\n1) (critical) *inverse alignment with big_nose* What is meant by *inverse alignment*? How is *alignment* even defined? If the authors introduce a term, it would be great to state explicitly what it means. It would be great if the authors could write something like *We say that classifier 1 and classifier 2 are aligned if ...* and analogously for *inverse aligned*. From Figure 3, I would guess that *alignment* means that the relative change metric is monotonically inreasing for the two classifiers (?).\n\n2) *rosy_cheeks is a common aligned classifier which does not appear to have an obvious reason* I do not understand this sentence. What is meant by *obvious reason*? Also, as already mentioned, *aligned classifier* is not defined.\n\n**4.5.2 GroupDRO on CelebA**\n\n1) *we were unable to generate pre-trained weights using the provided code due to memory issues that we were unable to resolve.* Not a requested change, but I just would like to appreciate the honesty of the authors here. Quite refreshing.\n\n**6 Conclusion**\n\n1) *We also observe that if uncorrelated attributes are presented to the classifier, the classifier may still construct spurious feature relationships.* This is an intersting observation. Do the authors have a hypothesis for why this happens?\n\n[1] Wachter, Sandra, Brent Mittelstadt, and Chris Russell. \"Counterfactual explanations without opening the black box: Automated decisions and the GDPR.\" Harv. JL & Tech. 31 (2017): 841.\n\n[2] Kladny, Klaus-Rudolf, et al. \"Deep Backtracking Counterfactuals for Causally Compliant Explanations.\" Transactions on Machine Learning Research (2024).", "broader_impact_concerns": "No broader impact concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors adjusted the experiments as discussed and revised the text to address the comments related to the omissions and discrepancies between the claims and the evidence, with reviewers noting that the comments have been sufficiently addressed.  All reviewers recommend acceptance and justify it by stating that 'the methodology is novel and interesting', 'being simple to implement and understand', 'sufficient experimental results', 'elegant, rigorous methodology'
Decision: Accept (poster)