Explaining Visual Counterfactual Explainers
Authors: Diego Velazquez
Pau Rodriguez
Alexandre Lacoste
Issam H. Laradji
Xavier Roca
Jordi Gonzàlez
Venue: Accepted by TMLR ()

Abstract:  Explainability methods have been widely used to provide insight into the decisions made by statistical models, thus facilitating their adoption in various domains within the industry. Counterfactual explanation methods aim to improve our understanding of a model by perturbing samples in a way that would alter its response in an unexpected manner. This information is helpful for users and for machine learning practitioners to understand and improve their models. Given the value provided by counterfactual explanations, there is a growing interest in the research community to investigate and propose new methods. However, we identify two issues that could hinder the progress in this field. (1) Existing metrics do not accurately reflect the value of an explainability method for the users. (2) Comparisons between methods are usually performed with datasets like CelebA, where images are annotated with attributes that do not fully describe them and with subjective attributes such as ``Attractive''. In this work, we address these problems by proposing an evaluation method with a principled metric to evaluate and compare different counterfactual explanation methods. The evaluation is based on a synthetic dataset where images are fully described by their annotated attributes. As a result, we are able to perform a fair comparison of multiple explainability methods in the recent literature, obtaining insights about their performance. We make the code and data public to the research community.

Published in Transactions on Machine Learning Research (04/2023)
Evaluating Visual Counterfactual Explainers
Diego Velazquez Computer Vision Center
doelazquez@evve.ukk uab.cat
Pau Rodriguez* Apple
pau.rodriguce:@pppe..co
Alexandre Lacoste
alezandre. hwwp://rw..ucceeeeeeeeeeeee.ee..
ServiceNow
Issam H. Laradji ServiceNow
jssm.lenadji@sersiccno..ccmm
Xavier Roca
zavier.rocaa@uall
Universitat Autonoma de Barcelona
Jordi Gonzalez
jordi pordi.onzzaez@uuabcco cat
Universitat Autonoma de Barcelona.
Reviewed OpenReview: https: https://openrevieu
openreview. net/ forum? id= RYeRRNRRRR
Abstract
Explainability methods have been widely used to provide insight into the decisions made by statistical models, thus facilitating their adoption in various domains within the industry. Counterfactual explanation methods aim improve our understanding of model by perturbing samples in way that would alter its response in an unexpected manner. This information is helpful for users and for machine learning practitioners to understand and improve their models. Given the value provided by counterfactual explanations, there is a growing interest the rescarch community to investigate and propose new methods. However, we identify two issues that could hinder the progress this field. (1) Existing metrics do not accurately reflect the value of an explainability method for the users. (2) Comparisons between methods are usually performed with datasets like CelebA, where images annotated with attributes that do not fully describe them and with subjective attributes such " "Attractive". In this work, we address these problems by proposing evaluation method with a principled metric to evaluate and compare different counterfactual explanation methods. The evaluation based on a synthetic dataset. where images are fully described by their annotated attributes. As a result, we are able to perform fair comparison of multiple explainability methods in the recent literature, obtaining insights about their performance. We make the code and data public to the research community.
1 Introduction
The popularity of deep learning methods is tostament to their offectiveness across a multitude of tasks in different domains. This effectiveness has led to their widespread industrial adoption (e.g., self-driving cars, screening systems, healthcare, etc.), where the need to explain a model's decision becomes paramount. However, due to the high level of complexity of deep learning models, it is difficult to understand their decision making process [Burkart & Huber 2021). This ambiguity has slowed down the adoption of these systems in
* Work done while ServiceNow. https://github. com/dvd42/Bex

---

Published in Transactions on Machine Learning Research (04/2023)
Original
Latent Space
Pixel Space
W
W
i
J
i
Figure 1: Comparison of perturbations performed in latent and pixel space. Note how the perturbations performed in latent space convey meaningful information (e.g., changes in the font character), while the ones performed in pixel space resemble adversarial attacks and do not provide any valuable insights into the classifier's reasoning.
critical domains. Hence, in order to ensure algorithmic fairness in deep learning and to identify potential biases in training data and modcls, it is key to explore the reasoning behind their decisions (Buhrmester et al. 2021
In an attempt to convincingly tackle the why question, there has been surge of work in the field of explainability for machine learning models (Joshi et al.) 2018 Mothilal et al. 2020 Rodríguez et al. 2021 The goal of this field is to provide explanations for the decisions of a classifier, which often come in the form of counterfactuals. These conterfactuals provide insight as to why the output of the algorithms is not any different and how it could changed (Goyal et al.) 2011)). Basically a counterfactual explanation answers the question: " "For situation X why was the outcome Y and not Z", describing what changes in a situation would have produced different decision.
Ideally, counterfactual methods produce explanations that are interpretable by humans while reflecting the factors that influence the decisions of a model (Mothilal et al.) 2020). So given an input sample and model, counterfactual explainer would perturb certain attributes of the sample, producing a counteresample í.e., counterfactual, that shifts the model's prediction, thus revealing which semantic attributes the model is sensitive to. In this work, we focus on the image domain given the recent surge of explainability methods for image classifiers (Joshi et al.] 22018] RRoriguez et al. 2002 [Singla et al., [0019 Chang et al.) 2018). A particular challenge of the image domain is that changes in the pixel space and resemble adversarial attacks (Goodfellow al.] 2014b) (see Figure [ for example), explainers tend to search for counterfactuals in latent space produced by, e.g., variational autoencoder (VAE) (Kingma & Welling 2013), or by conditioning on annotated attributes DDnton et al. 2019: Joshi
difficult to interpret current
ct al. 2018 Singla et al. 2019 Rodriguez et al. 2021
Although explanations produced the latent space are easier to interpret than the pixel space, they depend on the chosen or learned decomposition of the input into attributes latent factors. In the case of VAEs, these factors could be misaligned with the real underlying generating process of the images. Moreover, different methods in the literature rely different autoencoding architectures generative models to infer semantic attributes from images, which make their counterfactual search algorithms not comparable. In the case of annotations, since datasets do not provide access to the whole data generating process, they tend lo focus on arbitrary aspects of the input (such as facial attributes for CelebA (Liu et al. 2015b)), ignoring other aspects that could influence a classifier's decision boundarics such as illumination, background color, shadows, etc. This raises the need for evaluating explainers on known set of attributes that represent the real generative factors of the input. In this work, we propose to fill this gap by introducing new
2

---

Published in Transactions on Machine Learning Research (04/2023)
explainability benchmark based on synthetic image dataset, where we model the whole data generating process and samples are fully described by controlled set of attributes.
An additional challenge when evaluating explainers is that there is no consensus on the metric that should be used. While there has been some effort to provide a general metric to evaluate explainers (Mothila et al, 2020 Rodríguez et al., 2021), most. of the proposed metrics could be easily gamed to maximize the score of given explainer without actually improving its quality for a user. For example, since current metrics reward producing many explanations, the score can be increased by (i) producing random samples that cannot be related to the ones being explained. This has motivated measuring the prozimity of explanations (Mothilal
ct al. 2020). (ii) Repeating the same explanation many times. This motivates measuring diversity Mothilal et al., 2020j. However, we found that it is possible to maximize existing diversity measures by always performing the same perturbation to a sensitive attribute while performing random perturbations to the rest of attributes that describe a sample. As a result, although one counterfactual changing the sensitive attribute would suffice, an explainer could obtain a higher score by producing more rodundant explanations. We argue that instead of providing many explanations, explainers should be designed to produce the minimal set of counterfactuals that represent each of the factors that influence model's decision. (iii) Providing uiinformatvv or trivial explanations (Rodriguez et al. 2021). This has motivated us to compare the model's 's predictions with those expected from an "oracle classifier". In terms of information theory, unexpected events tend to be more informative. In this sense, repeating the same explanation multiple times provides little information and increasing diversity increases the amount of information. In addition, the amount of information of an event is usually measured with respect to some underlying probability distribution. Tf we know that a classifier is trained to detect a certain object, we expect the classifier to learn the conditional probability distribution (label given input) of the training set. Thus, deviations from this distribution are unexpected and informative. However, we do not know the real label for new samples and thus, we need to resort to some form of oracle or ground truth classifier in order to compare it with the model being explained. This ground truth classifier must have access to the real data generating process in order to infer the correct class from images, and the data generating process of image-label distributions is typically a causal process governed by the laws of physics. That is why we refer to this classifier as "causal classifier", In this work, we address these problems by proposing a fair way to evaluate and compare different counterfactual explanation methods.
Our contributions can be summarized as follows: (i) we present benchmark to evaluate counterfactuals generated by any explainer in a fair way (Section 3 (ii) we offer insights on why existing explainability
methods have strong limitations such as an ill-defined oracle (Section 3.3 ; (iii) we introduce a new set of
metrics to evaluate the quality of counterfactuals (Section 3.4;; I; and (iv) we evaluate 6 explainers across different dataset configurations (Section
2
Related Work
Explainability methods. Since most successful machine learning models are uninterprotable (He al.
2016 Jégou et al. 2017; LeCun al., 1989], modern explainability methods have emerged to provide explanations for these types of models, which are known as post-hoc methods. An important approach to post-hoc explanations is to establish feature importance for a given prediction. These methods (Guidotti let al. 2018 Ribeiro et al. 2016: Shrikumar et al. 2017: Bach et al., 2015p involve locally approximating the machine learning model being explained with a simpler interpretable model. However, the usage of proxy models hinders the truthfulness of the explanations. Another explainability technique is visualizing the factors that influenced a model's 's decision through heatmaps (Fong et al. 2019 Elliout et al., 2021 Zhou et al. 2022). Heatmaps are useful to understand which objects present in the image have contributed to classification. However, heatmaps do not show how areas of the image should be changed and they cannot. explain factors that are not spatially localized (e.g., size, color, brightness, etc).
Explanation through examples or counterfactual explanations addresses these limitations by synthesizing alternative inputs (counterfactuals) where a small set of attributes is changed resulting in a different. classification. These counterfactuals are usually created using generative models. A set of methods condition the generative model on attributes annotated in the dataset by using a conditional Generative Adversarial
3

---

Published in Transactions on Machine Learning Research (04/2023)
Table I: Comparison of explainers considered in this work. First column indicates whether counterfactuals are found with gradient descent. Second column indicates whether the explainer takes into account changes in pixel space during optimization (e.g., visual similarity loss) Last column indicates if the explainer performs feature selection to generate counterfactuals.
Method
Gradient based Optimizes x-space
Feature selection
DICE
Mothilal et al.
2020
X
X
DiVE
Rodriguez al.
2021
x
GS Laugel al. 2017
v
StylEx
Lang et al. 2021
X
x
Latent CF
[Balasubramanian et al.
2020
V
x X
xFM Joshi al. 2018
X
Network (GAN) (Joshi al. 2018 Liu et al. 2019 Sauer & Geiger 2021 Van Looveren et al. 2021 Yang
et al. 2021). However, this approach restricts the explanations to the provided attributes which do not reflect the entirety of the image properties, making the applicability of these methods challenging where annotations are scarce. In order to generate counterfactuals without recurring to annotated attributes, another set of methods uses VAFs or unconditional GANs Goodffelow et al. 2014a) that do not depend on attributes
during generation (Rodríguez et al. 2021 Denton et al.) 2019 Pawelezyk et al. 2020 Perez et al. 2018 Mothilal et al. 2020). See Table for a comparison of the methods considered in our work.
Explainability Benchmarks. DIVE (Rodríguez et al.) 2021) and DICE (Mothilal et al. 2020 propose metries that allow researchers to evaluate the quality of an explanation. These metrics evaluate the proximity of explanations to their original sample, and how diverse these are. Unfortunately, they are easy to game. For example, an explainer could maximize diversity by always modifying the same counterfactual attribute but randomly perturbing other non-counterfactual attributes to produce new redundant explanations. We propose a more general, harder to game metric that allows us to evaluate a set of explainers in order to identify their strengths and weaknesses through fair comparisons. Further, the set of attributes of a dataset can influence the evaluation of the explainability methods. CelebA (Liu et al.] 2015a is a common dataset used for generating counterfactual explanations (Rodriguez al. 2021 Denton et al., 2019), and it is labeled with a series of attributes, such as "Attractive", that fail to fully describe the true underlying factors that generated the images (e.g, illumination, occlusions, contrast, etc). Likewise, there is no guarantee that unsupervised disentanglement methods such as VAEs identify the true factors of variations without making strong assumptions (Arjovsky et al., 2019). We sidestep these problems by evaluating all explainers in a common latent space with known attributes that fully describe the samples. Recently [Pawelzykk et al. (2021) published a benchmark (CARLA) with an exlensive comparison of several counterfactual explanation methods across 3 different tabular datasets. Our work differs from CARLA in three important ways: (1) we propose principled metric to compare counterfactual explanation methods, (2) we introduce new synthetic benchmark that allows comparing multiple explainers in a fair manner in the same latent space. (3) We focus counterfactual visual explanations, which require access to common latent space for fair comparison since pixel-level counterfactuals are difficult to interpret (e.g., adversarial attacks).
3 Problem Setup
In the following lines we describe a principled framework to quantify the quality of counterfactual explainers
and show how it can be applied to compare multiple methods in the literalure. In Section 3.1 we define the
data generation process, in Section 3.2 we define the counterfactual generation process, in Section 3.3 we define the concept of optimal classifier used to compare the predictions of a model, and in Scction 3.4 we define the metric used to evaluate counterfactual explanation methods. A notation table can be found in Table 4
4

---

Published in Transactions on Machine Learning Research (04/2023)
3.1 Data generation
Many explainability methods in the literalure are designed for the image domain (Rodriguez et al. 2021; Joshi et al. 2018 Lang et al.] 2021 Singla et al.) 2019) Chang et al.) [018). In this area, most datasets can be described with a data gencrating process where a set of latent variables (2) result in an image (x) and a corresponding label (y), see Figure 2a However, not all the latents that generate the image have an impact on the label (Zina). For example, the image brightness does not affect the presence of a dog. In addition, some latents can be correlated with the label (Zcorr). For instance, whenever there is a dog there is usually dog collar. Formally, we consider a data generating process where set of latent variables E R are sampled from a prior p(z), and generator that produces images p(z/z). Labels are generated using P(U/Rcausa)) where Zeausal is subset of z containing direct causal parents of y (Figure 2aj. We also define Zcorr as the set of attributes that are correlated to but not part of Zrausal Sometimes, these correlated attributes may have stronger predictive power, but relying on them would lead to unreliable predictions. For instance using the sky background for classifying airplanes. To generate datasets, we rely on a structural causal model (SCM) (Pearl 2009), corresponding to a sequence of stochastic equations producing random variables based on the causal parents in the causal graph as described in Figure 2a
In order to obtain a known mapping between z and we propose to leverage synbols (Lacoste et al. 2020 synthetic dataset generator with many controllable attributes (font, character, color, rotation, size, etc). In addition, using a synthetic dataset allows us to control the effect of z on x and specify the amount of change in r relative to the amount of change in z (and vice-versa). Using synbols, we train an image generator x = =(> which is used to generate subsequent datasots. In summary, given an image we task an encoder q with predicting the altributes that describe the image (z). We also train a generator g reconstruct æ from z. Finally, we leverage g to generate new datasets. The generator g is provided to the explainers to offer a differential mapping from to x. We believe this is strength of our benchmark compared to using datasets of natural images, since it allows for unambiguous generation of synbols due to the unique attribute space for sampling.
3.2 Counterfactual generation
Given an image I and classifier f(x), counterfactual explanation method (explainer) produces x', a perturbed version of x that shows some insight about the sensitivity of f to the semantic attributes that describe x. The perturbation is commonly performed on a learned latent space z. In general, explainers are tasked to learn an encoder and find a useful latent space, but this task is hard and still under active research. In order to bring a better comparison between explainers, we provide them access to the generating function g and z so that explanation are generated in the same latent space. This gives us the opportunily to let. explainers work directly in latent space by defining h(z) == f(g(z)). In other words, we define an explainer as:
(z) = e(z, h).)
(1)
where z is the ith counterfactual explanation from z found by explainer e on the latent classifier h. Working in latent spaces greatly simplifies the task of an explainer, but we will see that there are still a variety of challenges to be addressed. Namcly, the notion of optimal classifier stable classifier may be ill-defined may not always exists.
3.3 Optimal Classifier
Counterfactual explanation methods tend to produce trivial explanations by perturbing the attribute being classified from the input (Rodríguez et al.] 2021). It is likely that an explainer that changes the model's 's predictions by perturbing non-causal attributes (such as the background of an image) is more informative when it comes to exposing unwanted biases of the model. To distinguish between these two kinds of explanations, an " is required, whose predictions are contrasted with those of the model. If an explanation changes
2z Beors could be correlated to y for two different reasous: i) y ti) confounder a such that > Note that a may be element of Reausal or outside of the scene, such as the photograph. In this work, we consider deterministic generators. more general formulation would be g(øl2)
5

---

Published in Transactions on Machine Learning Research (04/2023)
SCR
0
Classifier Sample
y
Classifier
h(z) =0 0
h(z)
CR
Heausal
Sample
Z
Lud
aunsaal
eurr
EPP
NCP
h(z)
Ricassal (8)
Y
X
Oracle
7
(a)
(b)
(c)
3.1 (b) Successful counterfactual explanation as defined in (Mothilal et al. 2020 Joshi et al. 2018). That is, a successful counterfactual changes (gray) the classifier prediction (red) for the sample (point). The dashed square represents the maximum L1 norm of the perturbation performed by explainer (c) Our definition of
Figure 2: (a) Example of a causal graph satisfying the problem setup of section
successful counterfactual explanation (gray) considers any change where oracle (green) behaves differently from the classifier (red). EF: estimator flips, NCF: non-causal flips, CF: causal flips.
both the oracle and the model's predictions, the explanation is deemed trivial and discarded. However, if the explanation only changes one of the two, the explanation is non-trivial. In the absence of a human oracle who knows the causal attribute being classified, the authors resort to optimal predictor ground truth classifier. However, the concept of optimal predictor tends to be ill-defined and varies with its application, hence while assuming the existence of ground truth classifier, we must procced cautiously. To show that, we next define the concepts of Bayes classifier, causal classifier, and finally the causal classifier with non-reversible generator used in this work.
Causal classifier with reversible gencrator. The causal classificr makes prodictions solely basod on the causal parents of y in the causal graph G. In latent space: heausal (z) arg max, p(y) Zeausa))- When the generator x = g(z, <x) is reversible, we obtain fcausal (x) heausal (g - (x)). Interestingly, this classifier is robust changes of p(z) as long as p(ylZcausal) and p(z (z) remain unchangod.
Causal classifier with non-reversible generator. Tt. is worth noting that when the generator is not reversible, given can lead to many 2, which prevents from directly recovering from æ. A natural choice is to rely on the posterior distribution f(x) Σ, p(z) he)ssusa where p(z|ar) p(z)p(zz/z) However, this posterior now depends on p(z), making the new classifier no longer independent to distribution shift when p(z) is changed to c.g. (z). This leads to the following negative result (sec A.1 for the proof, along with an example):
Proposition 1. If there exists a pair z, z' s.t. g(z) = g(z') and (zausal(z) * heausal(z'), then for any deterministic classifier f(x), there is a prior pz(z s.t. the accuracy of f is 0 with respect to heausal
This shows that since the concept of optimal predictor is commonly ill-defined and application-deepedeett, we must proceed with care when assuming the existence of a ground truth classifier.
3.4 Evaluating Counterfactual Explanations
The goal for counterfactual generation methods is to find all the altributes that make a classifier behave differently from a causal classifier (see Figure 2c). Note that Mothilal et al. (2020) only considered counterfac- tuals that change the predictions top region Figure 2c These definitions do not cover cases such as when the oracle changes its prediction
classifier (Figure 2bb, and RRodriuuez et al.) (2021) only considered the
while the classifier's stay the same. Following Mothilal et. al. (2020); Rodríguez et. al. (2021) Joshi al.
6

---

Published in Transactions on Machine Learning Research (04/2023)
2018), we also measure the similarity between the original example and the counterfactuals used to explain it The reason is that counterfactuals should be relatable to original samples so that human can interpret. what is the sensitive semantic attribute. Next, we define the components of the proposed metric (Eq. 7
Proximal change (Joshi et al., 2018; Mothilal et al., 2020). An explanation must be relatable to the original sample, thus it needs be proximal. That is, the change needs to stay within a certain radius r from Using L1 norm, the set of proximal z' is defined follows:
(z) = {z' 2| z r
(2)
Estimator Flip (EF) (Joshi al., 2018: Mothilal al., 2020 This defined as proximal change on z leading to a change in prediction of the estimator h (see Figure 2b
EF(z) = f2' h(zz) + h(z) Pr.
(3)
Non-Causal Flip (NCF). Counterfactuals obtained by estimator flips (EF) are common in the literature as they do not require the knowledge of heausal- However, if we have access to heausal: we can detect a new set of explanations: a proximal change z' that changes the prediction of h but not of heausal:
NCF = (z' EF(z)) beansat = henusal (2)} Pr.
(4)
Causal Flip (CF). Additionally, access to heausal allows us to detect another new set of explanations: a proximal change in z' that changes the prediction of heausal but not
CF(z) =F(2)=(2' h(zz)= = i(zz) A (eausal(z) * heausal()) n.p.
(5)
Thus, we define the set of succossful counterfactual explanation (SCE) as follows:
SCE(z) = (NCFUCF).
(6)
In summary, having knowledge of the causal factors (access to heaussl) allows us to evaluate counterfactuals explanations in a new way as illustrated in the following example. Given a dog classificr and an image of a dog, counterfactual example that changes the background of the image in way that alters the classifier's prediction (NCF) will almost certainly provide valuable insight about the model's behaviour. The same can be said about counterfactual example that removes the dog from the image without altering the classifier's prediction (CF) (see Figure 2cb. Note that these counterfactuals cannot be detected without causal knowledge, which is only available if we have access to the entire data generating process i.e., synthetic dataset.
Orthogonal and complement subset. Note that both EF and SCE are possibly infinite sets and cannot be easily interpreted by humans. We could return the explanation minimizing some notion of distance on z or x, however a good explainer should return a useful and diverse set of explanations.
To this end, we propose metric that only takes into account the subset of orthogonal and complementary explanations. Otherwise, it is trivial to report many erplanations that are modification of an existing explanation without being useful. For instance, modifying the hair color to trigger change in gender classification is good finding, but changing the hair color again, and removing some clouds in the sky would not constitule a useful explanation. Hence, only admitting orthogonal explanations enforces a useful diversity. However, we also admit complementary explanations. That is, darker hair triggers a change in gender classification and lighter hair also triggers a change, these are two useful explanations. In short, given two explainers that find counterfactuals by perturbing the most sensitive attribute, the orthogonality and complementary requirements ensure that the one that provides more diverse set of counterfactuals by also

---

// add perturbation to the set Published in Transactions on Machine Learning Research (04/2023)
Algorithm 1 Orthogonal Set
Input: original sample E IR, successful counterfactuals Esc E [Raxd threshold 7
Output: an orthogonal set of counterfactuals
Aac < Eec - ;;
// calculate perturbation vector // sort perturbations by increasing norm 1/ initialize set of orthogonal perturbations
indices - argsort(| (cc
North + Ae bellindice(00]
for i 1 to do - Arclindices(i]]
// select the next perturbation // calculate similarity of with all elements in the set if (Vj E abs(sim;) <) or (7j E simy + < 7) then
sim < cos(p, Aorth)
Aorth + [orthipp
end
end
return 2 + Aorth
// return set orthogonal counterfactuals
perturbing less sensitive attribules scores higher. This is important because it rewards explainers that give a more complete description of the model to the user. There may be use cases where only the most sensitive attribute matters. However, everything else being equal, we argue that, in general, it is favorable to have access to a diversity of explanations.
The explainer is responsible for returning explanations produced with orthogonal or complementary perturba- tion vectors. To verify whether explanations are orthogonal or complementary Concretely, sort the explanations by how proximal they are to the original sample and add the first one to the set. Then we iterate through the rest and sequentially add every subsequent explanation that is orthogonal or complementary to all the explanations currently in the set (see Algorithm [ for implementation). The resulting orthogonal and complement set referred to as SC))(z). We use the cardinality of this set to evaluate the performance of explainers:
greedy algorithm""
= ISCE (2)l-
We consider the proposed sctup to be fairer than previous works, since: (1) all explainers are compared the same latent space, resulting in fair evaluation, (2) uninformative explanations are discarded leveraging knowledge of the causal factors, (3) designed be more difficult to game by repeating counterfactual explanations, and (4) it rewards explainers that return more complete set of explanations.
4 Experiments
In this section we give overview of the different methods (sce Tablc and datascts that are comprised within our benchmark. Since we provide access to a common interpretable latent space, we evaluate explainers that do not depend on a concrete latent decomposition. The code is written in PyTorch (Paszke et al.) 2017) and is made public along with the datasets and pretrained weights for the models used in this work. Implementations details can be found in the Appendix (A.2).
Latent-CF CF (Balasubramanian et al. y020) A simple method that performs a adversarial perturbations in the latent space until counterfactual with confidence higher than threshold tol is found.
DiCE (Mothilal et al.) 2020): A method that aims to produce a diverse set of counterfactual examples directly from a series of attributes or latent space by proposing a series of perturbations that change the predictions of a classifier. This is achieved by gradient- based optimization of multiple loss functions with respect the attributes or latents and the classifier:
L hinge loss (asss) y, margin) X dist(zzzzz) + Addpp _
diversity (z')) (z'),
(8)
(A)
(B)
(C)
4The complexity the resulting algorithm given number explanations is (n)))
8

---

Published in Transactions on Machine Learning Research (04/2023)
7
1
16 clasters and 9.5 correlatize
10 clusters and correlationn
03
h
h
h
6
6
h
h 9
h
RUW Houd
Traegation
Feet
Lategrid De
Preglationy
6
6
g
g
9
6
6
6
9
g
00
Craylationi DOUR
4
5
5
D
D
C
J
f
f
f
V
V 2
r
7
r
2
2
2
2
F
SCATA
Figure 3: Interpolations produced by our learned gen- erator (g(z)). Images are changed as the attribute's s value changes smoothly from one value to another (left-right). From top to bottom: rotation, scale, h- translation, v-translation, char, font.
Figure 4: Average attribute perturbation for each method/seee sccnario... Gradient based methods perturb almost all attributes while gradient-agnoste methods perturb only one or two. DiVE focuses almost solcly on font.
where optimizing (A) pushes the prediction of the classifier f towards y up some margin, (B) ensures that counterfactuals (z') are close to the original samples (z), and (C) maximizes the distance between each pair of counterfactuals.
2018 A method equivalent to DiCE without the diversity term (C). xGEM (Joshi et al.,
DiVE RRdrrguez el al. 20 : A method similar to DiCE that leverages the Fisher Information (FI) to find non-trivial counterfactuals, t.e. samples that change the classifier prediction without changing the causal attribute ccausal; thus focusing on spurious correlations. This is done by masking out latent dimensions with the highest FI while optimizing a cost equivalent to Eq.
Growing Spheres (GS) (Taugel et. al., 2017): A method that given data point identifies its closest. neighbour classified differently referred to enemy. This is done by finding the smallest (2-ball around z that contains an enemy. Once is found the dimensions with small changes in e with respect to are discarded through a feature selection process, maximizing the sparsity of e z.
StylEx (Lang et al. 2021 They find a latent perturbation in a direction that maximizes the difference in the output of the classifier for the original sample and its perturbed counterpart.
Informed Search (IS): An explainer that knows about the data generation process in Figure 2a Thus, IS generates explanations by perturbing the spuriously correlated attributes Zeorr.
4.1 Datasets
We design synthetic benchmark based the synbols dataset (Lacoste al. 2020). In this benchmarks images are fully defined by 3 categorical attributes (48 fonts, 48 characters, 2 background colors) and 4 continuous attributes (x-translation, y-translation, rotation, scale), see Figure 3
An advantage of synbols the large amount of values in its categorical attributes such as character and font. This allows us to design different scenarios by introducing spurious correlations based on subsets of these attributes. From now on, we assume Zeausal = char E [1..4]] and set heausal = Zeausal mod 2, creating a binary classification problem. Then we leverage the font altribute to introduce spurious correlations (Zeorr in Figure 2ap. Note that increasing the number of fonts in Zeoer (the rest will be in Zina increases the random chance of finding a counterfactual by accidentally switching the font. Likewise, increasing the amount of correlation between &çorr and makes spurious correlations easier to find since the classifier latches stronger on them. We hepothesize that stronger correlations will benefit gradient-based explainers, which will find higher gradient curvature for highly correlated fonts. To explore how explainers behave under different
Since already provide an interpretable set latent attributes we evaluate only the Attribute finding (AttFind) algorithm
from the paper
9

---

Published in Transactions on Machine Learning Research (04/2023)
Table 2: Score (Eq. 7 and percentage of trivial counterfactuals () obtained by each explainer for each of the Values represent an average score across batches for the entire dataset for
different datasets described in 4.1 different runs.
Correlation
0.50
0.95
0.50
0.95
# Spurious
Explainer
S#
Trivial (%6
Trivial (%)
IS (Oracle)
2,40 =0.30
2.07
#0.20
0.00
=0.00
0.00
#0.00
DICE Mothiaal et al.
2020
1.18 =0.01
1.17 #0.01
9.37
#0.11
6.78
#0.26
DIVE Rodriguez el al.
2021
1.02 =0.00
1.00
+0.01
2.51
#0.09
1.68 +0.02
6
GS [Tugel el al. 201
1.01 20.00
1.01 20.00
4.49
1:040
2.34 10.15
StylEx Lang el al
2021
1.04 20.00
1.17 20.00
2.41 1:000
1.58 10.00
Latent- CF Balasn bratnattian ot al.
2020
0.82 20000
0.93
10000
0.00
1:0000
0.00 10.00
xGEM (Josh et al.
2018
1.18 #0.02
1.15 #0.01
12.46 +0.08
6.45 +0.07
IS (Oracle)
2.80
3.63 #0:20
0.00
0.00 #:000
DICE Mothilal et al.
2020
1.13
1.19 20:01
8.62
6.70 1::08
DIVE Rodriguez
2021
1.00
1.04
2.28
1.44 10:04
10
GS Laugel ct 2017
1.01
1.00 10.01
4.91 10.25
1.95 10:05
StylEx Lang braanian
2021
1,15
1.12
3.37 10.00
1.62 1:000
Latent-CP CF [aalasurrame
2020
#0.00
0.81 #0.00
0.00 =0.00
0.00 #:000
xGEM Joshi al. 2018
1.15 20.00
20.00
10.17 20
6.38 10.11
scenarios, we consider 6 and 10 spurious fonts with 50% and 95% correlation with resulting in a total of 4 scenarios. Further, we introduce 5% of noise the the Zeausal attribute (character) to encourage classifiers to also rely on the font.
4.2 Results
We evaluate the performance of six different methods with the metric defined in Eq. 7 Each method is evaluated in several different. datasets with varying levels of difficulty as described in 3.1 and 4.1
It is hard to diversify. good explainer should be able to predict the behavior of the model with respect to changes in the different attributes that generate the data. In the case of a classifier, finding the attributes that induce it to change its prediction in order to reveal if it relying on attributes that are independent from the class being predicted is desirable goal. In the pursuit of this goal, an explainer should idcally populate SCE(z) (see Eq. 6) with explanations altering each of the attributes that are correlated with the data. However, as shown in Table [ exxplainers fail to consistently find more than one altering attribute.
Performance saturates with 6 fonts. We observe that mothods do not significantly increase the mumber of successful counterfactuals when adding more spurious fonts (Table 2p. This is, partially, because methods tend focus on changing the Zeausal attribute character as seen in Figure J which leads to trivial counterfactuals. When adding more fonts, the font identification task becomes more difficult for the classifier, which makes it more sensitive to characters and exacerbates this problem. For more extensive ablation illustrating this phenomenon see Figure
Gradients tend to perturb most of the attributes. Figure ! offers insight into how each method perturbs z and we can see that gradient- based methods tend to perturb almost all attributes equally, exploring the perturbation space in many directions. In the extreme, we found that Latent-CF slightly modifies all the latent attributes, producing countcrfactuals that resemble adversarial attacks. While modifying all the altributes increases the chances of finding 1 good explanation on average, it also prevents the explainer from finding multiple non-trin trivial diverse explanations. On the other hand, methods that are aranient-agnostic focus on perturbing one or two attributes, resulting in more narrow search space. This increases the risk of methods focusing on Zeausal (Figure 2a). This is evidenced in Figure g where StylEx and GS considerably perturb the character attribute. Interestingly, the perturbation pattern of DiVE shares some similarities with StylEx and GS due to gradient masking.
DIVE focuses on changing the font. As shown in Figure 4 DiVE perturbs almost exclusively the Zeorr attribute (font), specially for high correlation values, this indicates that the method successfully distinguishes
10

---

Published in Transactions on Machine Learning Research (04/2023)
Table 3: From left to right: We report the percentage of estimator flips EF (Eq. 3) (Joshi et al., 2018
Mothilal ct al., 2020), percentage of successful counterfactuals SCE (Eq. 6 and what percentage of those are
Non- Von-Causal Flips (Eq. 4 and Causal Flips (Eq. 5). Values represent an average score across batches for the
entire dataset for 3 different runs.
Correlation
0.50
0.95
0.95
0.50
0.95
#Spuzious
Explainer
EF (%6)
SCE (8)
Cauzal Flip Rase (%)
Non-Causa Flip Hale (%0
(Orucle)
76.97
67.5
62.25
20.
20.0
DIOR Mothlal of a.
2020
32:90 10:00
31.96 1:0:1
55.42 22:40
56.67 2.
26.46 $1:50
23.22 11:00
73.54 11
70.78 1::30
DIVE [orriaaes
25.02
36.58
07.5
60.83
6.56 2b:s
3.44
90.45
96.56
GS Laugel d al.
2017
36.14 #1.22
34.94 $0.40
31.67 #7.10
31.67
#15.03
0,00 =0.0
0.00 20.m
100 20.02
100 20.40
Skylex
Lain
2121
23.66
24.51
25. 07
25,80
17.02
21:10
82,98
78.60 60 t0.00
Latent-
CF Baasspraaattiaa ct al. 2020
20:98 +0.00
24.18 20.00
20.96 =s0.00
24.18 =0.00
0.00 s0.00
0.00 st.0
100 20.00
100 20.0p
xGEM
Joen
70.61
75,98
76.67
78.83
4.18 s0.00
2.96
95.82
97.04 +0.00
(Oracle)
35.33 +0..0
71:19
200088
54.33 10
#2.45
51.25 =1:20
0.00
0.00 20.00
100 20.00
100 20.00
DIC Mothilal of a..
2020
31.77 1:0.45
333.25
1:0.93
45.00
1::33
30.17 20:73
26.16 as:aa
27.65
20:00
73.84
72.35 10.70
10
DIVE
GS La
Bocriggs of
Tigel di al. 201
37.38 +0.35
1.0:31
36.38
1.0.25
5:0.4
44.58
60:00
=:2..2
A1:25
54.58
40.42
=5.92
0,00 =0.00
6.83 1.577
0.00 =0.0
10.0
90.17 10.57
100 20.00
97.75 10.00
100 20.00
Stylex
Lang
1.0.50
1.0.00
23.65
10.00
23.77
10.00
10.00
0..00
10.00
1..00
Latent
Debuunetraannnn Facmnian et al. 2020
22:00 +0.00
23.33
=:0.00
21.98
=0.00
20.24
=0.02
0,00 =0.00
0.00 =au0
100 20.00
100 20.00
xGEM
+0.58
$3.20
62.92
#0,8
#0:22
+0.27
explainer
explainer
3.5
3.0
Q5
3.0
stylex
stylox
xgom
2.5
Candiniity
Cardisiiyy
St
2.0
1.5
1.0
10
0.25
0.50
0.70
0.35
* Spurious
Correlation
Figure 5: Sensitivity of every explainer to varying amount of correlation levels (left) and number of spuriously correlated attributes (right) (sce Section 4.1 measured with our score (Eq. 7 Note how the performance of the explainers, excluding the oracle (IS), does not scale and not sensitive to the level correlation and the amount of correlated variables. This supports our findings that explainers are unable to provide a diverse set of explanations and focus changing the causal attribute Zeausal-
between Zeausu) and Zcorr attributes. However, it is not able to consistently perturb the font in the right way to produce a diverse set of counterfactuals as evidenced by its score (Table 2
Non-triviality is not enough. Table (right)) shows the average percentage of trivial counterfactuals found by each method. We obscrve that methods that tend to produce a higher number of successful explanations (left) tend to also produce a larger number of trivial counterfactuals (right), which are discarded in our metric.
Quality over quantity As seen in Table 3 some explainers obtain high percentage of successful counterfactuals SCE, sometimes even higher than the oracle (xGEM, DiVE). However, this is not reflected in their score Sy (Table 2), which is considerably lower than the oracle's. This is because even though the explainers can find a high number of counterfactuals they are discarded by our metric since they are not orthogonal or complementary and thus redundant. Further, note that the score measured using estimator flips EF (Eq. 3) (Joshi et al. 2018 Mothilal al. (2020) is not correlated with our score Sy (Table 2). For example, xGEM obtains higher score than the oracle (1S) despite the latter returning a more complete set.
11

---

Published in Transactions on Machine Learning Research (04/2023)
of explanations. This shows how previously proposed metrics (Joshi et al 2018 Mothilal et al. 2020 can be gamed by explainers by generating many redundant explanations that fail to fully the describe the model's behaviour. Figure [] also supports this finding, showing how the performance of the oracle (IS) is the only one affected by the amount of spuriously correlated attributes and their level of correlation (sce Section 4.1
Explainers exploit bad classifiers. As seen in Table 2 and in Figure 3 explainers are not significantly affected by the amount of spurious correlation zoor introduced. This indicates that, in contrast with the oracle (IS), methods produce explanations by changing the font attribute Zcausal (as seen in Figure 4 without changing the classifier's prediction, thus creating a successful counterfactual (Eq. 5 These counterfactuals expose failure cases of the classifier and are therefore useful, since they show that the classifier is unable to classify some characters. Table 3 shows that DiCE (Mothilal et al. 2020) and StylEx (Lang et al.) 2021) produce a high amount of these counterfactuals, while GS (Laugel et al. 2017) and Latent-CF (Balasubramanian ct al. 2020) always change the classifiers prediction and thus produce none. The oracle (IS) is not designed to perturb Zeausal in any way so it cannot produce any causal counterfactuals.
In Figure 0 we show two ways in which explainers obtain causal counterfactuals, Note that besides confusing the classifier by modifying the character's diacritic, explainers can create new characters entirely by merging two letters together or even adding an accent mark to consonant. This behaviour is unavoidable in the absence of an optimal classifier.
Additional insights As seen in Table 2 explainers are unable to generate a diverse set of counterfactual explanations. However, Table 3 highlights some differences between methods when it comes to other metrics. If the objective is to maximize the number of estimator flips EF (Eq. 3) or the number of successful counterfactuals SCE (Eq. 6) we recommend using xGEM. If the objective is to maximize the number of causal flips (Eq. 5 we recommend using StylEx or DiCE. That said, we have shown that explainers generate high amount of redundant counterfactuals, and thus we recommend caution when choosing them based on how they maximize these individual metrics.
5 Limitations
As discussed in Section 3.3 the core limitation of explaining image classifiers via latent perturbations is the lack of accurate reversible generators. If the generator is not reversible, given æ can lead many z, which prevents the direct recovery of z from It might be a good idea to bypass the pixel space completely and work directly z, this however, would produce explanations outside the image domain and therefore, uninterpretable by humans, which is ultimately not very useful. It could be argued that the generator used in this work could be modified to yield better image reconstructions given any z, however this will always be hindered by the aforementioned limitation. More generally, most methods rely on some some sort of latent. decomposition in order to search for counterfactuals a latent space. However, it is still not clear how the true latent variables of the data generating process are not identifiable (Locatello et al. 2019). ; In this work we circumvent this problem by using synthetic datasct. On the other hand, Khemakhen et al. (2020) showed that, with further assumptions, it is possible to identify the latent variables (Khemakhem et al. 2020). Moreover, in a temporal setup, it is possible to identify which of these latent variables are the causal ones (Lachapelle et al.) (022). Finally, in a multi-task setup where distribution shift occurs, it is possible to identify which variables are robust to distributions shift and hence, likely to be the causal ones. In summary, although it would be possible to approximate the true latent factors in some cases, it would require making additional assumptions about the data.
We make effort to establish fair, principled metric that is useful. However, this metric does not depict all the properties of an explainer such fragility speed. It is possible, albeit unlikely, that our definition of seeful/informatiee explanation might not always align with that of the usr example, it is possible in some cases for trivial explanations to be informative, however trivial explanations are easily obtained by explainers, while non-trivial ones are elusive. Thus, we focus on the latter. It is important to note that our definition of SCE (Eq. 6 fits the context of image classifiers best. To evaluate explainers other domains (i.e., algorithmic recourse (Karimi et al.] (021)) more flexible definition should be adopted. Lastly, the generator we use in this work can generate images with certain implicit biases.
12

---

Published in Transactions on Machine Learning Research (04/2023)
DICE
DIVE
StylEx
XGEM
i
i
a
á
T
T
Causal Flip counterfactuals obtained by changing/removiig the diacritic in the character.
2
2
O
2
E
U
Ó
O
6
a
1
1
á
à
d
O
J
u
Ö
6
6
J
j
D
Causal Flip counterfactuals obtained by obscuring the character or creating a new one entirely.
i
6
6
2
L
W
1
i
P
2
S
b
D
O
L
6
g
Figure 6: Some of the Causal Flip counterfactuals (Eq. different subcategories.
6
obtained by each method separated into two
6 Discussion
Benchmark In this work, we have introduced more comprehensive definition of good counterfactual (Section 3) that instantiate metric (Section 3.4) well fair evaluation setup (Section 3.1 in the form of benchmark. Previous evaluation sctups use datasets like CelebA where the causal data generation process is unknown and use metrics that are easy to game. In contrast, our evaluation setup uses a more comprehensive and fair metric while providing control over the entire data generating process and therefore knowlodge of the causal factors, providing a tool to evaluate propertics of explainors that are impossible to evaluate in a non-synthetic setup. Even though knowledge of causal factors rare when working in real world scenarios,
possible to adapt our metric to take only into account: an orthogonal and complement. set of estimator flips EF (Eq. 3 which do not require causal knowledge. However, any evaluation schema that does not include causal information would be incomplete. Further, if an explainer fails to provide set of useful and diverse explanations for our simple synthetic dataset il is very unlikely that it is able to do for real datasets. Nevertheless, we recommend users to also evaluate explainers using real world data.
Oracle We argue that successful counterfactuals should be considered in the perspective of human. In the absence of a human, we must resort to an optimal classifier, whose task is to contrast the prodictions of the model with the optimal prediction and spot unexpected behaviors; acting as an oracle. Without an oracle, it is not clear how we could assess whether a model is working as intended. We show that the optimal classifier is commonly ill-defined in the image domain, bocause not always possible to access an invertible image generator (Section 3.3). Therefore, it cannot be expected that classifier trained on pixel space achieves optimal performance.
13

---

Published in Transactions on Machine Learning Research (04/2023)
Results Our experimental results could indicate that the different counterfactual explainers in the literature perform similarly and there has been little improvement in the recent years (Table 2 Although most of them find single explanation in average, we found that they do it in different ways (Figure
7
Conclusion
In this paper present benchmark that provides unified metrics for evaluating different counterfactual explanation methods. The benchmark consists of synthetic images fully described by their annotated attributes which are accessible to the explainers through differentiable generator. We hope our findings encourage further research on fair evaluation benchmarks.
Acknowledgements
This work was supported by the Generalitat de Catalunya under the Industrial Doctorate Program (grant number 2020DI62), and by the Spanish Ministry of Economy and Competitivennss (MINECO) and the European Regional Development Fund (ERDF) under Grant PID2020-120311RB--100 funded by
References
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint 190:::99900003 2019.
Scbastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140, 2015.
Rachana Balasubramanian, Samuel Sharpe, Brian Barr, Jason Wittenbach, and C Bayan Bruss. Latent-ef: simple bascline for reverse counterfactual explanations. arXiv preprint orXiv:2012.09930 2020.
Andrew Brock, Jeff Donahuc, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. aixiv preprint arXiv:1809. 11096, 2018.
Vanessa Buhrmester, David Münch, and Michael Arens. Analysis of explainers of black box deep neural networks for computer vision: A survey. Machine Learning and Knowledge Extraction, 3(4):966 989, 2021.
Nadia Burkart and Marco F Huber. A survey on the explainability of supervised machine learning. Journal of Artificial Intelligence Research, 70:245-317, 2021.
Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image classifiers by counterfactual generation. arXie preprinl wxXiw:107.08804, 2018.
Emily Denton, Ben Hutchinson, Margaret Mitchell, and Timnit Gebru. Detecting bias with generative counterfactual face attribute augmentation. 2019.
Andrew Elliott, Stephen Law, and Chris Russell. Explaining classifiers using adversarial perturbations on the perceptual ball. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 10693 10702, 2021.
Ruth Fong, Mandela Patrick, and Andrea Vedaldi. Understanding deep networks via extremal perturbations and smooth masks. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2950 2958, 2019.
Ian Goodfellow, Jean Pouget-Abadic, Mchdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advonces in neural information processing systems, 27, 2014a.
14

---

Published in Transactions on Machine Learning Research (04/2023)
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arx: 14 12. 6572, 2014b.
Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. Counterfactual visual explanations. In International Conference on Machine Learning, pp.
PMLR, 2019.
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Dino Pedreschi, Franco Turini, and Fosca Giannotti. Local rule-basod explanations of black box decision systems. arXiv preprint arXiv: 10820, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Tn Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Simon Jégou, Michal Drozdzal, David Vazquez, Adriana Romero, and Yoshua Bengio. The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In Proceedings of the IEEE conference on compuler vision and pollern recognilion workshops, pp. 11 11-19, 2017.
Shalmali Joshi, Oluwasanmi Koyejo, Been Kim, and Joydeep Ghosh. xgems: Generating examplars to explain black-box models. arXie preprinl 180v:
2018.
Amir-Hossein Karimi, Beruhard Schölkopf, and Isabel Valera. Algorithmic recourse: from counterfactual explanations to interventions. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp. 353 362, 2021.
Tyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics, pp. 2207-2217. PMLR, 2020.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXi::31126614, 2013.
Sébastien Lachapelle, Pau Rodriguez, Yash Sharma, Katie E Everett, Rémi Le Priol, Alexandre Lacoste, and Simon Lacoste-Julien. Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ica. In Conference on Causal Learning and Reasoning, pp. 428-484. PMLR, 2022.
Alexandre Lacoste, Pau Rodríguez, Frédéric Branchaud- Maanchaud-Charron, Parmida. Atighehchian, Massimo Caccia, Issam Laradji, Alexandre Drouin, Matt Craddock, Laurent Charlin, and David Vázquez. Synbols: Probing learning algorithuss with synthetic datasets. arXiv preprint ar XrXi:22099000 15, 2020.
Oran Lang, Yossi Gandelsman, Michal Yarom, Yoav Wald, Gal Elidan, Avinatan Hassidim, William T Freeman, Phillip Isola, Amir Globerson, Michal Irani, el al. Explaining in style: Training a gan to explain a classifier in stylespace. In Proceedings of the LEEE/CVF International Conference on Computer Vision, pp. 693 702, 2021.
Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. Inverse classification for comparison-based interpretability in machine learning. arxiv preprint arXiv: orXi:: 2017.
11111712.888 08443,
Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, (4):541-551, 1989.
Shusen Liu, Bhavya Kailkhura, Donald Loveland, and Yong Han. Generative counterfactual introspection for explainable deep learning. In 2019 IEEE Global Conference on Signal and Information Processing (GlobalSTP), pp. 1 5. IEEE, 2019.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Compuler Vision (ICCV), December 201 2015a.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 201
15

---

Published in Transactions on Machine Learning Research (04/2023)
Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Ractsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference on machine learning, pp. 4114 4124. PMLR, 2019.
Nya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint
arXiv: 03888888,, 2016.
Lya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arxiv: 01100101, 2017.
Ramaravind K Mothilal, Amit Sharma, and Chenhao Tan. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference Fairness, Accountability, and Transperency, pp. 607-611 2020.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Martin Pawelczyk, Klaus Broelemann, and Gjergii Kasneci. Learning model-agnostic counterfactual explana- tions for tabular data. In Proceedings of The Web Conference 2020, pp. 3126-3132 2020.
Martin Pawelczyk, Sascha Bielawski, Johannes van den Heuvel, Tobias Richter, and Gjergji Kasneci. Carla: A python library to benchmark algorithmic recourse and counterfactual explanation algorithms, 2021.
Judea Pearl. Causality. Cambridge university press, 2009.
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 32, 2018.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 'why should i trust you?" explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135 1144, 2016.
Pau Rodríguez, Massimo Caccia, Alexandre Lacoste, Lee Zamparo, Issam Laradji, Laurent Charlin, and David Vazquez. Beyond trivial counterfactual explanations with diverse valuable explanations. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1056-1065, October 2021.
Axel Sauer and Andreas Geiger. Counterfactual generative networks. orXie preprint arXiv:1001.06446 2021.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International conference on machine learning, pp. 3145-3153. PMLR, 2017.
Sumedha Singla, Brian Pollack, Junxiang Chen, and Kayhan Batmanghelich. Explanation by progressive exaggeration. anXiv preprint arXiv:1911.00483, 2019.
Arnaud Van Looveren, Janis Klaise, Giovanni Vacanti, and Oliver Cobb. Conditional generative models for counterfactual explanations. arXiv preprint arXiv:2101110128, 2021.
Fan Yang, Ninghao Liu, Mengnan Du, and Xia. Hu. Generative counterfactuals for neural networks via allribull-informed perturbation. ACM SIGKDD Explorations Newsletter, 23(1):59 68, 2021.
Yilun Zhou, Serena Booth, Marco Tulio Ribeiro, and Julie Shah. Do feature attribution methods correctly attribute features? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 9623-98 028-9633, 2022.
16

---

Published in Transactions on Machine Learning Research (04/2023)
A Appendix
A.1 Proof of Proposition 1
Proposition 1. If there crists a pair 2, z' s.t. g(z) = 9(z') and (za & heausallz'), then for any deterministic classifier f(x), there is prior (z) s.t. the accuracy of f is 0 with respect to heausal.
Proof. Let S = {=(2|(((()) f(o()))
+ Heausai (2)) and show that |S| + O. Since g(z) g(z'), we have that f(((z)
f(g(z))) Also, since heausal (z) * (z))) we have either f()(z) + heansal or f(g(z') # heausal (z'). Finally, any prior p'(z) with no mass outside of S satisfies the proof.
As an example, let's consider a cube classifier from 2d projections of cubes. We can train a classifier to predict the correct class from the 3 visible faces. However, since the back of the cube not visible there exists datasets such that the back the cube distorted in a way that makes not a cube. The point being made here is that even though heausal exists, feausal does not always exists. Further assumptions needs to be made which priors p(z) valid for robustness to distribution shift, e-g,, humans, we make symmetry assumption for classifying cubes. Perhaps future line of rescarch in explainability should seek to find what assumptions are made by classifiers. In the rest of this work, will make the assumption that if g(z) = g(z',, then (z) = (z') for all pairs z, z'
The proof simple but it contrasts with the known result that heausal is robust to any distribution shift over p(z).
A.2 Implementation Details
In an effort to ensure reproducibility we provide implementation details in this section. The total run-time for all experiments is -37 hours on a single Titan-X GPU. Detailed documentation will be released upon publication. Below we describe the training setting for the encoder q(2|<r the generator g(z) and the classifier to be explained f(z), along with the settings for each of the explainers considered in this work.
A.2.1 Training details
Encoder The encoder is based on BigGAN's (Brock et al. 2018 Rodriguez et al. 2021 discriminator architecture with classifier on top and it is trained on Synbols (Lacoste et al. 2020) dataset. Given an image I we task the encoder with predicting the attributes that describe it. It is trained for 100 epochs with
a batch size of 64. We use AdamW [Loshchilov & Hutter 2017) with a learning rate of 0.001 and a weight LonoduulerLLshhhiooooo contains discrete and continuous attributes, we optimize them separately minimizing:
decay of 0.0001 with a cosine annealing learning rate scheduler Loshchilov & Hutter 2016 Since Synbols
Ldiscrete + Loontimuous
(9)
where Laiscrete is the average cross entropy loss for each categorical attribute and Crontimouus is the L distance between the original continuous attributes and the ones predicted.
Generator Similar to the encoder, the generator is also based BigGAN (Brock et al. 2018 architecture and trained with the same srper-parameters and learning rate scheduler. Given set of attributes and the image x that they describe, we train the generator to reconstruct x from z. However, in order to provide a high-dimensional input to the generator instead of directly using the Synbols attributes leveraged in our benchmark E IR? (Section 4.1),, we use embedding layers to project each categorical attribute (character and font) into a 3-dimensional space. The embeddings are concatenated with the 5 continuous Synbols attributes obtaining z R We train the generator minimizing the following criteria:
6This assumption could be relaxed by saying that the probability of this assertion being violated unlikely under prodefined of salid distribution shifts. We treat the background color continuou attribute
17

---

Published in Transactions on Machine Learning Research (04/2023)
Table 4: Notation Table.
Notation
Description
An input image
The label of the imput image
A set of latent variables
Zeausal
A subset of z containing the causal parents
z
counterfactual ual explanation for z
((a)
The classifier to be explained
h(())
The latent classifier to be explained
heausel (z)
The latent classifier to that is robust under change of p(z)
p(z)
Prior distribution over z Deterministic image generator
g(z)
g(zlz)
Stochastic image generator
(((())
An encoder
e(2, h,
function that generates explanations
=ax x' +1 (1 ( /|I
(10)
where a = 0.2, x and x' are the original and reconstructed image respectively and g(x) are the learned encoder features for image z. Lastly, to reduce the amount of noise in the generated images we use a discriminator network D trained alongside the generator to distinguish between x and x'. Specifically the generator is trained every 3rd iteration. So, the criteria for the generator optimize becomes:
Lrec + log(1 - D(x')) *
(11)
where X = 0.01 and D(x') is the discriminator's estimate of the probability that the reconstructed image x' is real.
Classifier The classifiers we set to explain are ResNet-18 HHe et al. 2016) architectures trained on the different beuchmaakk described in Section 4.1 All the classifiers are trained for 10 epochs with a batch size of 256. We use AdamW with a learning rate of 0.01 and a weight decay of 0.0001 with a cosine annealing learning rate scheduler.
A.2.2 Explainer evaluation
In this section we will detail the hyper-parameters chosen for each of the methods analyzed in this work well some details regarding the benchmark. Given the varied nature of the explainers and their hyper-parameters will refer to the works where the explainers are introduced when describing the effect the hyper-parameters have on each method. All of the typer-parameters where chosen through random search.
Common setup Across our experiments there are a few settings that are common for all methods. To save time, instead of producing counterfactuals for the entire validation dataset we select a balanced subset with a total of 800 correctly and incorrectly classified samples with different levels of confidence. We do this by selecting the 100 samples closest to the required level of confidence f(x) (+0.1, +0.4, 10.6, +0.9).
The explainers required to produce 10 counterfactuals per sample, if method is originally conceived to produce only one, we follow (Mothilal et al. 2020) to create 10 samples close to the original sample in z space and task the method with producing an explanation for each of them. The attributes in z are standardized using the mean and standard deviation of the attributes in the training set. To prevent explainers from gener- ating counterfactuals that the generator g(z) cannot interpret we clip every coordinate in z the maximum and minimum values for each attribute found in the training set. We set the batch size 12 across experiments.
18

---

Published in Transactions on Machine Learning Research (04/2023)
Since the categorical attributes in space (font and character) are produced by embedding layers and projected into a 3-dimensional space each (Section A.2.1 ; they must be treated differently from the continuous attributes. When measuring if an explanation z' is proximal to the original sample z we establish three different radius r values (see Eq. 2 one for the font, one for the character and one for the continuous attributes. For the character and lont. altributes we set lo be the maximum pairwise distance between the embedding representations of each attribute and for continuous attributes we set 1. Making this distinction allows the explainer to change from any given font/chares /character to another while preventing it from modifying every attribute once. When verifying if counterfactuals fulfill our orthogonality condition we look at perturbations between continuous attributes (Xom))) However, for embedded attributes (font and character) we transform their embedded representation in z space into:
Leat = Softmax([ c' d'-w,, 1222 -1), Vi E [1..48]
(12)
where c' is the representation of the categorical attribute character/ /ont for a given counterfactual z', w are the weights of the embedding layer used to map the 48 characters /fonts to their 3 dimensional representation c' and is a temperature factor set to 0.33. We choose the softmax function so that any small change in that maps to a different categorical attribute can be orthogonal. In order to turn z into a perturbation vector we cancel the perturbation on the original attributes coordinate by setting:
dest 2'stt x (1-14 1c)
(13)
where 1 is vector of 1s and 1. is a one-hot representation of the categorical attributes for the original sample. Thus given sample verify that two counterfactual explanations z' and z" are orthogonal by measuring cosine similarity: cos(z'ms> z"at) for categorical attributes and cos(z.ont - -2 z z) for continuous attribules.
Growing Spheres (GS) Laugeel et al., 2017) For this method we found the best configuration was setting the initial radius n 10 and the number of candidates n to 50. For a detailed description of the role of these
see (Laugel al. 2017).
StylEx (Lang et al., 2021 For this method we found the best configuration was sctting threshold to 0.3 using the "Independent" selection strategy and the amount of shift applied to each coordinate to 0.8. This last parameter is not mentioned in the Stylex (Lang et al. 2021) paper, but it can found as parameter undor the name shift__szz size in the implementation they provide. For a detailed description of the rolc of these hyper-parameters see (Lang et al. 2021).
DICE (Mothilal et al. 2020) For this method the best results were obtained by setting to learning rate 0.1, the reconstruction weight of the loss function A to 1 and the diversity weight of the loss function Xg to 1. For detailed description of the role of these byper-parameters see (Mothilal et al. 2020). We set the maximum number of iterations to 50 in case the algorithm does not converge.
DiVE (Rodríguez et al., 2021) For this method we found the best configuration was setting the learning rate to 0.1, the weight of the praximity loss term to 0.0001, the factor that controls the sparsity of the latent space y to 0.1 and the weight of the diversity loss to 0.001. For a detailed description of the role of these hyper-parameters see (Rodriguez al. 2021). We set the maximum number of iterations to 50.
xGEM (Joshi et al.) 2018 This method shares some parameters with DIVE. We found the best configuration for this explainer was setting the learning rate to 0.1 and the weight of the proximity loss term to 0.001. We set the maximum number of iterations to 50.
Latent-CF (Balasubramanian et al., 2020) For this method we set the learning rate to 0.1, the probability of target counterfactual class p to 0.1 and the tolerance tol to 0.5. For a detailed description of the role of these prper-parameters see (Balasubramanian et al. 2020). We set the maximum number of iterations 50 in case the algorithm does not converge.
19

---

Published in Transactions on Machine Learning Research (04/2023)
A.3 Ethical Concerns
Research in explainable Al is crucial for safe deployment, of machine learning solutions in real life. In this work, we have shown that the lack of a principled evaluation of such systems has slowed-down advancements in the field. These findings should not discourage future research in explainable AI, on the contrary, we hope that our work unblocks the current state of the art and spurs further progress.
On the other hand, the research described in this paper does not (i) directly facilitate injury living beings, (ii) raise safety, privacy or security concerns, (iii) raise human rights concerns, (iv) have a detriment effect on people's livelihood economic security, (v) develop or extend harmful forms of surveillacce, (vi) severely damage the environment, or (vii) deceive people in ways that cause harm.
Lastly, our research does not use human-derived data, or has involved extensive annotation by human research participants. The synthetic dataset used in this paper has not been deprecated for technical, legal, or ethical reasons.
20


Reviews:
[{"review_id": "nqRM8aJFBk", "reviewer": "Reviewer_JUXh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. The idea to take the oracle into consideration is great. I also agree that it is important to distinguish the two different cases where the model's predictions are different from the oracle into causal and non-causal label flips.\n\n2. I like the discussion on the optimal classifier showing the impact of whether we can identify z from x on the robustness against the distribution shift of p(z).\n\nWeakness:\n\n1. I think the definition of successful CFE (SCE) should be more flexible, although it might fit the image classification scenario the best. For example, if the goal is algorithmic recourse, i.e., to rely on CFE to answer the question: how to modify features for better prediction results. Then any flip is okay, while the constraint may go beyond the requirement on L1 distance.\n\n2. The flow and presentation of the work can be improved, see requested changes.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work aims to address issues in benchmarking counterfactual explanations (CFEs) by creating a new dataset based on an exisitng synthetic dataset generator where the latent factors and the causal relations are given. They also propose two novel metric for evaluating CFEs, which benchmark the diversity and causality of CFEs obtained by existing methods. They also report important observations that helps better understand existing CFE methods for image data.\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Proposition 1 is listed without any explanation, the authors might want to add some implications of it after showing it on P5.\n2. The connection between the distribution shift on p(z) mentioned in the discussion of optimal classifiers and the main claims of this paper needs to be clarified.\n3. I did not get the argument that it is more useful to perturb non-causal attributes. It can be the case that even if you perturb the causal attribute slightly under the L1 norm constraint, the ground truth label may not always change while the prediction can change.\n4. The argument, these counterfactuals cannot be detected without causal knowledge, may not be true. It is possible to detect some of them without any causal knowledge. This is because the generated examples can be out of distribution, especially considering p(z|y) if y is flipped.\n5. It is better to use a running example from the introduction, instead of suddenly mentioning the example in P7.\n6. The choice of latent space is important, it might be better to show sensitivity analysis against different latent spaces (encoder).\n7. It is better to provide space and time complexity of the proposed algorithm 1.\n8. For computing the orthogonal and complement set, cosine similarity and a predefined threshold \\tau are used. It would be better to consider different similarity metrics and show a sensitivity analysis of the threshold.\n", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "XOzIrWeYb4", "reviewer": "Reviewer_oDaz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:** \n+ The paper tackles important problems \n+ The paper proposes a evaluation method for explainers\n\n\n**Weaknesses:**\n- The authors give a different definition of proximal than the one given in Rodriguez et al. 2021. Indeed, while they measure the proximity in the latent space, Rodriguez et al. refer to proximity as \"an explanation is proximal, if it is perceptually similar to the original image, for instance, a ninety degree rotation of an image would be a sparse but not proximal\", thus referring to it in the input space. \n- In the paper the authors discuss the importance of generating explanations that are proximal, diverse, and non-trivial. However, in Mothilal et al. 2020 and Russell 2019 the issue of sparsity is also discussed, i.e., an explanation should only change a minimal set of attributes. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper tackles two important problems in the explainability field: \n1. existing metrics do not reflect the goodness of the explanations, and \n2. the datasets where explainers are evaluated are often annotated with set of attributes that might not be complete, and the attributes themselves might be subjective. \n\nWhen proposing a solution, the paper focuses on the image domain, and it proposes: \n1. a synthetic benchmark where it is possible to evaluate the explainers and where the distribution that generates the data is known, \n2. a novel metric to evaluate the explainers on such benchmark which exploits the predictions made by an ``oracle classifier'' to decide whether an explanation is useful or not.  ", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would like the authors to answer to the questions that I outlined in the weaknesses. \n\nAnother point that I think is worth discussing is whether it is desirable to always have \"non-trivial explanations\". While we are machine learning scientists and thus we want to see non-trivial explanations to help us \"debug the model\" and get further insight, this might not be the perspective of a doctor or an other non specialist that is using a ML method. If we take the case of a doctor for example, it is very likely that they want to see the causal features as explanation, in order to assess whether it can trust the prediction or not. I thus think that including such discussion would be very helpful for the community, also to highlight that a \"one size fits all\" explainer does not exist, but that the desired properties for an explainer very much depend on the stakeholders of the system. \n\nFurther, for the sake of presentation the authors should rewrite algorithm one using pseudocode and giving more intuitions about the orthogonality requirement that they are asking the explainers to respect. I worry that it might penalise explainers that are producing good explanations just because they are not orthogonal. \n\nFinally, at page 2 the authors reference a Figure that is in the appendix. I think they should move it to the main body.\n\nTypos to fix: \n* Page 4: Atractive \n* Page 6: explanationmust ", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "tT0V744DN8", "reviewer": "Reviewer_qDzk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nStrengths:\n* The mtehodology that they propose is a fair methodology to evaluate counterfactual explanations methods.\n* To the best of my knowledge, it is the first work to introduce the notion of \"oracle\" for evaluating counterfactual explanations. Proposing a metric to numerically evaluate this notion it is not trivial and it is an strong point.\n\nWeaknesses:\n* Evaluation is only performed in one dataset and the conclusions are given generally. From my perspective, it appears that making the statement \"Our experimental results indicate that the different counterfactual explainers in the literature perform similarly and there has been little improvement in the recent years\" based solely on the results from a single dataset may be overly audacious.\n* Having an \"oracle\" can be difficult in many real-world problems.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this article, the authors present a benchmark with unified metrics to fairly evaluate different counterfactual explanation methods on image data. To achieve this, they use a synthetic image dataset that is fully described by annotated attributes.\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.  The proximal change is defined as $$P_r = \\{ z'\\quad | \\quad ||z-z'||_1\\leq r \\}$$ Is there any reason to use the L_1 norm here?  Needs clarification.\n2.  Minnor typo: When defining proximal change you wrote \"explanationmust\". Needs to be separated.\n3.  How do you choose the value of tau in algorithm 1? \n4. It is not clear what the classification problem is here. Is the classifier tasked with correctly classifying which character the input belongs to? It would be helpful to add a brief section or statement explaining this.\n5. In section 4.1, why is $\\hat{h_{causal}} = z_{causal}mod2$? This needs to be clarified.", "broader_impact_concerns": "This methodology could be used for evaluating new counterfactual explanations methods.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "cwUTvnE885", "reviewer": "Reviewer_roq8", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Evaluating explanations is a complex problem, and having standardized benchmarks and metrics is crucial for the development of the field. This paper makes a valiant effort at doing so.\n- Taking the effort to evaluate multiple explanation methods.\n\nWeaknesses:\n- Since one of the key contributions of this paper is proposing a synthetic dataset, I think that the paper could describe how this is done more clearly. Currently, this is described in Section 3.1 in two paragraphs. Since synbols has already been used as a benchmark in interpretability papers clearly defining the delta between what this paper does and synbols is important. \n- The paper makes the argument that CelebA is a bad dataset for this evaluation task because the causal generation process of the image is unknown. But even if some attributes are missing, does this invalidate the dataset because all datasets are missing some attributes, and can we be sure that these synthetic datasets have captured all attributes in an explicit manner? For example, the thickness of characters may perhaps be implicitly captured by the font, but is not explicitly captured as an attribute. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes 1) a synthetic dataset with images with many annotated attributes; 2) a set of metrics to evaluate the quality of counterfactual explanations, with the goal of addressing two issues that the authors pointed out, firstly flaws of existing explainability metrics, and secondly that there are not many datasets with a complete set of attributes that can affect a classifier's decision.\n\n\n\n\n", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Causal and vectorial notation are mixed and make the mathematical details harder to understand than necessary; please go through all the notation again and trying to define terms and math styling (e.g. when to bold things) more consistently. For example, in Section 3.1, \"a set of latent variables (\\mathbf{z}) results in an image (x) and a corresponding label (y).......have an impact on the label (\\mathbf{z}_ind)\". The table of notation in the appendix should probably also be in the main paper.\n- Clearly explaining how this paper builds on synbols to generate the synthetic dataset (see Weaknesses section above). \n- In Table 1, in the Domain column, all rows appear as \"z\", which makes that column not very informative.\n- A lot of important content in this paper is in the Appendix. For example, limitations section, counterfactual explanation images, broader impact. Some of these should be in the main paper.", "broader_impact_concerns": "I don't have concerns about broader impact of this paper, but feel that the broader impact section should be in the main paper rather than the Appendix, as it is right now.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed a controlled benchmark and a new metric for the evaluation of visual counterfactual explanation methods. The result points out some weaknesses of current evaluation practice and ways to mitigate these issues. The code and data are also published for reproducibility and reuse. The majority of reviewers suggest acceptance of this work, and I concur. 
Decision: Accept (poster)