Learning to Counter: Stochastic Feature-based Learning for Diverse Counterfactual Explanations
Authors: Vy Vo
Trung Le
Van Nguyen
He Zhao
Edwin V. Bonilla
Gholamreza Haffari
Dinh Phung
Venue: Submitted to ICLR 2023 (2023)

Abstract: Interpretable machine learning seeks to understand the reasoning process of complex black-box systems that are long notorious for lack of explainability. One growing interpreting approach is through counterfactual explanations, which go beyond why a system arrives at a certain decision to further provide suggestions on what a user can do to alter the outcome. A counterfactual example must be able to counter the original prediction from the black-box classifier, while also satisfying various constraints for practical applications. These constraints exist at trade-offs between one and another presenting radical challenges to existing works. To this end, we propose a stochastic learning-based framework that effectively balances the counterfactual trade-offs. The framework consists of a generation and a feature selection module with complementary roles: the former aims to model the distribution of valid counterfactuals whereas the latter serves to enforce additional constraints in a way that allows for differentiable training and amortized optimization. We demonstrate the effectiveness of our method in generating actionable and plausible counterfactuals that are more diverse than the existing methods and particularly in a more efficient manner than counterparts of the same capacity.

Under review as a conference paper at ICLR 2023
LEARNING TO COUNTER: STOCHASTIC FEATURE- BASED LEARNING FOR DIVERSE COUNTERFACTUAL EXPLANATIONS
Anonymous authors Paper under double-bling review
ABSTRACT
Interpretable machine learning seeks understand the reasoning process of com- plex black-box systems that are long notorious for lack of explainability. One growing interpreting approach is through counterfactual explanations, which go beyond why system arrives at certain decision to further provide suggestions on what user can do to alter the outcome. counterfactual example must be able to counter the original prediction from the black-box classifier, while also satisfying various constraints for practical applications. These constraints exist at trade-offs between
and another presenting radical challenges to existing works. To this end, we propose a stochastic learning-bast based framework that effectively balances the counterfactual trade-offs. The framework consists of generation and a feature selection module with complementary roles: the former aims to model the distribution of valid counterfactuals whereas the latter serves enforce additional constraints in way that allows for differentiable training and amortized optimiza- tion. We demonstrate the effectiveness of our method in generating actionable and plausible counterfactuals that are more diverse than the existing methods and particularly in a more efficient manner than counterparts of the same capacity.
INTRODUCTION
Recent advances in machine learning, especially the successes of deep neural networks, have promoted the use of these systems in various real-world applications. Such models provide remarkable predictive performance yet often at a cost of transparency and interpretability. This has sparked controversy over whether to rely algorithmic predictions for critical decision making, from graduate admission (Waters & Mikkullanenn 2014, Acharya et af.) 2019). job recruitment (Ajunwa et al., 2016) to high-stakes cases of credit assessment (Lessmann al. 2015) criminal justice (Lipton 2018
Gifford 2018). Progress in interpretable machine learning offers interesting solutions to explaining the predictive mechanisms of black-box models. One useful interpreting approach is through counterfactual examples, which sheds light on what modifications to be made to an individual's profile that can counter an unfavorable decision outcome from a black-box classifier. Such explanations explore ih scenarios that suggest possible recourses for future improvement. Counterfactual explanability indeed has important social implications both personal and organizational level. For instance, job applicants who get rejected by the CV screening algorithm of a company are likely to benefit from feedbacks like 'getting more referral' or 'being fluent at least 2 languages', , which would help them better prepare for future applications. At organizational level, by engaging with job candidates in this way as a form of advocating for transparency in decision making, companies improve employer branding and attractiveness to top talents. Internally, organizations can also validate whether any prejudice or unfairness towards a particular group is implicitly introduced in historical data and consequentially embedded in the classifiers producing biased decisions.
Related works. Recent years have seen explosion in literature counterfactual explanability, from works that initially focused on one or two specific characteristics or families of models to those that deal with multiple constraints and various model types. There have been many attempts to summarize major themes of research and discuss open challenges in great depth. We therefore refer readers to Karimi et al. (202bb) Verma al. 2020); Guidotti (0022) for excellent surveys of

---

Under review as a conference paper at ICLR 2023
methods in this area. We here focus reviewing algorithms that can support multiple even diverse counterfactual generations a property that has received less attention.
Dealing with the combinatorial nature of the task, earlier works commonly adopt mixed integer
programming (Russell, 2019). genetic algorithms (Sharma et al. 2020), or SMT solvers (Karimi
et al., 2020a). Another recent popular approach is sradient-based optimization (Mothilal et al. 2020
Bui et all22222)) In a similar fashion with adversarial learning (Goodfellow et al.] 2014). it involves
iteratively perturbing the input data point according to an objective function that incorporates desired constraints. The whole idea of diversity is to explore different combinations of features and feature values that can counter the original prediction while accommodating various user needs. To support diversity. Russell (2019) in particular enforces hard constraints on the current generations to be different from the previous ones. Such a constraint will however be removed whenever the solver cannot be satisfied. Meanwhile, Mothilal et al. (2020) and Bui et al. (2022) add another loss term for diversity using Determinantal Point Processes (Kulesza et al. 2012). whereas the other works only demonstrate the capacity to generate multiple counterfactuals via empirical results. Moreover, all of these algorithms are computationally expensive. Along the line, Redelmeier et al. (202T) attempts to model the conditional likelihood of mutable features given the immutable features using the training data. They then adopt Monte Carlo sampling to generate counterfactuals from this distribution and filter out samples that do not meet counterfactual constraints. Amortized optimization emerges as a more effective strategy that explicitly models the distribution of counterfactual examples via a generative model such as Variational auto-encoder (VAE) (Mahajan et al.) 2019 Pawelezyk et al., 2020 Downs et al., 2020) or a Markov decision process (MDP) model under a reinforcement learning setting (Verma et 2002). Obtaining such a distribution, sampling of counterfactuals can therefore be done straightforwardly.
Contributions. In this paper, we propose learning- based framework diverging markedly from the previous approaches. We reformulate the combinatorial search task into stochastic optimization problem that can be solved efficiently via gradient descent. Whereas the previous works model the generative distributions via MDP Verma et al. [002). VAE or VAE-based counterparts (Mahajan) cet al., 2019 Pawelczyk et al. 2020 Downs et al. 2020 we construct a learnable generation module G that directly models the conditional distributions of individual features such that they form valid counterfactual distribution when combined.
Another point of difference of our framework lies in the usage of Bernoulli sampling to ensure that only minimal changes are introduced to the generative counterfactuals. In prior works, standard metrics such as L.1 or T.2 is often used to penalize the distance between the counterfactual and original data point. Verma et al.] (2020) criticizes this approach as non-obvious, especially for handling categorical features. Avoiding the use of distance measures, we optimize feature selection module S to output Bernoulli distribution for each feature representing the likelihood of the feature being mutated. S is flexible module that can adapt to different user-defined constraints about mutability of features.
Similar to most works, our framework is developed to deal with heterogeneous tabular data. However, instead of one-hot encoding every categorical feature and treat each level as an individual numerical feature, we propose the opposite strategy: to discretize the numerical features and treat them as categorical. The benefits are four-fold: (1) we can conveniently apply one functional form over feature distributions, which then only requires one reparameterization trick feature-wise; (2) it helps expand the original input space that may later support generalization; (3) we believe that it yields more useful explanations and easier for human users to follow the suggestions than forcing them to meet hard requirements from specific numerical values; (4) it helps reduces privacy risks when revealing the counterfactual suggestions to the public. To facilitate end-to-end differentiable training, we employ the Gumbel-Softmax reparameterization trick for effective treatment of categorical features. This is the first time this strategy is used in this line of research.
Our contributions can be summarized as follows
We introduce Learning to Counter (L2C) stochastic feature- based learning approach for generating counterfactual explanations that address the counterfactual desirable properties in a single end-to-end differentiable framework.
2

---

Under review as a conference paper at ICLR 2023
We employ amortized optimization to efficiently generate actionable and diverse counter- factual explanations at 100% validity and adequate trade-off with sparsity. Our method additionally enforces plausibility by optimizing the generative examples on the input data manifold while maintaining one-hotness in the output representations for the categorical features.
Through extensive experiments on real-world datasets, L,2C is shown to balance the trade- offs among counterfactual constraints more effectively than the existing methods whereas achieving more diverse explanations and sufficient robustness in out-of distiibribtttonn settings. To the best of our knowledge, we are the first amortized method that supports diverse local counterfactual generations.
In the following, provide detailed description of the characteristics highlighted in bold.
2 DESIDERATA OF COUNTERFACTUAL EXPLANATIONS
The ultimate goal of this line of research is to provide practical guidelines as to what actions an individual can take to achieve desired outcome. Desiderata of counterfactual explanations have been extensively discussed in previous literature (Karimi et al. 2020b Verma et al. 2020) Guidotti 2022 Verma et al., (022). For pragmatic reasons, this paper focuses counterfactual explanations with the following characteristics:
Validity: By definition, counterfactual example must flip the original black-box outcome.
Actionability: Counterfactual explanations should be specific to individual user preferences and only suggest actionable feasible changes. In particular, changes should be made mutable features e.g., Work Experience or SAT scores, while leaving inmutable features unchanged e.g..
Gender or Ethnicity.
Sparsity: Counterfactuals should be close to the original example where a minimal number of features are modified.
Diversity: Diverse explanations are preferable to capture different preferences from the same user so that they can freely explore multiple options to select the best fit.
Plausibility: Plausible or realistic counterfactuals are to lie close to the training data manifold and obey the input domain. An example of implausible instances is one with numerical feature such Age being above 100 years old, or categorical feature with more than one category is assigned non-zero values in its one-hot representation.
Scalability: Inference should be done simultaneously and efficiently for multiple input examples.
While validity and actionability are the two must-have criteria to generate a practical counterfactual explanation, satisfying some, if not all, of the other constraints at the same time is a challenging task. First, there a trade-off between validity and spursity (Verma et al.) 2020). To counter the original outcome, it is naturally easier change large number of features without considering whether the features are mutable or not. On the other hand, strongly enforcing sparsity results in a smaller subset of features that be changed, which can compromise diversity since expect counterfactual
states to differ from one to another substantially.
With respect Plausibility, this property is often violated when categorical features not handled properly (Verma et al.] 2020). A common approach to pre-process categorical features is through one-hot encoding. However, the output representation of a counterfactual example may maintain one-hotness or probabilistic format where the sum of all levels equals to 1. This can lead to a discrepancy where its equivalent one-hot representation (i.e., the plausible representation) fails to produce the counterfactual outcome. Appendix B.2 will investigate this issue in more detail. Lastly, scalability is an important consideration and particularly significant in the face of diversity. This because the majority of previous methods optimize each input data point separately, and individual runs are additionally required to produce several counterfactuals. These algorithms are thus highly time-expensive for large datasets. Monte Carlo sampling (Redelmeier et al. 1021) l) and amortized optimization (Mahajan et al. 2019: Downs et al.) 2020; Verma et al. (002)) are recently adopted to boost efficiency. However, none of these methods addresses the diversity constraint nor is equipped with appropriate treatment for categorical features. We now explain how our framework L2C satisfies all the above constraints.
3

---

Under review as a conference paper at ICLR 2023
3 STOCHASTIC SEATURE-BASER COUNTERFACTUAL LEARNING
3.1 PROBLEM SETUP
Let X' denote the input space where æ: is input vector with N features of both continuous and categorical types. As discussed previously, we discretize the numerical features into equal-sized buckets, which gives us an input of N categorical features wherein each feature æ; has c, levels. We apply one-hot encoding on each feature and flatten them into a single input vector 2 E {0, 1] where D Ex= C₁- Concretely, feature H; is now represented by the vector 2, Oc, where the set of one-hot vectors Oc, is defined as 10: Σj=1 21) = 1.
Let f be the black-box classifying function and y f be the decision outcome the input valid counterfactual example o associated with æ is one that alters the original outcome into a desired outcome y' * y with y f(ž). Let Σ denote the corresponding one-hot representation of æ.
Actionability indicates that some features can be mutable (i.e., changeable), while others should be kept inmutable (i.e., unchangeable). Without loss of generality, let us impose an ordering the set of N features such that the first K features are mutable features (i.e., the ones that can be modified) and denote K : {1, ... K} C {1,..., N). For each mutable feature (i.e., 2; or the one-hot vector 2j with i E IK), we aim to learn a local feature-bass perturbation distribution P(z; z) where 2, Dc., while leaving the immutable features unchanged.
To assure fair comparison, we train the same black-box classifier for both ours and the baseline methods. Specifically, the input to the classifier is the representation where only categorical features are one-hot encoded while numerical features are retained at their original values. Later, whenever necessary consult the black-box model, for every numerical feature we convert the one-hot representation ž; back to (a;, b.] where (as, bi] is the original value range of the feature.
3.2 METHODOLOGGY
We now detail how L2C works and addresses each counterfactual constraint. Our framework is summarized in Figure
For each mutable feature 2, with i E K, we learn a local feature-based perturbation distribution P(ž, 2) (i.e., & E O, which is categorical distribution Cat(p; z) with category probability p; = P12. Ps...... P.c.]. We form counterfactual example ž by concatenating 2 Cat(p; 2) for the mutable features and x; for the immutable features. To achieve validity, we learn the local feature-based perturbation distribution by maximizing the chance that the counterfactual examples z counter the original outcome on æ.
Additionally, learning local feature-basse perturbation distributions over the mutable features allows us to conduct a global counterfactual distribution P(ž 2) over the counterfactual examples defined above. Sampling from this distribution naturally leads multiple counterfactual generations efficiently, and we also expect that individual samples ž, together can form diverse combinations of features, thereby promoting diversity within the generative examples.
As previously discussed, too much of diversity can compromise sparsity. Dealing with this constraint, for each mutable feature z;, we propose learn local featuur-based selection distribution that generates a random binary variable s; ~ Bernoulli(T; 2) wherein we replace 2, by 2, ~ Cat(p; 2) s; 1 and leave 2; z; s; Therefore, the formula to update ž, is
zz = s:) 8;
The benefit of having = [Ta]ke is thus to control sparsity by adding one more channel to decide if we should modify mutable feature z;. Appendix B..4:.B.ppeeeeee 4reeeeetss an ablation study showing that without the selection distribution, the perturbation distribution alone can generate diverse counterfactuals but it changes plenty of mutable features. Meanwhile, optimizing the selection distribution jointly helps harmonize the trade-off between diversity and sparsity.
4

---

Under review as a conference paper at ICLR 2023
3.3 OPTIMIZATION OBJECTIVE
In this section, explain how to design the building blocks of our framework L2C. As shown in Figure [ our framework consists of two modules: counterfactual generator 9 and feature selector S. The counterfactual generator G is used to model the feature-based perturbation distribution, while feature selector S is employed to model the feature- based selection distribution.
Specifically, given one-hot vector representation of data example we feed to to form ((2)
We then apply the softmax activation function to Gi(z) to define the feature- based local distribution (i.e., Cat(p: z))) for 2;
exp {gis(z)} exp (9)
Piz(2)
Vj 1, Gz
S takes to form S(z) (x))xK We then apply the Sigmoid function S,(z) to define the feature-bet based selection distribution (i.e., Bernoulii z)) for 2, as
Te (2) 1 exp (z))
To encourage sparsity by reducing the number mutable features chosen to be modified, we regularize S through L1-norm [| with rT)) {>)<))
To summarize, given an one-hot vector representation z data example æ. we use G to work out the local feature- nasture-based perturbation distribution Cat(p,(z)) for every i K. We then sample 2 ~ (at(p;(=)) for every i E K. Subsequently, we use S to work out the local feature-base selection distribution Bernoulli(T, (=)) for every I. We then sample 8; Bernoulli 2) and update 2, (1 - 8: 2; 8;ž; for every i E K. Finally, we concatenate 2; for E and z; for K
form the counterfactual example
G and S are parameterized with neural networks over total parameters O. For ž to be valid and sparse counterfactual associated with a desired outcome y/' we propose the following criterion
ming
Ex (CE(f(Z). + E. [I++))]]]]]
where f the black-box function, CE is the cross-entropy loss, || ||2 is L1-norm, a loss weight.
Notice that ž formed by concatenating many one-hot vectors is an incompatible representation to the classifier f. which in fact requires both continuous and one-hot features. our implementation, we reconstruct the numerical features by taking the middle point of the range corresponding to the selected level. We refer to this process as one-hot decoding. Specifically, the input to the one-hot decoder is If the feature originally is categorical feature, we set 3; 2, Otherwise, we set 2;; = a, (2k (ob) which the middle point of the interval (a, [a, (A: - 1)b ((0;--;), as as+++1 k-(bb -a))
E |
2
where [a,, b.] is the range of the feature and ε, corresponds to the level & {1, (1,... e.) (i.e., Esx =
and žU = 0 if * k). To facilitate the continuous relaxation Scction 3.4 we rewrite
ass + (2j 1)(b: a:)
Rij
(2)
The final optimization objective is now given as
ming Ex
[CB(F()). + Ex [I](()]]]]
(3)
3.4 REPARAMETERIZZATIO FOR CONTINUOUS OPTIMIZATION
Our L2C involves multiple sampling rounds back and forth to optimize the networks. To make the process continuous and differential for training, adopt the following reparameterization tricks:
5

---

Under review as a conference paper at ICLR 2023
Ir Cat (P:)
III (px) R2
Generator g
If 1. perturb
II Cat (Px) RN
feature else retain
X2
One-hot Encoder
Z/2
8 s;) +
Ber (1t)
Xy
N
Selector
(1))
Concat N vectors 2/
Bern ) 5p
clasaiffer boundary
81 Z2
N
orginaa Input
5
One-hat
counterfactul local space
81 X2
XN
Decoder
Figure I: For illustration purpose only, the figure assumes all features are mutable. Given an input a, discretize the numerical features into categorical levels and one-hot encode all features into representations z. For every feature i, the generator G learns a local perturbation distribution Cat(p;| so that together they form a distribution of diverse counterfactual representations ž. Intuitively, G aims to construct "bridge" across the decision boundary travelling from the input to local space of counterfactuals. Simultaneously, the selector S learns to output the distribution Multi-Bernoulie capturing the probability of each feature i being modified. Every pair of feature sample (ž,, s;) passed through operation the blue box, which decides whether to accept the change being made to the feature given by 2; and enforces feature actionability accordingly. The output is then decoded into the representations compatible with black-box system. G and are jointly trained via socc-propagation according to the objective given in Eq. 3 where the first loss term ensures G produces valid counterfactuals whereas the second term encourages to produce sparse feature combinations. We assure plausibility by optimizing both distributions on input data manifold and utilizing reparameterization trick to handle categorical features. Finally, through amortized optimization, L2C supports scalability for efficient generations of multiple counterfactuals.
1) Sampling & ~ Cat(p: x)
To obtain differentiable counterfactual samples, we adopt the trick (Jang et al. 2016 Maddison et al.) 2016). Given
classic cempeeratureddpeendee the categorical variable z; with category probability [Pa, Pr2. .... Pic, The relaxed representation is sampled from the Categorical Concrete distribution as 2, ~ Col-CConcete(logpu.... Pii, log Pic.) by
exp
{log Piy + G2)/T)
1=1 exp {(log (2) + Gt)/T}
with temperature т, random noises Gj independently drawn from Gumbel distribution G5 llloo-llogg log u.), Ue Uniform(0, 1). As discussed, we apply this mechanism consistently to the one-hot representations of all features. The continuous relaxation of Eq. (2) can be gained by simply using the one-hot relaxation Σ,.
2) Sampling Bernoulli variables of categories. With temperature т, random noises Go and G. G, sog(-log log ut), We Uniform(0, 1), the continuous representation s; is sampled from Binary Concrete distribution as 8, Bin-Concrete(m,, π.) by
Bernoulli(m; 2) We again apply the Gumbel-Softmal trick to relax
log(-
exp((1og n, (z) Gn)/T) exp(( TT(2)) + )/7) exp (log ( Ga)/T)
EXPERIMENTS
We experiment with popular real-word datasets: German Credit (Dua & Grafi 2017). Graduate Admission (Acharya et al. 2019), Student Performance (Cortez & Silva 2008) and Small Business Administration (SBA) (Li et al., 2018). For the last 2 datasets, we adopt the setup from Bui et al. (002),, which introduces temporal and geospatial shifts respectively to the testing sets. The goal

---

Under review as a conference paper at ICLR 2023
is to additionally validate how robust the proposed method is to covariate shifts. For each dataset, we select a fixed subset of immutable features based on our domain knowledge and suggestions from (Verma et al., 2022). While implementing the black-box classifiers and the baseline methods, we standardize numerical features to unit variance and one-hot encode categorical features. Note again that, for our method only, we discretize numerical features into equal-sized buckets and decode the numerical features back to their original representations whenever feeding them to the black-box model. Appendix A describes our tasks and model design in greater detail with the discretization setup specifically reported in Appendix A.3] Our code repository can be accessed at 4open. wwwencceeeeeeeeeeceeeeeeee AD511
Table I: Description of quantitative evaluation metries. C denotes a set of counterfactual samples generated by an interpreting method for given input example.
Property
Metrle
Description
Validity
Validity
counter the original black-box decision outcome. Coverage = 100% if there exists at least valid counterfactual in C.
Proportion of samples
Coverage
Sparsity/
Sparsity
Proportion of features heing changed, averaged over the numher of samples in C. L1 distance the input data point counterfactuul sample across the numerical features, averaged over the number of samples in C.
Actionability
Prox Num
Prox Cat
inpul data point a counterfactual sample across calegorical features, averaged over the number of samples in C.
Harmming distance
Diversity
Diveralty
Hamming distance a pair of counterfactual samples across all features where numerical features are discretized. metric is averaged over all pairs samples in
Plausibility
Manifold Dist.
1.1 distance of counterfactual sample closest training data points across the numerical
features. Nearest instances are determined by a pre-trained KNN algorithm. The metric is averaged over the number samples in C. Proportion of categorical features in counterfactual sample having a proper one-hot or probabilistic representation which sum of values at all levels equal The metric averaged over the number of samples in C.
Valid Cat
4.1 PERFORMANCE METRICS
Following the past works Mothilal et al.] (2020); Redelmeier et al.] (2021); Verma et al. (2022),
Table summarizes the commonly used metries for quantitatively assessing how well method satisfies the constraints outlined in section [] As for Divers veersittt irrsityy ty, a widely adopted measure is the
pairwise distance between counterfactual examples, with distance defined separately for numerical and categorical features (Mothilal et al. 2020 Redelmeier et al.] (02II. Though this approach is meaningful for interpreting categorical features, we however find quite obscure for numerical features. This motivates us to discretize numerical features again when computing Divers ity, specifically via how often it switches to a different categorical level. The computation of Di versi ty only considers valid counterfactuals, so if none of the examples in C are valid, D Diversi ty is set to zero.
which captures how often a feature gets altered as well as how much the change
4.2 BASELINES AND EVALUATION SETUP
We compare our method against existing amortized approaches: Feasible-VAL (Mahajan et al., 2019). CRUDS (Downs al. 2020), FastAR (Verma et al. 22022), and popular non-amortized methods that support multiple counterfactual generation: Mothill et al. 2020). CERTIFAI (Sharma et al. 2020). MCCE (Redelmeier et al. 202T) and COPA (Bui et al. 2022). DiCE offers several search strategies: Random, KD-tree, or Genetic algorithm. HiCE-Random is adopted in our experiment since it is reported to consistently yield the best performance across datasets (Verma et al., (022). We do not consider MOC (Dandl et al.) 2020), which does not support Python implementation, or MACE (Karimi al. 2020a) due to its convergence negatively depending on the SAT solver, which is also extremely expensive on large datasets (Verma et al. 2022).
Here we consider a general setting of binary classification where a counterfactual outcome y/ is opposite to the original outcome y, whether y is positive or negative. From each method, we generate a set of 100 counterfactual explanations. During generation, most methods, including ours, require multiple iterations of searching for the optimal set of counterfactuals based on the optimization constraints. To assure a fair comparison on efficiency, a global maximum time budget of 5 minutes is imposed to search for set of 100 counterfactuals per input sample. For amortiedd baselines,
7

---

Under review as a conference paper at ICLR 2023
the algorithms directly the testing sets, while for amortized methods, we train the base generative models on the training sets and use the testing sets only for evaluation. We tune the generative models via grid search and report the best settings in Appendix which also includes the description about our special treatment for FastAR.
5 RESULTS AND DISCUSSION
We train each black-box classifier on 5 model initializations and report the average results in Table and T We also report Time that records the total inference time in seconds for a set 100 counterfactuals of all testing inputs. Note that COPA has only been shown to work effectively linear classifiers. Appendix DisiixxDpprooee provides several illustrative examples for qualitative assessment.
Given the same time budget, our method 1.2C succeeds in generating 100% valid counterfactuals with full coverage. Together with DiCE and Feasible- 1.2C first satisfies the most important criterion of counterfactual explanation and resolves the trade-off against validity. Since the trade-off between sparsity and diversity often hinders comparison, we attempt to approximate the numerical proximity level of the closest baseline DiCE by setting sparsity threshold between and during inference. The thresholds are chosen in such way that no more than 2 numerical features get modified in one sample. Valid counterfactuals exceeding the threshold are filtered out. Recall that we have specified fixed set of immutable features for each dataset, based which we can work out the maximum sparsity threshold counterfactual explanation should adhere to (See Appendix (.T).. Feasible- VAE, MCCE and CRUDS cannot fulfill this constraint for all datasets. This is critical drawback of Feasible- VAE whose implementation in fact does not consider mutability of features. It is also evident that I.2C is the second-fastest generator while consistently achieving the best diversity of all methods.
Table 2: Comparison of interpreting methods in-distribution datasets and the corresponding black-box architectures. Lower is better. Higher is better. Bold indicates the best performance for each dataset. Underline indicates the best performance among methods achieving 100% Validity.
Method
Prox Num
Prox
Diversily Sparsily
Validily
Cuverage Manifold
Valid
(111
Cat (8)
(EIf
(8)U
(831
(6)
Dist
Cat (h)
(l.ogistic Regression)
Spursity:
80.00%
L2C (Ours)
0.44
30.01
37.81
29.46
100,00
100.00
0..4
100.00
33
DICE
15.75
11.77
1.150
F-VAB
0.70
45.96
2.13
34,07
100,00
100.00
0.14
0.00
36
COPA
0.69
19.77
42.12
44.00)
44.00
0.29
17,583
MCCE
1.05
66.17
33.40
71.24
48.74
100.00
0.35
57.44
2
CERTIFAI
1.16
51.77
9.92
59.00
53.52
61.49
0.48
101.00
50.258
FastAR
0.01
CRUDS
0.74
61.76
1.55
67,50
44.00
44.00
0.23
9.74
42,920
Graduate Admission (Neural Network)
Max. Sparsity: 85.71%
L2C (Ours)
1.12
33.52
39.59
43.56
100.00
100,00
1.04
101.00
DICE
1.17
22.00
33.06
33,75
100,00
100,00
1.30
73.50
412
R-VAB
0.86
86.50
1.54
92.29
100.00
100,00
0.00
MCCE
0.99
69.57
22.98
82.61
43.79
$4.60
0.35
68.64
FastAR
0.66
58.69
1.07
72.03
87.41
87.41
0.32
72.30
5,405
CRUDS
0.84
75.75
1.39
86.14
51.48
$2.00
0.20
11.00
21,460
As shown in Figure 2 L2C and DiCE are the two most effective in balancing the trade-offs among counterfactual constraints, which L.2C converges this level of performance significantly faster. To quantify the effectiveness in balancing the trade-off between sparsity and diversity, we propose taking
Harmonic mean of Diversityy and (1 Sparsityl It follows the motivation Fl -core
in measuring Precision against Recall, which thus equals to Eiversisyt (1-Sparsaty) the superiority of our method in addressing this trade-off based on the proposed score where L2C specifically outperforms the second-best method by 2.43% 18.33% across datasets.
2nniversitye (1-SpasssSSpppp- ty). Figure 3 shows
In Appendix B.3
we further evaluate the effects of different threshold levels, showing that setting the thresholds higher than the current levels can further improve inference time and diversity while maintaining good balance against sparsity. We also refer readers to Appendix [.I and B.2 where discuss plausibility and robustness out-of-distribution datasets in detail.
8

---

Under review as a conference paper at ICLR 2023
L2C
DICE
F-VAE
(hours)
(hours)
Time (hours)
(hours)
Nam
Num
Prox Nuim
Prox Num
Prex
Sparsity
Sparsity
Sparsity
Sparsity
Homogeneity German Credit
Homogenelty Graduate Admission
Homogeneity
Homegeneity
Student Performance
SBA
Figure 2: Visualizing methods with 100% Validity: L2C, DiCE and Feasible- VAE across important constraints. Homogeneity 1 Di versi ty and i me is converted hours for scaling down the value range. All metrics are preferably lower to be better. 1.2C lies closely towards the middle indicating that our method reaches more effective balancing point.
BICE
DICE
COPA
COPA
MECE
MCCE
CERTIFAI
CERTIFAL
CORTIFAA
CEATIPAL
FactAR
FasDAR
FactAR
PastAl
CRUDS
16
26
50
10
20
33
Petiomaace
Figure 3: Comparing method capacity in balancing the trade-off of diversity and sparsity based on the Harmonic mean of Diversity and (1 Sparsity).
CONCLUSION AND FUTURE WORK
Diverging from the previous approaches, contribute novel counterfactual explanation framework that effectively resolves the trade-offss among counterfactual constraints. Here we target a broad class of differentiable machine learning classifiers. To fit non-differentiable models in framework, one could use policy gradient (Sutton et al. 1999) or attempt to approximate such models as decision trees random forests with differentiable version (Yang al. 2018 Lucic et al. (022).
Although we have covered the essential desiderata counterfactual explanation, addressing causal relationships among features is beyond our scope in this paper. Handling correlational or causal constraints is indeed a non-trivial challenge, which often requires domain knowledge and/or the true structural causal models. There only few works addressing this, notably MMhajann et al. (2019); Karimi et al. (202]); Verma et al. (2022), by assuming access to partial knowledge or utilizing constraints from user specifications. However, there have been no proper quantitative metrics to evaluate whether such constraint is satisfied in practice. Recall that in this work, we assume the conditional distributions of each feature P(z,(ar be independent. We thus can model the joint distribution among any 7: features as P(z1, ..., 2t/ = I (xr,(=). Sharing the same motivation, one can extend L2C impose constraints such joint distribution among the related features to ensure the generative samples respect causal constraints.
In recent years, there is increasing concern regarding the privacy risks of counterfactual explana- tions (Shokri et al. [021] Goethals et al. 2022), Handling the quality of explanations while taking security and privacy into account remains an interesting open challenging this research area. Notice one of the main differences of our work is that we discretize data into categorical features. Thus, we believe this approach would help reduce the privacy risks of model stealing or sensitive data exposure when releasing the counterfactual suggestion to the public. Our future works will focus on tackling this issue more rigorously.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Mohan S Acharya, Asfia Armaan, and Aneeta S Antony. A comparison of regression models for prediction of graduate admissions. In 2019 international conference on computational intelligence in data science (ICCIDS), pp. 1-5. IEEE, 2019.
Ifcoma Ajumwa, Sorelle Friedler, Carlos E Scheidegger, and Suresh Venkatasubramanian, Hiring by algorithm: predicting and preventing disparate impact. Available at SSRN, 2746078, 2016.
Ngoc Bui, Duy Nguyen, and Viet Anh Nguyen. Counterfactual plans under distributional ambiguity. arXiv preprini 2rXiv:2201. /2487, 2022.
Paulo Cortez and Alice Maria Gonçalves Silva. Using data mining to predict secondary school student performance. 2008.
Susanne Dandl, Christoph Molnar, Martin Binder, and Bernd Bischl. Multi-objective counterfactual explanations. In International Conference on Parallel Problem Solving from Nature, pp. 448 469.
Springer, 2020.
Michael Downs, Jonathan Chu, Yaniv Yacoby, Finale Doshi-Velez, and Weiwei Pan. Cruds: Counterfactual recourse using disentangled subspaces. ICML WIII, 2020:1-23, 2020.
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL httt::://ccchi// ics iccsuci. edu/ml
Richard Gifford. Legal technology: Criminal justice algorithms: Ai in the courtroom. Proctor, The, 38(1):32-33, 2018.
Sofie Goethals, Kenneth Sörensen, and David Martens. The privacy issue of counterfactual explana- tions: explanation linkage attacks. arXiv preprint arXiv:2210. 12051, 2022.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv: 141.:224 2014.
Riccardo Guidotti. Counterfactual explanations and how to find them: literature review and bench- marking. Data Mining and Knowledge Discovery, pp. 1-55, 2022.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv: 1.01 144, 2016.
Amir-Hossein Karimi, Gilles Barthe, Borja Balle, and Isabel Valera. Model-agnostic counterfactual explanations for consequential decisions. In International Conference on Artificial Intelligence and Statistics, pp. 895-905. PMLR, 2020a.
Amir-Hossein Karimi, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. survey of algorithmic recourse: definitions, formulations, solutions, and prospects. arXiv preprint arXiv:2010.04008 2020b.
Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. Algorithmic recourse: from coun- terfactual explanations to interventions. In Proceedings of the 2021 ACM conference fairness, accountabiliry, and transparency, pp. 353-362, 2021.
Jack Klys, Jake Snell, and Richard Zemel. Learning latent subspaces in variational autoencoders. Advances in neural information processing systems, 31, 2018
Alex Kulesza, Ben Taskar, et al. Determinantal point processes for machine learning. Foundations and Trends in Machine Learning, 5(2-3):123-286, 2012.
Stefan Lessmann, Bart Baesens, Hsin-Vonn Vonn Scow, and Lyn C Thomas. Benchmarking state-of- the-art classification algorithms for credit scoring: An update of research. European Journal of Operational Research, 247(1):124-136, 2015.
Min Li, Amy Mickel, and Stanley Taylor. "should this loan be approved or denied?": large dataset with class assignment guidelines. Journal of Statisties Education, 26(1):55-666 2018.
10

---

Under review as a conference paper at ICLR 2023
Zachary C Lipton. The mythos of model interpretabillty: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):11-57,000 2018.
Ana Lucic, Harrie Oosterhuis, Hinda Haned, and Maarten de Rijke. Focus: Flexible optimizable counterfactual explanations for ensembles. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 5313-5322, 2022.
Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv: 1.00112, 2016.
Divyat Mahajan, Chenhao Tan, and Amit Sharma. Preserving causal constraints counterfactual explanations for machine learning classifiers. arXiv preprint 1912.037// 2019..
Ramaravind K Mothilal, Amit Sharma, and Chenhao Tan. Explaining machine learning classifiers through diverse counterfactual explanations, In Proceedings of the 2020 conference on fairness, accountability, and transparency. pp. 607-617, 2020.
Martin Pawelczyk, Klaus Broelemann, and Giergji Kasneci. Learning -adel-agnostic counterfactual explanations for tabular data. In Proceedings of The Web Conference 2020, pp. 3126-3132, 2020.
Annabelle Redelmeier, Martin Jullum, Kjersti Aas, and Anders Løland. Mcce: Monte carlo sampling of realistic counterfactual explanations. arXiv preprint arXiv:2111.09799 2021.
Chris Russell. Efficient search for diverse coherent explanations. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 20-28, 2019.
Maximilian Schleich, Zixuan Geng, Yihong Zhang, and Dan Suciu. Geco: quality counterfactual explanations in real time. arXiv preprint (rXiv:2101.01292 2021.
Shubham Sharma, Jette Henderson, and Joydeep Ghosh. Certifai: A common framework to provide explanations and analyse the fairness and robustness of black-box models. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pp. 166-172, 2020.
Reza Shokri, Martin Strobel, and Yair Zick. On the privacy risks of model explanations. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 231-241, 2021.
Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. Advances neural information processing systems, 12, 1999.
Sahil Verma, John Dickerson, and Keegan Hines. Counterfactual explanations for machine learning: A review. arXiv preprint arXiv:2010. 10596, 2020.
Sahil Verma, Keegan Hines, and John P Dickerson. Amortized generation of sequential algorithmic recourses for black-box models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 5512-85199 2022.
Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harv. JL & Tech., 31:841, 2017.
Austin Waters and Risto Mikkullanen.. Grade: Machine learning support for graduate admissions. Ai Magazine, 35(1
31))))6444000)000)))))) (:644666644 2014.
Yongxin Yang. Irene Garcia Morillo, and Timothy M Hospedales. Deep neural decision trees. ICML Workshop on Human Interpretability in Machine Learning., 2018
11

---

Under review as a conference paper at ICLR 2023
A EXPERIMENTAL DETAILS
A.1 DATASET STATISTICS
This section provides additional details about each dataset and experimental setup.
German Credit (Dua & Grafi 2017): This dataset includes information of customers taking credit at a bank. The task is to classify customer as good (label 1) or bad (label 0) credit risk. The immutable features include Foreign worker, Number of liable people, Personal status, Purpose (Verma et al. 2022).
Graduate Admission AAcharya et al. 2019): The set contains data of Indian students' applications Master's program. The original target variable is an ordinal variable on the scale of [0 1] indicating the chance of student being admitted, where 1 indicates the highest chance. We set threshold of 0.7 and re-categorize students as either "having higher chance" (2 7.70-label 1) or "having lower chance" (< 0.70-label 0). The binary classification task is to determine if a student profile has a higher chance being successful at their application. The immutable feature is chosen to be University Rating.
Student Performance (Cortez. & Silva 2008): This dataset records the performance of students at two schools Gabriel Pereira and Mousinho da Silveira. The task is to predict whether a student achieves final score above average (label 1) or not (label 0). The train and test splits contain data of students from these two schools separately (Bui et al. 2022). The set of immutable features contains Mother's education, Father's ss education, Family education support, First period grade.
Small Business Administration (SBA) (Li al. 2018): The dataset presents data on loan approvals of small businesses from 1989 to 2014. The train split contains records of the period 1989-2006, and the test split includes observations of the period 2006-2014. The task is to classify business with a high (label 1) or low risk (label 0) of loan default (Bui et al.) 2022). The immutable features consist of From Urban or Rural, New or Existing business, Active loan in Recession not.
Regarding the underlying black-box models, we experiment with Logistic Regression for the linear classifier and Neural Network for the non-linear classifier. We train linear classifiers on German Credit and Student Performance, and non-linear classifiers on Graduate Admission (with 3 layers and 40-dimensional hidden units) and SBA (with 3 layers and 30- 30-dimensional hidden units). For each task, we further sample 20% random observations of the training sets as validation sets and train 5 black-box models with the same architecture but with different initializations. Since we have specified a fixed subset of immutable features, we therefore can derive the maximum sparsity level an explanation method should obey
No. immutable features
Total no. features
Table 3: Dataset statistics and Black-box performance.
Black-box Architecture
Dataset
Train/Dev/Test
Features
Test Accuracy
Max. Sparsity
German Credit
(400/10//00
20
Logistic Regression
67.00%
80.00%
Graduate Admission
320 100
Neural Network
90.60%
85.71%
Student Performance
339/84 /2//////222 /22
Logistic Regression
94.69%
71.43%
SBA
755/ 11//88//1159
12
Neural Network
90.39%
75.00%
12

---

Under review as a conference paper at ICLR 2023
A.2 MODEL DESIGN
We parameterize G and S each with a 2-layer neural networks. Each layer consists of a 50- dimensional dense layer, followed by ReLU activation. The final layer of is another dense layer that outputs logit vector of the same dimension as the input, representing the counterfactual distribution. & produces a probability vector, thus laking Sigmoid as the output layer. We set the sparsity loss coefficient a = 0.001 and use the same architecture for all tasks. We train our model with Adam optimizer for 200 epochs, at 0.2 and learning rate of 0.0001.
A.3 DISCRETIZATION MECHANISM
Numerical features are discretized based on quantile values using Python function qeut which requires specifying the maximum number of buckets/levels and later adjusts it depending on the input data distribution. We set the maximum buckets be 4 such that each bucket has least. 100 data points.
Choosing very few is likely to cause under-fitting since there are very few useful combinatorial patterns that can counter the original label. We need sufficient diversity for effective learning. However, too many buckets are undesirable since it can hurt generalization due to some following reasons: (1) each bucket would contain too little data and the chosen middle value (used to decode one-hot vector as described in section 3.3 may not represent the bucket well, and (2) the model has more combinations of features to explore, thus can converge to sub-optimal combinations that cannot generalize well on unseen test points. The impact of discretization also depends on the proportion of numerical features in the data compared with categorical features. It would be less severe if there are more categorical features to provide sufficiently useful patterns for generalization. therefore decide split data into equal-sized buckets in the hope of balancing the trade-off.
this regard, we
Table reports how the numerical features of each dataset are discretized. For illustration purpose only, round the edge values to the nearest whole number.
B ADDITIONAL EXPERIMENTS
B. ROBUSTNESS
Table that L2C can adequately interpret out-of-distribution examples. One useful application when, for example, a student from different school wishes to transfer to the school which provides the data to train the models. Our system can inform the student of whether their academic performance suffices or what they should further do to increase the chance of being admitted. To explain this capability, since our framework optimizes local counterfactual distributions for each feature, L2C can cover sufficiently large space of outputs to support generalization. This strategy is believed to be more effective than those optimizing for single output data point, around which one can imagine that the distribution, if ever there was one, would have an extremely sharp peak, therefore more likely to fail under severe shifts. Note that robustness analysis is irrelevant for the non-amortized methods which are directly optimized on the testing sets our experiments.
B.2 PLAUSIBILITY
Table also demonstrate the capability of maintaining proper representations for categorical features throughout the generative process. In our framework, the one-hot encoded representations are relaxed to be continuous during training through the lemperature-dependenn Gumbel-Softmax trick. In principle, as the temperature approaches zero, the continuous representations get closer to the binary vectors i.e., discrete Multi Belti-BBrooulli samples, and we thus can expect the continuous samples to approximate the behavior of the discrete samples. During inference, we revert to discrete sampling and maintain one-hot representations in the generative samples.
There are many methods with Val id Cat far under 100% meaning that the output representations for some categorical features are neither one-hot nor a probability vector. We often infer an expla- nation by selecting the categorical level with the maximum vector value. The inferred category by
https::/ wwdatsssppppaa.o.m http://///aaaaaass.crc/////////s
qcut.
13

---

Under review as a conference paper at ICLR 2023
Table 4: Comparison of interpreting methods out-of-distributioo datasets and the corresponding black-box architectures. Lower is better. Higher is better. Bold indicates the best performance for each dataset. Underline indicates the best performance among methods achieving 100% Validity.
Method
Frox
Froz Cat (8)
Divarsicy Sparsity
Validity Coverage Manifold
Valid
Time (s1)
(NIT
($72
(4)
Dist.
Cat (k)
Student Performance (Logistic Regression)
Max. Spursity:
71.43%
L2C (Ours)
0.54
22.42
25.79
23.02
100,00
100,00
1.63
101.00
36
DICE
0.73
12.78
12.40
100,00
100.00
91.36
2,5T8
F-VAB
0.88
61.69
8.23
72.63
100.00
100.00
0.52
0.00
36
COPA
0.79
29.37
25.56
49.55
67.26
67.26
0.64
0.02
18,774
MCCE
0.96
63.65
24.97
74.03
60.98
93.10
0.84
71.38
CERTIFAT
1.05
0.00
11.88
28.57
98.23
98.23
1.69
100.00
834
FasAR
CRUDS
0.96
66.32
3.50
75.94
39.89
60.09
0.84
0.00
39,571
SBA (Neural Network)
Max. Sparsity: 75.00%
LZC (Ours)
1.09
6.75
14.20
17.67
100.00
100,00
6.22
100.00
310
DICE
1.18
0.69
T1408
12.78
100,00
100,000
6.09
97.78
5,523
R-VAR
76.62
0.00
1,716
MOCE
0.88
44.6.
8.11
81.54
33.21
69.28
3.33
25.00
CERTIFAT
4.97
0.00
12.87
66.67
91.23
25.04
100.00
1,833
FIStAR
0.51
21.91
0.16
63.41
86.77
86.77
0.97
86.13
55,175
CRUDS
0.70
63.51
1.8
87.84
58.51
62.24
2.12
24.01
248.719
convention corresponds to equivalent one-hot representation, which is the plausible input format fod to the black-box classifier. Hence, a discrepancy can occur in that the output representation yields the desired outcome whereas the equivalent one-hot one may not, which poses critical reliability issue (Guidotti 2022).
We investigate the violation of plausibility due the failure maintain one-hotness in the categorical features of generative example. When processing categorical features, previous strategies include imposing regularization (Verma al. 2020), clamping each one-hot column be specific categorical value (Wachter et al., 2017 Downs et al. 2020), relying on genetic algorithms or SMT solvers for automatic treatment (Karimi et al. 2020a Schleich al. 2021 or simply filtering them out (Lucic et al., 2022). It is clear that these techniques are insufficient to ensure all levels of categorical feature are effectively handled, resulting in plenty of methods failing achieve 100% on Valid Cat reported in Section
We hypothesize that the black-box outcome on the output representation of a categorical feature is sometimes not aligned with the outcome on its equivalent one-hot representation. Specifically for categorical feature, some methods output continuous representation like [0.05, 0.01, 0.02, 0.1] that may be not be summed to 1. However, when returning the qualitative explanation to users, we need to return the inferred categorical level, which is 4 in this example. This level is thus equivalent to the one-hot representation [0, 0, O, 1]. Since the compatible representation of category features to the black-box classifiers is one-hot format, there may be a mismatch in the prediction on these two different representations for the categorical feature that is, the continuous can counter the original outcome while its equivalent one-hot one may not.
Given output representation, for example [0.05, 0.01, 0.02, 0.1). we adopt the categorical level with the maximum value as the explanation i.e., 4 in this case. Given this level, we encode the feature back into the proper one-hot representation i.c., [0, 0, O, 1]. We assess Validite and Covera Cagerage of generative examples given the new representations and study this behavior in DiCE (Mothilal et al. 2020) and Feasible VAE (Mahajan al.) 2019) the baseline methods performing best on these metries. revealss that on some datasets, the performance of these methods deteriorates when evaluated on the equivalent one-hot representations.
B.3 EFFECT OF SPARSITY THRESHOLDS
Figure iillustrates that increasing the sparsity thresholds imposed on the numerical features boosts time efficiency and diversity without compromising too much of sparsity. Here we also want to highlight the flexibility of our framework in controlling the quality of counterfactual generations during inference. Users can now freely set additional constraints to filer out unsatisfactory examples based their preferences without re-training re-optimization like the previous methods.
14

---

Under review as a conference paper at ICLR 2023
Table 5: Analysis of Validity and Coverage of DiCE and Peasible-VAB given the original Output and
its Reverted one-hot representation. Bold highlights severe drops in performance. Higher is better.
Method
Version
German Credit
Graduate Admission
Student Performance
SBA
Validity ($)1
Output Reverted
100.00
100.00
100.00
100.00
DICE
58.20
91.28
98.12
98.93
Oulput Reverted
100.00
100.00
100.00
96.59
F-VAE
100.0
99,95
99.33
57.85
Coverage
($)
Oulput 100.00
100.00
100.00
100.00
DICE
Reverted
99.90
100.00
100.00
100.00
Output 100.00
100.00
100.00
97.33
F-VAR
Reverted
100.00
100.00
100.00
72.10
Admission
Performance
SBA
Inference Time/Sample (s)
0.22
Harmonic Mean 1%))
Diversity (%6
50
0.23
0.10
80
23-
0.3
0.3
0.6
0.7
2.0
0.4
0.7
3.0
0.3
2.4
15
Sparsity Threshold
Sparsity Threshold
Sparsity Threshold
Figure 4: Effect of various sparsity thresholds on (left) time efficiency, measured via average inference time per sample (in seconds), (middle) the "balanced score" Harmonic mean (Diversity, Sparsity) and (right) diversity. The current levels reported in the main paper are denoted as square markers.
B.4 ROLE OF THE FEATURE SELECTOR
Here we validate the importance of learning the local feature-based selection distribution via the feature selector S. In this regard, try removing S from L.2C framework and replace the probability vector π (2)) with a binary mask vector m E [0, 1| where m, = if E (i.e., a mutable otherwise. We thus use m; in substitution of s; to update the counterfactual representations ž as previously done. We now only optimize the generator G to learn the feature- based perturbation distribution, and the training objective Eq. (3) excludes the regularization term for sparsity accordingly. Table 6 inveestiigttss the performance of L2C under this alternative setup, in comparison with the proposed method that jointly optimizes S and L2C now still achieves 100% of Va lidityyyyyyyy and Coverage, so we only report the relevant metrics that remain.
feature) and m,
In this study, we do not impose any sparsity threshold because without the Selector S, we lose the flexibility in tailoring the quality of counterfactual generations to potential user preferences. This is the first drawback. Second, though taking more time to infer explanations, the selector introduces significant sparsity to gain effective balance of the trade-off against diversity. Furthermore, these results support our claim about the role of the generator & in that the perturbation distribution alone yield impressively diverse explanations with sparsity remaining under the required maximum level. This a benefit of learning an entire feature distribution in which sometimes the feature sample falls into the original input value i.e., Σ, z; while combining adequately with other features.
15

---

Under review as a conference paper at ICLR 2023
Table 6: Analysis of L2C capacity to balance the trade-off between diversity and sparsity when optimized without the selector S. Lower is better. 1 Higher is better. Bold indicates the best performance for each dataset. Time records total inference time in seconds.
Method
Prox Num
Prox
Diversity Sparsity Harmonic (0)
Time (s)
Cat (8)
(8)
Mean (8)
German Credit (Logistic Regression) Max. Sparsity: 80.00%
Without Selector Proposed method
1.41
55.84
52.66
62.47
43.83
0.82
29.56
42.73
33.13
52.14
13
Graduate Admission (Neural Network) Max. Sparsity: 85.71%
Without Selector Proposed method
1.54
58.67
48.71
76.38
31.81
1.25
31.85
41.51
47.79
46.25
Student Performance (Logistic Regression) Max. Sparsity: 71.43%
Without Selector Proposed method
1.37
46.32
41.03
54.52
43.14
1.14
32.93
40.97
43.43
47.52
SBA (Neural Network) Max. Sparsity: 75.00%
Without Selector 2.89
12.74
36,79
70.91
32.49
15
Proposed method
1.77
6.26
27.90
37.64
38.55
30
C BASELINE EXPERIMENT
We tune the base generative models of amortized baselines under various different settings. We determine the best settings via metrics: Cove rage and Divers ity. When there a trade-off, Coverage is chosen to be the deciding criterion.
For VeA MMahajan al. 2019), tune the hidden dimensions of the VAE encoder within {10, 30, 50, 70, 90) and regularization term on Validity within (42, 62, 82, 102, 122]. For CRUDS (Downs al. 2020). the base model Conditional Subspace Variational Auto-encoder (Klys et al. 2018). In the original paper, the network only has 1 hidden layer of 64 nodes, which we find to be of low capacity. We thus experiment with layers and different hidden dimensions within {16, 32, 64). For FastAR (Verma al., 2022), the hyper-pane paper-pprraeters include manifold distance À and entropy loss coefficient. Across datasets, Verma al. (2022) shows the best Coverage under X 0.1. We thus set X = 0.1 as well in our experiments, while focus on tuning the latter hyper-parameter within {0.01, 0.05, 0.1, 0.5, 1.0]. For the remaining hyper-parameters we adopt the best values reported by the authors. Table 7 reports the best settings of these methods on each dataset.
FastAR Specifies. A pre-trained FastAR model can only interpret one decision outcome chosen the desired one (often the positive label). We must therefore train separate FastAR models on the positive and negative subsets and combine the results. We further find that although it is straightforward obtain multiple generations one optimal counterfactual state for a given input. Thus in the hope of achieving better diversity, we train 100 different model initializations and accordingly collect a set of 100 explanations for evaluation. The inference time accumulates result, which is the reason why our reported results on time efficiency for FastAR are different from what are reported in the authors' paper.
single model, FastAR algorithm is optimized for
Table The best hyper-parameter settings for amortized methods.
Method
Hyper-parameter
German Credit
Graduate Admission
Student Performance
SBA
Encoder hidden size
10
70
F-VAE
Validity coefficient
42
62
62
42
Layer hidden size
16
CRUDS
Layer 2 hidden size
16
16
16
16
FastAR
Entropy coefficient
0.01
0.01
1.0
16

---

Under review as a conference paper at ICLR 2023
Table 8: Discretization of numerical features in each dataset.
Feature
Bucket values
No. Data points
German Credit
(4, 122
Duration (Months)
24.24]
(24, 72
230
( (248 1570
334
Credit Amount
(1,570, 3,410
3,,10 18,425
[9, 28
334
Age
(28 38
75
Graduate Admission
(290, 312] ]
189
GRE score
(312. 323
(323, 340
162
(92, 104
176
TOFFI. score
[004, 110]
175
[110. 120
149
( (7.8) 8
170
Undergraduate GPA
( (8,9) 9]
163
19. 9.10
Student Pertormance
[5, 16]
289
Age
[16,17] 17
179
[17 22
(-0 [] (4.32)
466
School absences
183
(-0, 10] (10.13]
252
First period grade
245
(13,19)
152
(-0. 10
228
[10,
103
Second period grade
(31,13]
166
[13, 19
152
SBA
(-0. 60
575
( [60 84
671
Loan term in months
( (84 240
544
(240. 306
312
(-0, 21
842
(2,3]
221
No. employees
(3.8) (8, 650 ]
540
499
[-0,2] (2. 130]
1658
No. jobs created
444
(-0. 1] ( [1,41
881
No. jobs retained
619
( (4,
602
(-107 15,074
1576
Charged amount
(15,074, 1,509,538
526
[.2660 30,026
588
30,026, 60,946
463
Gross amount approved
66.946. 300,056 ]
537
(300,056. 2,350,015
514
2,,922 14,943
550
SBA's guaranteed amount approved
14,943 41 41,541
501
(41,541 239,682
525
239,682, 2.115,002
526
(0.30, 0.50
1052
Proportion of gross amount
(0.50, 0.75
458
guaranteed by SBA
(0.75, 1.00
592
17

---

Under review as a conference paper at ICLR 2023
D QUALITATIVE EXAMPLES
Here we illustrate some examples of our generated counterfactuals for each dataset. As discussed, we report the discretized values for numerical features, since we believe it provides more useful and flexible suggestions to human users. For illustration purpose only, we round the edge values of the numerical intervals the nearest whole number. Immutable features are italicized.
Table 9: Counterfactual examples from German Credit dataset. *DM: Deutsche Mark
Original input
German Credit
Counterfactuals (Good credit risk)
(Bad credit risk)
Duration (months)
12
Credit amount (DM)
3,410 18,425
1,570 3,410
1.570- 3,410
1,570- 3,,10
3,410
18.425
1,570 3,410
Age
28 38
28 38
75
38
28 38
Checking (DM)
No
No
200+
200+
Credit history
Paid back duly
Paid back duly
Paid back duly
Paid back duly
Paid back duiy
Paid back duly
Purpose
Retraining
Retraining
Retruining
Retraining
Retraining
Retraining
Savings account (DM)
Under 100
1,000+
Under 100
500 1,000
100 500
Under 100
Present employment
7+ years
4 7 years
Unemployed
4 7 years
7+ years
4 years
Installment
Under
Under
25
Female
Female
Female
Female
Female
Female
Personal status
not single
not single
not single
not single
not single
not single
Other debtors
Co-applcaant
Cp-appiiannt
guarantor
Co Co-applicant
apppliain
None
Present residence
7+ years
7+ years
7+ years
7+ years
Under year
Under year
Property
None
Car other
None
None
None
None
Other installments
None
bank
None
None
None
None
Housing
No. existing credits
4-5
4-5
Job
Skilled
Skilled
Unemplayed
Unemployed
Skilled
Skilled
No. people being liable
0-2
0-2
0-2
0-2
Telephone
No
No
Yes
No
Yes
Yes
Foreign
Table 10: Counterfactual examples from Graduate Admission dataset.
Original input (Low chance)
Graduate Admission
Counterfactuals (High chance)
GRE Score
312
290
312
312 323
290 312
290 312
290 312
TOEFL Score
92 104
104 110
104
110
110
120
110 120
92
104
Undergraduate GPA
7-8
8-9
10
9 10
8-9
9 10
University Rating
2/5
2/5
2/5
2/5
2/5
2/5
Statement of Purpose
2/5
2/5
5/5
2/5
2/5
2/5
Letter of Recommendation
2.555
4/5
2.5/5
2.555
2.5/5
2/5
Research Experience
No
No
No
Yes
No
Yes
18

---

Under review as a conference paper at ICLR 2023
Table 11: Counterfactual examples from Student Performance dataset.
Original input (Fail)
Student Performance
Counterfactuals (Pass)
Age
16
16- 17
16- 17
16 17
16- 17
15-
School absences
0-4
0-4
4-
0-4
4-
First period grade
10-13
10 13
10
10- 13
10-13
Second period grade
0-10
13- 19
13- 19
13 19
13 19
13 19
Mother's education
Primary
Primary
Primary
Primary
Primary
Primary
Father's 's education
None
None
None
None
None
None
Weekly study time
2 5 hours
2 5 hours
2 5 hours
2 5 hours
5 10 hours
2 2-5 hours
Family educational support
No
No
No
No
Wanting higher education
Yes
Yes
Internet access at home In a romantic relationship
Yes
No
Yes
No
Yes
Yes
No
No
No
Yes
Yes
Free time Going out frequency Health status
Moderate
High
Moderate
Moderate
Moderate
Low
Very often
Very often
Very often
Rarely
Very often
Often
Good
Good
Good
Very good
Good
Good
Table 12: Counterfactual examples from SBA dataset.
Original input (Low default risk)
SBA
Counterfactuals (High default risk)
Loan in months
60 84
60 84
60 84
60
84-240
60 84
No. employees
3-
3-8
-
3
8- 650
8 650
No. jobs created
0-2
0-2
2 130
0-2
130
0-2
No. jobs retained
0
4 535
0-1
I
0-1
4 535
Churged off amount
Above 15K
Above
Gross upproved
61K 300K
61K 300K
4K 30K
30K 61K
61K 300K
30K 61K
Approved guaranteed amount
42K - 240K
42K 240K
42K 240K
42K 240K
15K 42K
42K 240K
% gross amount guaranteed
75 100
75 100
75 - 100
50 75
50 -0-33 75
75 100
From Urban or Rural
Urbam
Urbun
Urbun
Urban
Urban
Urban
Existing business
Loan backed by real estate
No
No
No
No
No
Active loan in recession
No
No
No
No
No
No
19


Reviews:
[{"review_id": "o-ItBoHGc7", "reviewer": "Reviewer_DTNp", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a method to generate counterfactual explanations based on two components: (i) a feature selector and (ii) and an end-to-end network. While (i) makes sure that relevant features are selected, (ii) enables the model to enforce additional constraints (e.g., dealing with categorical variables).", "summary_of_the_review": "Overall, the work lacks novelty and originality as well as critical discussions around limitations and shortcomings.", "strength_and_weaknesses": "Strengths:\n1. The addressed problem is relevant, timely and interesting to the XAI community.\n2. The paper is fairly well written.\n3. The methods’ evaluation is extensive.\n\nWeaknesses:\n1. The overview of related work as well as the contextualization of the proposed solution with respect to related work are weak.\n2. The paper accumulates many ideas from previous works on counterfactual explanations; various aspects discussed in the paper are not novel and have already been proposed in previous papers:\n\n(i) Amortized inference has been used in the following works: \nGuo et al (2021),\nDan Ley et al (2022),\nVerma et al (2022).\n\n(ii) Dealing with heterogenous tabular data has been extensively studied in:\nKarimi et al (2020)\nPawelczyk et al (2020) - Sampling categorical & Bernoulli data have also been used by Pawelczyk et al in their auto-encoder \nPawelczyk et al (2022) - Dealing with categorical variables.\n\n(iii) Dealing with diverse counterfactuals was initially studied in Mothilal et al., (2020)\n\n3. The discussion does not really critically address limitations or related improvement strategies. E.g.: What are possible extension strategies for handling domain constraints or expert knowledge? Do you see any tensions between diversity and privacy?...\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "hJ3VPKIQ3X", "reviewer": "Reviewer_nrfX", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Counterfactual generation is a multi-model problem in the sense that satisfying all the constraints simultaneously is considered a challenging task. In this paper, the authors propose a stochastic feature-based learning approach to meet all the constraints. The proposed method discretizes each continuous feature and learns the generator and selector to generate diverse counterfactual samples. The generator learns a categorical distribution for feature-based perturbation. The selector serves to select feature-based selection distribution. They are multi-nomial and Bernoulli distributions, respectively. And they are both parameterized by a neural network. The loss is built based on the cross-entropy loss from the prediction and the target counterfactual class and an L1 norm that encourages sparsity. The proposed method are  compared with the most popular methods on 4 popular tabular data sets.", "summary_of_the_review": "The paper proposes a novel method that can generate diverse counterfactual samples for understanding a pre-trained black-box classifier. The proposed method introduces two neural networks, a generator and a selector, that take the discretized input space. Although the discretization of the input space looks novel to me, the novelty is limited. The loss function that trains the generator and selector takes cross-entropy loss between the predicted label and target class label, and the L1 norm, which is not new at all.  ", "strength_and_weaknesses": "Strength:\n1) The proposed method addresses multiple constraints that counterfactual generation generally faces at the same time. \n2) The optimization is amortized by two neural networks. \n3) The evaluation metrics include various aspects of counterfactual generations. The results support the claim overall. \n4) The discretization of continuous features looks interesting. \n\nWeaknesses:\n1) The metric, Manifold distance, is only measuring L1 distance among the nearest samples. I think one of the most important plausibility measurements is how much overlap the counterfactual sample lies in the target class in terms of distribution distance. L1 or L2 distance seems limited in regard to measuring plausibility. \n2) The discretization of numerical features seems interesting to me. However, the trade-off of using discretization is unclear. In particular, how do we choose the level c_i? Does the choice of c_i affect the metrics? And are those comparisons evaluated in the discretized space or the original space?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ah6AE4DaLN", "reviewer": "Reviewer_67mR", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The goal of the paper is to generate counterfactual explanation (CFs) of classification models. The paper points our several problems with existing CF generation methods, many of which are rooted in the fact that these methods are not able to handle categorical features particularly well. The previous methods also work in a posthoc manner which affects their runtime performance. The paper aims to solve these problems by training the CF generator. The crucial design choice is to discretize all feature as categoricals and learn their distributions. Usage of softmax ensures that the post-processing of one-hot encoded features does not lead to invalid CFs. The paper also shows experiments with several datasets showing better performance than existing methods.", "summary_of_the_review": "Overall, the paper uses intriguing ideas to get around various issues for generating CFs. However, many choices seem adhoc and could be better justified. It is also not clear how well the setup would work with high dimensional datasets where discretization might lead to very large number of features. ", "strength_and_weaknesses": "**Strengths:** Usage of softmax to ensure validity of one hot encoded features is a neat idea.\n\n**Weaknesses:** Overall, the proposed methodology does not seem to solve the issues with plausibility. I also think some choices seem quite adhoc and need better justification. Please see detailed comments below:\n\n1. As far as I could see, the distribution of each categorical feature is being learnt independently which may lead to unrealistic combinations (for instance, very young people with decades of work experience). The paper does not discuss how this issue should be resolved.\n\n2. It is also not clear if discretizing numerical features is a very good idea. How would this scale to high dimensional datasets? How should the number of discretization buckets be chosen?\n\n3. On a related note, discretized numerical features are converted back to the original representation by assigning them the middle value. Does this lead to any accuracy issues (since the precise feature value is lost)?\n\n4. The usage of Bernoulli is an interesting idea but it also adds a few more knobs (every feature $i$ has a corresponding $\\pi_i$). How could one handle sparsity at the scale of all the features?\n\n5. The experimental setup makes a few assumptions that seem a bit questionable. I am not sure why selecting a random perturbation method (DiCE Random) is a good strategy. Similarly, assigning only 3 seconds for a single CF generation seems a bit unfair to posthoc CF generation methods. It is also not very clear to me why numerical and categorical plausibility is considered separately in Table 1 when there might be dependencies between them.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "BkHlqySo6j", "reviewer": "Reviewer_Xqce", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a method to generate counterfactual explanations that are *valid, actionable, sparse, diverse, plausible,* and computationally *scalable*. They achieve this by learning an amortized local counterfactual distribution conditioned on the original data point. To learn it, they train a categorical counterfactual generator from which they sample the attributes to be modified with a selector that models a Bernoulli distribution. To enforce sparsity, they regularize the L1 norm of the selector. The proposed method achieves high validity and diversity rates compared with previous state of the art while having a relatively low computational cost.", "summary_of_the_review": "The method is sound and the text is clear. My main concern is whether the author's contribution is significant enough for ICLR, I would appreciate if the authors could comment on that. The submission could also be improved by adding implementation details and a discussion about the limitations of this work. Thus, I temporarily recommend a score of 5 until these points are discussed.", "strength_and_weaknesses": "Strengths\n=======\n* The proposed method is sound\n* The paper is clear and very well written, I really liked Table 1\n* The computational cost is small\n* I liked the OOD datasets experiment\n* The authors provide the code\n\nWeaknesses\n=========\n* The diversity measure is ill-defined (here and in previous state of the art). For instance, imagine an instance defined by N attributes and for which changing attribute number 1 generates a counterfactual. Now we can generate an arbitrary number of counterfactuals by modifying attribute number 1 and any other attribute (to increase hamming distance). However all these counterfactuals provide the same information (that the classifier is sensitive to attribute 1). I suggest you add some comment about this in the paper.\n* From the paper, it is difficult to obtain all the details to reproduce their method. For instance, how is the selector instantiated? Is it just a sigmoid on top of a neural network? How is that neural network defined? Could you add implementation details (in general)?\n* Research-wise, this work does not introduce any new ideas (previous ideas are properly cited), this work is rather on the application side.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject