CounterNet: End-to-End Training of Prediction Aware Counterfactual Explanations
Authors: Hangzhi Guo
Thanh Hong Nguyen
Amulya Yadav
Venue: Submitted to ICLR 2023 (2023)
Keywords: Counterfactual Explanation
Algorithmic Recourse
Explainable AI
Interpretability

Abstract: Counterfactual (or CF) explanations are a type of local explanations for Machine Learning (ML) model predictions, which offer a contrastive case as an explanation by finding the smallest changes (in feature space) to the input data point, which will lead to a different prediction by the ML model. Existing CF explanation techniques suffer from two major limitations: (i) all of them are post-hoc methods designed for use with proprietary ML models --- as a result, their procedure for generating CF explanations is uninformed by the training of the ML model, which leads to misalignment between model predictions and explanations; and (ii) most of them rely on solving separate time-intensive optimization problems to find CF explanations for each input data point (which negatively impacts their runtime). This work makes a novel departure from the prevalent post-hoc paradigm (of generating CF explanations) by presenting CounterNet, an end-to-end learning framework which integrates predictive model training and the generation of counterfactual (CF) explanations into a single pipeline. We adopt a block-wise coordinate descent procedure which helps in effectively training CounterNet's network. Our extensive experiments on multiple real-world datasets show that CounterNet generates high-quality predictions, and consistently achieves 100% CF validity and very low proximity scores (thereby achieving a well-balanced cost-invalidity trade-off) for any new input instance, and runs 3X faster than existing state-of-the-art baselines. 


Under review as a conference paper at ICLR 2023
COUNTERNET: END-TO-END TRAINING OF PREDIC- TION AWARE COUNTERFACTUAL EXPLANATIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Counterfactual (or CF) explanations are a type of local explanations for Machine Learning (ML) model predictions, which offer contrastive case as explanation by finding the smallest changes (in feature space) to the input data point, which will lead to a different prediction by the ML model. Existing CF explanation techniques suffer from two major limitations: (i) all of them are post-hoc methods designed for use with proprietary ML models CF explanations is uninformed by the training of the MI. model, which leads to misalignment between model predictions and explanations; and (ii) most of them rely on solving separate litenintensive optimization problems to find CF expla- nations for each input data point (which negatively impacts their runtime). This work makes a novel departure from the prevalent post-hoc paradigm (of generating CF explanations) by presenting CounterNett, an end-to-end learning framework which integrates predictive model training and the generation of counterfactual (CF) explanations into a single pipeline. We adopt a block-wise coordinate descent procedure which helps in effectively training CounterNet's ' network. Our extensive experiments on multiple real-world datasets show that CounterNet generates high- quality predictions, and consistently achieves 100% CF validity and low proximity scores (thereby achieving a well-balanced cost-invalidity trade-off) for any new input instance, and runs 3X faster than existing state-of-the-an baselines.
result, their procedure for generating
INTRODUCTION
counterfactual (CF) explanation offers a contrastive case - to explain the prediction made by a Machine Learning (ML) model data point x: CF explanation methods find new counterfactual example z', which is similar to x but gets a different (or opposite) prediction from the ML model. From end-user perspective, CF explanation methods' (Wachter al., 2017) may be more preferable (as compared to other methods of explaining ML models), as they can be used to offer recourse to vulnerable groups. For example, if person applies for loan and gets rejected by bank's 's ML algorithm, CF explanation methods can suggest corrective measures to the loan applicant, which can be incorporated future loan application to improve their chances of getting an approved loan.
Generating high-quality CF explanations is challenging problem because of the need to balance the cost-ine cost-invalidity trade-off (Rawal al., 2020) between: (i) the invalidity, i.e., the probability that a CF example is invalid, or it does not achieve the desired (or opposite) prediction from the ML model; and (ii) the cost of change, i.e., the L1 norm distance between input instance and CF example x'. Figure 1 illustrates this trade-off by showing three different CF examples for an input instance If invalidity is ignored (and optimized only for cost of change), the generated CF example can be trivially set to itself. Conversely, if cost of change is ignored (and optimized only for invalidity), the generated CF example can be set to x/2 (or any sufficiently distanced instance with different labels). More generally, CF examples with high (low) invalidities usually imply low (high) cost of change. To optimally balance this trade-off, it is critical for CF explanation methods to have access to the
'F explanations are closely related to algorithmic recourse (Ustun et al., 2019) and contrastive explanations (Dhurandhar et al., 2018). Although these terms proposed under different contexts, their differences from CF explanations have been blurred (Verma al., 2020; Stepin al., 2021), i.c. these terms used interchangeably.

---

Under review as a conference paper at ICLR 2023
decision boundary of the ML model, without which finding near-optimal CF explanation (i.e., x1) difficult. For example, it is difficult to distinguish between a (a valid CF example) and z/g (an invalid CF example) without prior knowledge of the decision boundary.
Existing CF explanation methods suffer from three major limitations. First, to best knowledge, all prior methods belong to the post-hoc explanation paradigm, i.e., they assume a trained black-box ML model input. This post-hoc assumption has certain advantages, e.g., post-hoc explanation techniques are often agnostic to the particu- lars of the ML model, and hence, they are generalizable enough to interpret any third-party proprietary MI. model, However, we argue that in many real-world scenarios, the -adel-agnostic approach provided by post-hoe CF expla- nation methods is not desirable. With the advent of data regulations that enshrine the "Right to Explanation" (e.g., EU-GDPR (Wachter et al., 2017)), service providers quired by law to communicate both the decision outcome (i.e., the ML model's 's prediction) and its actionable impli- cations (i.e., a CF explanation for this prediction) to an end- user. In these scenarios, the post-hoc assumption is overly limiting, as service providers can build specialized CF explanation techniques that can leverage the knowledge of their particular ML model to generate higher- quality CF explanations. Second, the post-hoc CF explanation paradigm, the optimization procedure that finds CF explanations is completely uninformed by the ML model training procedure (and the resulting decision boundary). Consequently, such post- hoc procedure does not properly balance the cost-invalidity trade-off (as explained above), causing shortcomings the quality of the generated CF explanations (as shown in Section 4). Finally, most CF explanation methods very slow they search for CF examples by solving separate time-intensive optimization problem for each input instance (Wachter et al., 2017; Mothilal et al., 2020; Karimi et al., 2021), which is not viable in time-constrained environment, the runtime is critical factor when deployed to end-user facing devices (Zhao et al., 2018; Arapakis et al., 2021).
N
Low Invalid OF
Medium Cost Valld
Cost Valid CF
Decision Boundary
Figure Illustration of the cost- invalidity trade-off in CF explanations for binary classification problems.
Contributions. We make a novel departure from the prevalent post-hoc paradigm of generating CF explanations by proposing CounterNet, a learning framework that combines the training of the ML model and the generation of corresponding CF explanations into single end-to-end pipeline (i.e.. from input to prediction to explanation). CounterNet has three contributions:
Unlike post-hoc approaches (where CF explanations are generated after the MI. model is trained), CounterNet uses (neural network) model-based CF generation method, enabling the joint training of its CF generator network and its predictor network. At a high level, CounterNet's CF generator network takes as input the learned representations from its predictor network, which is jointly trained along with the CF generator. This joint training key to achieving a well-balanced cost- cost-invalidity trade-off (as we show in Section 4).
We theoretically analyze CounterNet's objective function to show two key challenges in training CounterNet: (i) poor convergence of learning: and (ii) lack of robustness against adversarial examples. To remedy these issues, we propose a novel block-wise coordinate descent procedure.
We conduct extensive experiments which show that CounterNet generates CF explanations with ~100% validity and low cost of change (~~9.8% improvement to baselines), which shows that CounterNet balances the cost-invalidity trade-off significantly better than baseline approaches. In addition, this joint-training procedure does not sacrifice CounterNet's 's predictive accuracy and robustness. Finally, CounterNet runs orders of magnitude (~3X) faster than baselines.
RELATED WORK
Prior explanation techniques for ML models include LIME (Ribeiro et al., 2016), SHAP (Lundberg Lee, 2017). saliency maps (Selvaraju et al., 2017; Sundararajan et al., 2017; Smilkov et al., 2017), which highlight attribution importance for each data instance. Further, case-based methods provide
2

---

Under review as a conference paper at ICLR 2023
(similar) data samples as model explanations (Guidotti et al., 2018: Molnar et al., 2020; Chen et al., 2019; Koh & Liang, 2017). Our work is most closely related to prior literature on CF explanation methods, which focuses finding new instances that lead to different predicted outcomes (Wachter et al. 2017; Verma et al., 2020; Karimi et al., 2020; Stepin et al., 2021). CF explanations are preferred by human end-users as these explanations provide actionable recourse in many domains (Binns et al., 2018; Miller, 2019; Bhatt et al., 2020). Almost all prior work in this area belongs to the post-hoc CF explanation paradigm, which we categorize into non-parametric and parametric methods.
Non-parametria methods. Non-parametric methods aim to find a counterfactual explanation without the use of parameterized models. Wachter et al. (2017) proposed VanillaCF which generates CF explanations by minimizing the distance between the input instance and the CF example, while pushing the new prediction towards the desired class. Other algorithms, built on top of VanillaCF, optimize other aspects, such recourse cost (Ustun et al., 2019), fairness ( (Von Kügelgen et al., 2022), diversity (Mothilal et al., 2020), closeness the data manifold (Van Looveren & Klaise, 2019), causal constraints (Karimi et al., 2021), uncertainty (Schut et al., 2021), and robustness to model shift (Upadhyay et al., 2021). I). However, this line of work is inherently post-hoc and relies on solving separate optimization problem for each input instance. Consequently, running them is consuconnu,,ng, and their post-hoe nature leads to poor balancing of the cost- invalidity trade-off.
Parametric Methods. These methods use parametric models (e.g.. neural network model) to generate CF explanations. For example, Pawelczyk et al. (2020); Joshi et al. (2019) generate CF explanations by perturbing the latent variable of variational autoencoder (VAE) model. Yang et al. (2021); Singla et al. (2020); Nemirovsky et al. (2022) and Mahajan et al. (2019); Rodríguez et al. (2021); Guyomard et al. (2022) train generative models (GAN and VAE, respectively) to produce CF explanations for a trained ML model. However, these methods are still post-hoc in nature, and thus, they also suffer from poorly balanced cost-invalidity trade-offs. Contrastingly, we depart from this post-hoc paradigm, which leads to a greater aligmment between CounterNet's predictions and CF explanations, Note that Ross et al. (2021) propose cccourse-rriendyy MIL model by integrating recourse training during predictive model training. However, their work does not focus on generating CF explanations. In contrast, we focus on generating predictions and CF explanations simultancously.
3 THE PROPOSED FRAMEWORK: COUNTERNET
Unlike prior work, our proposed framework CounterNet relies on a novel integrated architecture which combines predictive model training and counterfactual explanation generation into a single optimization framework. Through this integration, we can simultaneously optimize the accuracy of the trained predictive model and the quality of the generated counterfactual explanations. Formally, given an input instance x E Rdd, CounterNet aims to generate two outputs: (i) the MI. prediction component outputs a prediction dz for input instance æ; and (ii) the CF explanation generation component produces a CF example x' E gd as explanation for input instance x. Ideally, the CF example x' should get a different (and often more preferable) prediction fa', as compared to the prediction j. on the original input instance (i.e., Oz' * (u). In particular, if the desired prediction output is binary-valued (0, 1), then da and ÿa should take on opposite values (i.e., Dn + Ux 1).
3.1 NETWORK ARCHITECTURE
Figure 2 illustrates CounterNet's architecture which includes three components: (i) an encoder network lu(-): (ii) predictor network ʃ()); and (iii) CF generator network g(-). During training, each input instance x E Rd is first passed through the encoder network to generate a dense latent vector representation of x (denoted by 2g = h(z)). Then, this latent representation is passed through both the predictor network and the CF generator network. The predictor network outputs softmax representation of the prediction 1-- = fzzz). To generate CF examples, the CF generator network takes two pieces of information: (i) the final representation of the predictor network Pz (before it is passed through the softmax layer), and (ii) the latent vector 2n (which contains a dense representation of the input zz). These two vectors are concatenated to produce the final latent vector 2 P2 ① Zz. which is passed through the CF generator network to produce a CF example x' = =2(2)) Note that passing the representation of predictor network Pa through the CF generator network implicitly conveys information about the decision boundary to the CF generation procedure, who leverages this knowledge find high-quality CF examples x!.
3

---

Under review as a conference paper at ICLR 2023
Neaal Imanerratio
Capy Block
Predictor sO
Trankke to Chocko Erooling
1-3
Selanm faxiie deving moking and
90
Black oxecanmiis
Encoder
h()
CF Generator
8()
Pxx
00%
Xx
Figure CounterNet contains three components: an encoder to transform the input into a dense latent vector, predictor network to output the prediction, and CF generator to produce explanations.
Furthermore, to ensure that the CF generator network outputs valid CF examples (i.e., Dh * Dee) the output of the CF generator network z' is also passed back as an input through the encoder and predictor networks when training CounterNet. This additional feedback loop (from the output of CF generator network back into the encoder and predictor networks) optimizes the validity of generated CF examples. As such, we can now train the entire network in way such that the predictor network outputs opposite predictions Ie and d2' for the input instance and the CF example x', respectively. Note that this "feedback loop" connection is only needed during training, and is removed at test time. This design aims to achieve better balance on the cost-invalidity tradeoff (as shown Section 4).
Design of Encoder, Predictor & CF Generator. All three components in CounterNet's architecture consist of a multi-layer perception (MLP)". The encoder network in CounterNet consists of two werd-forward layers that down-sample to generate latent vector R (s.t. k < d). The predictor network passes this latent vector z through two feed-forwart layers to produce the predictor representation p. Finally, the predictor network outputs the probability distribution with fully-connected layer followed by softmax layer. On the other hand, the CF generator network takes the final latent representation z' @ p as an input, and up-samples (using two feed-forward layers) to produce CF examples x' E Rd. Each red-ffoward neural network layer inside CounterNet uses LeakyRelu activation functions (Xu et al., 2015) followed by a dropout layer (Srivastava et al., 2014). Note that the number of feed-forward layers, the choice of activation function, etc., were hyperparameters that were optimized using grid search. See Appendix B.4 for implementation details.
predictions
Handling Categorical Features. To handle categorical features, we customize CounterNet's archi- tecture for each dataset. First, we transform all categorical features in each dataset into numeric features via one-hot encoding. In addition, for each categorical feature, we add softmax layer after the final output layer in the CF generator network (Figure 2), which ensures that the generated CF examples respect the one-hot encoding format (as the output of the softmax layer will sum up 1). Finally, we normalize all continuous features to the [0, 1] range before training.
3.2 COUNTERNET OBJECTIVE FUNCTION
CounterNet has three objectives: (i) predictive accuracy- the predictor network should output accurate predictions U.: (ii) counterfactual validity- CF examples x' produced by the CF generator network should be valid, i.c., they get opposite predictions from the predictor network (e.g. fe + De 1); and (iii) minimizing cost of change minimal modifications should be required change input instance
CounterNet work with altemate neuronal blocks, e.g., convolution, attention, although effective training of these neuronal blocks demands additional efforts (see Appendix H).
4

---

Under review as a conference paper at ICLR 2023
to CF example z'. Thus, we formulate this multi-objective minimization problem to optimize the parameter overall network A:
argmin
1 N ==1
Σ
A1
(yI- Dz,)
+ A2
(I.. (1- (x:)) + A3- (I; - 2
(1)
Prediction (E1)
Validity Loss (Ca)
Change Loss (Ca)
where N denotes the number of instances in our dataset, (Y1, A2, Xa) are hyper-parameters to balance the three loss components, the prediction loss Li denotes the mean squared error (MSE) between the actual and the predicted labels (μı and 9s. on instance I₁, respectively), which aims to maximize predictive accuracy. Similarly, the validity loss L2 denotes the MSE between the prediction on instance x; (i.e., Vs;), and the opposite of the prediction received by the corresponding CF example an (i.e., - V2:). Intuitively, minimizing L2 maximizes the validity of the generated CF example x' by ensuring that the predictions on z' and x; are different. Finally, the proximity loss La represents the MSE distance between input instance zi and the CF example xy,, which aims to minimize proximity, This choice of loss functions is crucial to CounterNet's s superior performance, as replacing L1, L2 and L3 with alternative functions leads to degraded performance (as we show in Section 4).
3.3 TRAINING PROCEDURE
The conventional way of solving the optimization problem in Eq. is to use gradient descent with backpropagation (BP). However, directly optimizing the objective function (Eq. as-is results in two fundamental issues: (1) poor convergence in training (shown in Lemma 3.1), and (2) proneness to adversarial examples (shown in Lemma 3.2).
Issue I: Poor Convergence. Optimizing Eq. 1 as-is via BP leads to poor convergence. This occurs because Eq. contains two different loss objectives with divergent gradients, Lemma 3.1 shows the gradients of L1 and L2 move in opposite directions. Consequently, the accumulated gradient across all three loss objectives fluctuates drastically, which leads difficulty in training.
Lemma 3.1 (Divergent Gradient Problem). Let L = |ly Úzllz and L2 = llú. - (1 -b if x' + xr < i < 1, y is a binary label, and 10x - yl < 0.5, then VL1 VL2 <.
Issue II: Adversarial Examples. Our training procedure should generate high-quality CF exam- ples x' for input instances x without sacrificing the adversarial robustness of the predictor network. Unfortunately, optimizing Eq. 1 as-is is at odds with the goal of achieving adversarial robustness. Lemma 3.2 shows that optimizing C2 with respect to the predictive weights oy decreases the robust- ness of the predictor f(-) (by increasing the Lipschitz constant of f() (Hein & Andriushchenko, 2017; Wu et al., 2021)), leading to its increased vulnerability to adversarial examples. Proof of Lemma 3.1 and 3.2 can be found in Appendix A.
Lemma 3.2 (Lipschitz Continuity). Suppose J is a locally Lipschitz continuous function param- eterized by 0, then it satisfies (ff(()) fo(x')l < K || 12'==a1] ='1l/. where its Lipschitz constant K- = suP=re3(z,e) (ll V (z (z)))) Let L2 = [lfe(x) - (1 fo(x')l12 if a + x. 0 < fe(.) < 1, f() + y, and is a binary label, then minimizing La w.r.t A increases the Lipschitz constant K.
We remedy these issues as follows: (1) to handle poor convergence in training, we adopt a block-wise coordinate descent procedure, which divides the problem of optimizing Eq. I into two parts: (i) optimizing predictive accuracy (primarily influenced by L1); and (ii) optimizing the validity and proximity of CF generation (primarily influenced by Le and E3). Specifically, for each mini-batch of m data points (ar(() pm we apply two gradient updates to the network through hackpropagation.
For the first update, we compute 1(1) = g(0) - V (>1 L1), and for the second update, we compute ((2) = (1) Vocn L2 + Ag Lz). This procedure ensures that gradients for C, and C2 are calculated separately, which lessens the divergent gradient problem (Lemma 3.1), and leads to significantly better convergence. (2) Moreover, to improve adversarial robustness of our predictor network. during the second stage of our coordinate descent procedure (when we optimize for X2 L2 + λa L2)), we only update the weights in the CF generator 0,, and freeze gradient updates in both the encoder fn and predictor A, networks. More formally. instead of updating the weights 0 of the entire network during the second update, only update the CF generator weights fg as
follows: 0g²) = 0ft) V Vogu (A2 L2 + A3- L3). This ensures that the Lipschitz constant of the predictor network does not increase (Lemma 3.2).
5

---

Under review as a conference paper at ICLR 2023
4 EXPERIMENTAL EVALUATION
We primarily focus our evaluation on heterogeneous tabular datasets for binary classification problems (which is the most common and reasonable setting for CF explanations (Verma et al., 2020; Stepin et al., 2021)). However, CounterNet be applied to multi-class classification settings, and it can also be adapted to work with other modalities of data, e.g., images, etc. (as shown in Appendix G, I).
Baselines. We compare CounterNet against seven state-of- tfte-of-the-art CF explanation methods: (i) VanillaCF (Wachter et al., 2017) - which generates CF examples by optimizing CF validity and proximity; (ii) DiverseCF (Mothilal al., 2020), ProtoCF (Van Looveren & Klaise, 2019), and UncertainCF (Schut et al., 2021) - which optimizes for diversity, consistency with prototypes, and uncertainty, respectively; (iii) VAE-CF (Mahajan al., 2019), CounteRGAN (Nemirovsky al., 2022), C-CHVAE (Pawelczyk et al., 2020), and VCNer (Guyomard et al., 2022) - which rely on generative models (i.e., VAE or GAN) to generate CF examples 3.
Unlike CounterNet, all of the post-hoc methods require a trained predictive model as input. Thus, for each dataset, we train a neural network model and use it as the target predictive model for all baselines. For fair comparison, we only keep the encoder and predictor network inside CounterNet's architecture (Figure 2), and optimize them for predictive accuracy alone (i.e., f1). This combination of encoder and predictor networks is then used as the black-box predictive model for our baselines.
Datasets. To remain consistent with prior work on CF explanations (Verma et al., 2020), we evaluate
CounterNet on four benchmarked real-world binary classification datasets: (i) Adult (Kohavi & Becker, 1996) which predicts whether an individual's 's income reaches $50K (Y-1) or not (Y=0); (ii) Credit (Ych & Lien, 2009) which uses historical payments to predict default of payment (2=1) or not (Y=0); (iii) HELOC (FICO, 2018) () which predicts if homeowner qualifies for credit (Y=1) or not
(Y=0); (iv) OULAD (Kuzilek al., 2017) which predicts whether MOOC students drop out (Y=1) or not (Y=0). We also provide experiments on four additional datasets in Appendix E.
Evaluation Metrics. For each imput x, CF explanation methods generate two outputs: (i) prediction úz: and (ii) a CF example x'. We evaluate the quality of both these outputs using separate metrics. For evaluating predictions, we use predictive accuracy (as all four datasets are fairly .alancdal..c.).....
For evaluating CF examples, we use five widely used metrics from prior literature (see Appendix B.3 for formal definitions): (i) Validity the fraction of input instances on which CF explanation methods output valid CF examples, i.e., the fraction of input data points for which fa + Da 1. High validity desirable, as implies the method's effectiveness at creating valid CF examples (Mothilal et al., 2020; Upadhyay et al., 2021). (ii) Proximity is the L1 norm distance between r and x' divided by the number of features (Wachter et al., 2017: Mothilal et al., 2020). (iii) Sparsity is the number of feature changes between and x' (normalized by the total number of features) (Wachter et al., 2017: Boursabzi-Sangder
al., 2021). Proximity and sparsity serve as proxies for measuring the cost of change of our CF explanation approach, it is desirable to have fewer modifications in the input space to convert into a valid CF example. (iv) Manifold distance the L1 distance to the k-nearest neighbor of x' (we use i- 1 based on (Verma et al., 2022)). Low manifold distance is desirable as closeness to the training data manifold indicates realistic CF explanations (Van Looveren & Klaise, 2019; Verma et al., 2022). (v) Finally, we also report the runtime for generating CF examples.
4.1 EVALUATION OF COUNTERNET PERFORMANCE
Predictive Accuracy. Table compares CounterNet's predictive accuracy against the base prediction model used by baselines. This table shows that CounterNet exhibits highly competitive predictive performance it achieves marginally better accuracy on the Credit dataset (row 2), and achieves marginally lower accuracy on the remaining datasets. Across all four datasets, the difference between the predictive accuracy of Counter- Net and the base model is ~ 0.1 %. Thus, the potential
Table I: Predictive Accuracy of CounterNet
Dataset
Base Model
CounterNet
Adult Credit HELOC
0.831
0.828
0.813
0.819
0.717
0.716
OULAD
0.934
0.929
Note that Yang et al. (2021) propose another parametric post-hoc method, but we exclude it in our baseline comparison because achieves comparahle performance C-CHVAR (as reported in (Yang al., 2021)).

---

Under review as a conference paper at ICLR 2023
Table 2: Evaluation of CF explanations: CounterNet achieves perfect validity (i.e., Val.), and incurs comparable (or lesser) cost of changes (i.e., Prox, Spar.) than baseline methods, with
comparable manifold distance (i.e., Man.). Bold and italicized cells highlight the best and second- best performing methods, respectively.
Method
Adult
Credit
HELOC
OULAD
Val.
Prox.
Spar.
Man.
Val. Prox.
Spar. Man.
Val.
Prox.
Spar.
Man.
Val.
Prox.
Spar.
Man.
VanillaCF
0.76
.202
_556
0.57
0.92
.123
.841
0.59
1.00
.154
.883
0.71
1.00
.101
.762
1.30
DiverseCF
0.54
.276
.662
1.16
1.00
.264
.918
1.68
0.90
.149
.434
1.34
0.68
.117
.565
2.51
ProtoCF
0.59
.250
.648
0.62
0.92
.197
.855
0.82
1.00
.168
.805
0.56
1.00
.107
.754
1.46
UncertainCF
0.36
.307
.713
1.23
0.62
.155
.217
0.80
0.55
.130
.161
0.94
0.59
.098
.734
2.23
C-CHVAE
1.00
.281
.721
0.94
1.00
.357
.853
1.85
1.00
.155
.790
0.81
1..00
.110
.797
2.11
VAB-CF
0.66
.287
.734
1.03
0.13
.201
.756
0.62
1.00
.221
.893
1.04
1.00
.115
.586
2.19
CounteRGAN
0,78
.327
.698
2.21
0.39
.260
.687
2.03
1.00
.271
.509
2.23
0.43
.087
.587
2.15
VCNet
1,00
.291
.755
0.19
1.00
.162
.939
0.16
1.00
.154
.786
0.39
1.00
.095
.903
1.33
CounterNet
1,00
.196
.644
0.64
1.00
.132
.912
0.56
1.00
.125
.740
0.56
|1.00
.075
.725
0.87
benefits achieved by CounterNet's joint training of predictor and CF generator networks do not come at a cost of reduced predictive accuracy.
Counterfactual Validity. Table compares the validity achieved by CounterNet and baselines all four datasets. We observe that CounterNet, C-CHVAE, and VCNet are the only three methods with 100% validity all datasets. With respect to the other baselines, CounterNet achieves 8% and 12.3% higher average validity (across all datasets) than VanillaCF and ProtoCF (our next best baselines).
Proximity & Sparsity. Table 2 compares the proximity/sparsity achieved by all methods. CounterNet achieves at least 3% better proximity than all other baselines on three out four datasets (Adult, HELOC, and OULAD), and it is the second best performing model the Credit dataset (where achieves 7.3% poorer proximity than VanillaCF). In terms of sparsity, CounterNet performs reasonably well, it is the second best performing model on the Adult and HELOC datasets even though CounterNet does not explicitly optimize for sparsity. This shows that CounterNet outperforms all baselines by generating CF examples with the highest validity and best proximity scores.
Dost-Invalidity Trade-off. We illustrate the cost-invaliding trade-off (Rawal et al., 2020) for all methods. Figure 3 shows that CounterNet lies the bottom left of this figure it consistently achieves the lowest invalidity and cost all four datasets. In comparison, VCNer achieves the same perfect validity, but the expense of ~44% higher cost than CounterNet. Similarly, C-CHVAE demands 7 1 % higher cost than CounterNet to achieve perfect validity. On the other hand, VanillaCF achieves comparable cost to CounterNet (10% higher cost), but achieves lower validity by 8%. This shows that CounterNet's joint training enables it to properly balance the cost-invalidity trade-off.
Table 3: Runtime comparison (in milliseconds). CounterNet outperforms all of baselines in runtime.
DiscseCF
Ivd
UheetaisCF
Method
Adult
Credit HELOC OULAD
C-BEVAE
VAL
VanillaCF
1432.09
1358.26
1340.42
1705.93
022
CounteeNi
DiverseCIF
4685.39
3898.43
3921.72
5478.17
ProtoCF
2348.21
2056.01
1956.71
2823.29
UncertainCF
379.95
60.80
7.91
6.81
040
C-CHVAE
3,28
568.28
2.68
4.79
10
0.1
0.30
0.50
VAE-CF
1.72
1.48
1.84
Proskoity
CounteRGAN
1.96
1.77
1.59
2,40
Figure 3: Illustration of the cost-invalidity trade-off across all four datasets. Methods the bottom left are preferable.
VCNet
1.39
1.23
1.13
1.81
CounterNet
0.64
0.39
0.44
0.79
7

---

Under review as a conference paper at ICLR 2023
Table 4: Ablation analysis of CounterNet. Each ablation leads to degraded performance, which in
tur, demonstrates the importance of different design choices inside CounterNet.
Ablation
Adult
Credit
HELOC
OULAD
Val.
Prox.
Val.
Prox.
Val.
Prox.
Val.
Prox.
CounterNet- BCC
0.86
.238
0.96
.210
0.86
.238
0.95
.101
CounterNet-S SingleeBP
0,64
.248
0.92
.251
0.93
.206
0.94
.110
CounterNct-Separrae
0.96
.257
0.99
.265
0.91
.161
0.94
.097
CounterNet-NoPass-2
0.97
.256
0.99
.339
0.98
.147
0.98
.101
CounterNet- Pesthho
1.00
.276
1.00
.247
1.00
.153
0.99
.099
CounterNet
1.00
.196
1.00
.132
1.00
.125
1.00
.075
Manifold Distance. Table 2 shows that CounterNet achieves the second-lowest manifold distance in average (right below VCNet, which explicitly optimizes for data manifold). In particular, CounterNet achieves the lowest manifold distance in OULAD, and is ranked second in Credit and HELOC. This result shows that CounterNet generates highly realistic CF examples that adhere to the data manifold.
Running Time. Table 3 shows the average runtime (in milliseconds) of different methods to generate a CF example for a single data point. CounterNet outperforms all seven baselines in every dataset. In particular, CounterNet generates CF examples ~3X faster than VAE-CF, CouneRGAN, and VCNet, ~5X faster than C-CHVAE, and three orders of magnitude (> 1000X) faster than other baselines. This result shows that CounterNet is more usable for adoption in constronsr environments.
4.2 FURTHER ANALYSIS
Ablation Analysis. We analyze five ablations CounterNet to underscore the design choices inside CounterNet. First, accentuate the importance of the MSE loss functions used to optimize CounterNet (Eq. 1) by replacing the MSE based Li and C2 loss in Eq. 1 with binary cross entropy loss (CounterNeet-BCE) Second, underscore the importance of CounterNet's two-stage coordinate descent procedure by using conventional one-step BP optimization to train CounterNet instead CounterNet-SSngleeBe
In addition, we validate CounterNet's architecture design by experimenting two alternative designs: (i) we use a separate predictor f X' + y and CF generator g X - X', such that f and g share no identical components (unlike in CounterNet, where 2₂ are shared with both f and g: CaunterNNet-Spaarati generator by excluding passing Pa (CounterNet-NoPass: Finally, we highlight the importance of the joint-training of predictor and CF generator in CounterNet by training the CounterNet in post-hoe fashion (CounterNet-- (CoutterNet-Posthoo)) i.e., we first train the predictor on the entire training dataset, and optimize CF generator while the trained predictor is frozen.
and (ii) we highlight the design choice of passing Pa. to the CF
Table 4 compares the validity and proximity achieved by CounterNet and five ablations. Importantly, each ablation leads to degraded performance as com- pared to CounterNet, which demonstrates Counter- Net's different design choices. CounterNet-BCE and CounterNet-SingleB3 perform poorly in compari- son, which illustrates the importance of the MSE- based loss function and block-wise coordinate de- scent procedure. Similarly, and CounterNet--NPauss-P achieve degraded valid- ity and proximity scores, which highlight the impor- tance of CounterNet's s architecture design. Finally, CounterNet-Postheee achieves comparable validity as CounterNet, but fails to match the performance of proximity. This result demonstrates the importance of the foint-training procedure of CounterNet in op- timally balancing the cost-invalidity trade-off.
Base Model
CounterNet NoFreezee
Perturbation Stabiiity
CounterNet
0.6
0.4
Sointerneet-Separad
0.2
0.05
0.10
0.15
0.20
Epsilon
Figure 4: Robustness of the predictor f()). CounterNet reaches the upper bound of ro- bustness (i.e., comparable the base model).
8

---

Under review as a conference paper at ICLR 2023
Adversarial Robustness. We illustrate that CounterNet does not suffer from decreased robustness of the predictor network resulting from optimizing for the validity loss L2 (as shown in Lemma 3.2). We compare the robustness of CounterNet's 's predictor network f(-) against two baselines: (i) the base predictive model described in Table I; and (ii) CounterNet without freezing the predictor at the second stage of our coordinate descent optimization CCounterNet-NoFrreee
Figure 4 illustrates the perturbation stability (Wu et al., 2021) of all three CounterNet variants against adversarial examples (generated via projected gradient descent (Madry et al., 2018)). CounterNet achieves comparable perturbation stability as the base model, which indicates that CounterNet reaches its robustness upper bound (i.e., the robustness of the base model). Moreover, the empirical results in Figure confirm achieves significantly poorer stability. We observe similar
Lemma 3.2 as CounterNet-NoFFees patterns with different attack methods on other datasets (see Appendix C.I). These results show that by freezing the predictor and encoder networks at the second stage of our coordinate descent procedure, CounterNet suffers less from the vulnerability issuc created by the adversarial examples.
Feasibility of CF Explanations. Finally, we show how CounterNet's training procedure can be adapted to ensure that the generated CF examples feasible. In particular, we attempt to use projected gradient descent during the training of CounterNet and enforce hard constraints during the inference stage in order to ensure that the generated CF examples satisfy immutable feature constraints (e.g., gender r should remain unchanged). At the training stage, CF example x' first generated from 9(-), and is pro- jected into its feasible space (i.e., x" = P²(x)). Next, we optimize CounterNet over the prediction dz and its projected CF example x" (via our block-wise coordinate descent procedure). During inference, we enforce that the set of immutable features remains unchanged. Table 5 shows that enforcing immutable features (via projected gradient descent) does not negatively impact the validity and proximity of the CF examples. This result shows that CounterNet can produce CF examples that respect feasibility constraints.
Table 5: Impact of the immutable feature constraints in CounterNet. CounterNet generates feasible CF explanations with- out sacrificing validity and proximity.
Dataset
Val. Diff.
Prox. Diff.
Adult Credit OULAD
0.0
.009
0.0
.005
0.0
.004
5 DISCUSSION & CONCLUSION
Although our experiments exhibit CounterNet's superior performance than post-hoc baselines, these two methods have somewhat different motivations. While post-hoe methods are designed for generating CF explanations for trained black-box ML models (whose training data and model weights might not be available), CounterNet most suitable when the ML model developers aspire to build prediction and explanation module from scratch, where the training data can be exploited to optimize the generation of CF examples. We anticipate CounterNet to be valuable for service providers who wish to comply with GDPR-style regulations without sacrificing their operational effectiveness (e.g.. reduced predictive power). Importantly, CounterNet can still be used to interpret proprietary ML models by forcing its predictor network mimic that proprietary model (see Appendix D.1).
CountenNet has two limitations: (i) We do not consider other desirable aspects in CF explanations, such diversity (Mothilal et al., 2020), recourse cost (Ustun et al., 2019), fairness (Von Kügelgen et al., 2022), and causality (Karimi et al., 2021). Further research is needed to address these issues. (ii) It is also important to ensure that generated CF examples do not amplify or provide support to the narratives resulting from pre-existing erestiiitigg race-based and gender- based societal inequities (among others). One short- tertt workaround is have humans in the loop. We can provide CounterNet's explanations as a decision-aid to a well-trained human official, who is in charge of communicating the decisions of MI. models to human end-users in a respectful and humane manner. In the long-run, further qualitative studies are needed to understand the social impacts of CounterNet.
This paper proposes CounterNet, which integrates predictive model training and CF example gener- ation into single end-to-end pipeline. Unlike prior work, CounterNet ensures that the objectives of predictive model training and CF example generation are closely aligned. We adopt a block- wise coordinate descent procedure effectively train CounterNet. Experimental results show that CounterNet outperforms state-of of-te-of-thh-an the-art baselines in validity, proximity, and runtime, and is highly competitive in predictive accuracy, sparsity, and closeness to data manifold.

---

Under review as a conference paper at ICLR 2023
6 ETHICS & REPRODUCIBILITY STATEMENT
Ethics Statement. Although CounterNet is suitable for real-time deployment given its superior performance in its highly aligned CF explanations and speed, one must be aware of the possible negative impacts of its CF explanations to human end-users. It is important to ensure that generated CF examples do not amplify or provide support the narratives resulting from pre-existing race- based and gender- workaround to have humans the loop. We can provide CounterNet's 's explanations as decision- aid to a well-trained human official, who is in charge of communicating the decisions of MI. models to human end-users in a respectful and humane manner. In the long-run, further qualitative and quantitative studies are needed to understand the social impacts of CounterNet.
societal inequities (among others). As we stated in Section 5, one short-term
Reproducibility Statement. To aid the reproducibility of this work, we provide the code in the supplement material, and will make the code public once it is accepted. We also provide the dataset used for evaluating this paper in this anonymous repository. In addition, we outline the choices of hyperparameters in Appendix B.4. A detailed description of our experimental implementation can be found in Appendix B. For theoretical analysis of our novel block-wisd wise coordinate descent procedure, provide complete proof in Appendix A.
REFERENCES
loannis Arapakis, Souneil Park, and Martin Pielot. Impact of response latency user behaviour in mobile web search. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval, pp. 279-283, 2021.
Arthur Asuncion and David Newman. Uci machine learning repository, 2007.
Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, and Peter Eckersley. Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* '20, pp. 148-657, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450369367. doi: 10. 01444/3351195.33356655 URL htthhtts::/// //ooii///////////oo wtloiioorg/1..1111 3351095. 3375624.
Reuben Binns, Max Van Kleek, Michael Veale. Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 'it's reducing a human being to a percentage' perceptions of justice in algorithmic decisions. In Proceedings of the 2018 Chi conference on human factors computing systems, pp. 1-14, 2018.
Jock Blackard. Covertype. UCI Machine Learning Repository, 1998.
Catherine Blake. Uci repository of machine learning databases. http://ww.. ics. uci. edu mlearn/LLRepositor html, 1998.
Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan Su. This looks like that: deep learning for interpretable image recognition. In Advances in neural information processing systems, pp. 8930-8941 2019.
Paulo Cortez and Alice Maria Gonçalves Silva. Using data mining to predict secondary school student performance. 2008.
Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18, pp. 590-601, Red Hook, NY, USA, 2018. Curran Associates Inc
FICO. Explainable machine learning challenge. https: / http://cccmmmcc/////m/ ty. fico. com/s. s/ exp explainable--made ne- www.learneeeeeeeeeeeee www.nleernnnnhhaeeeo chal lenge September 2018.
lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun (eds.). 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http::httt///ttee http://awxvvvvog//boaa/o
6572.
10

---

Under review as a conference paper at ICLR 2023
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. A survey of methods for explaining black box models. ACM computing surveys (CSUR), 51(5):1-42, 2018.
2018.
Victor Guyomard, Françoise Fessant, Thomas Guyet, Tassadit Bouadi, and Alexandre Termier. VCNet: A self-explaining model for realistic counterfactual generation. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases PKDD), 2022.
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. Advances in neural information processing systems, 30, 2017.
Shalmali Joshi, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. Towards realistic individual recourse and actionable explanations black-box decision making systems. arXiv preprint arXiv: 997700965, 2019.
Kaggle. Titanic machine learning from disaster. https://www. httpgg/lll.mmm///ttaaata.aaa overview, September 2018.
Amir-Hossein Karimi, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. survey of algorithmic recourse: definitions, formulations, solutions, and prospects. arXiv preprint arXiv:2010.044 0400000 2020.
Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. Algorithmic recourse: from counter- factual explanations to interventions. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 353-362, 2021.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International Conference on Machine Learning, pp. 1885-1894. PMLR, 2017.
Kohavi and B Becker. Uci machine learning repository: Adult data set, 1996.
Jakub Kuzilek, Martin Hlosta, and Zdenek Zdrahal. Open university learning analyties dataset. Scientific data, 4:1 2017.
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in neural information processing systems, pp. 4765-47744 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL httt:::://ppeeeeviie net forum? did=
IJZIBEZAb.
Divyat Mahajan, Chenhao Tan, and Amit Sharma. Preserving causal constraints counterfactual explanations for machine learning classifiers. arXiv preprint arXiv: /9/2.03277, 2019.
Tim Miller. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 267:1-38, 2019.
Christoph Molnar, Giuseppe Casalicchio, and Bernd Bischl. Interpretable machine learning-a brief history, state-of-the-art and challenges. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 417-431, Springer, 2020.
Ramaravind K Mothilal, Amit Sharma, and Chenhao Tan. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 607-617, 2020.
Daniel Nemirovsky, Nicolas Thiebaut, Ye Xu, and Abhishek Gupta. Countergan: Generating counterfactuals for real-time recourse and interpretability using residual gans. In Uncertainty in Artificial Intelligence, pp. 1488-1497, PMLR, 2022.

---

Under review as a conference paper at ICLR 2023
Martin Pawelczyk, Klaus Broelemann, and Gjergji Kasneci. Learning model-agnostic counterfactual explanations for tabular data. Proceedings of The Web Conference 2020, pp. 3126-3132, 2020.
Forough Poursabzi-Sangee
Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wort- man Vaughan, and Hanna Wallach. Manipulating and Measuring Model Interpretability. As- sociation for Computing Machinery, New York, NY, USA, 2021. ISBN 9781450380966. URL https:/ /oii .rrgrr/// 10. . 1141/341111111 1111//41116444443311 3445315.
Kaivalya Rawal, Ece Kamar, and Himabindu Lakkaraju. Can i still trust you?: Understanding the impact of distribution shifts on algorithmic recourses. arXiv preprint 1rXiv:2012.11788 2020.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference knowledge discovery and data mining, pp. 1135-1144, 2016.
Pau Rodríguez, Massimo Caccia, Alexandre Lacoste, Lee Zamparo, Issam Laradji, Laurent Charlin, and David Vazquez. Beyond trivial counterfactual explanations with diverse valuable explanations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1056-1065, 2021.
Alexis Ross, Himabindu Lakkaraju, and Osbert Bastani. Learning models for actionable recourse. Advances in Neural Information Processing Systems, 34, 2021.
Lisa Schut, Oscar Key, Rory Me Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. Generating interpretable counterfactual explanations by implicit minimisation of epistemic and aleatoric uncertainties. In International Conference on Artificial Intelligence and Statistics, pp. 1756-1764. PMLR, 2021.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference computer vision, pp. 618-626, 2017.
Sumedha Singla, Brian Pollack, Junxiang Chen, and Kayhan Batmanghelich. Explanation by progressive exaggeration. In International Conference on Learning Representations, 2020. URL
httpp:://ppenret wwt.wwwwwwwweeeeee...........eee http://www.eee/ommmoeeo-------- wtwpoooddddlllleeeeeeeeeedee
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint 1706::1700.03822.. 2017.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-1958, 2014.
Ilia Stepin, Jose M Alonso, Alejandro Catala, and Martín Pereira-Farina A survey of contrastive and counterfactual explanation generation methods for explainable artificial intelligence. IEEE Access, 9:11974-12001, 2021.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference Machine Learning, pp. 3319-3328. PMLR, 2017.
Sohini Upadhyay, Shalmali Joshi, and Himabindu Lakkaraju. Towards robust and reliable algorithmic recourse. Advances in Neural Information Processing Systems, 34, 2021.
Berk Ustun, Alexander Spangher, and Yang Liu. Actionable recourse in linear classification. In Proceedings of the Conference Fairness, Accountability, and Transparency, pp. 10-19, 2019.
Arnaud Van Looveren and Janis Klaise. Interpretable counterfactual explanations guided by proto- types. arXiv preprint arXiv: 1007. 2019.
Sahil Verma, John Dickerson, and Keegan Hines. Counterfactual explanations for machine learning: A review. arXiv preprint arXiv:2010. 10596, 2020.
12

---

Under review as a conference paper at ICLR 2023
Sahil Verma, Keegan Hines, and John P Dickerson. Amortized generation of sequential algorithmic recourses for black-bot box models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 8512-855 8519, 2022.
Julius Von Kügelgen, Amir-Hossein Karimi, Umang Bhatt, Isabel Valera, Adrian Weller, and Bernhard Schölkopf. On the fairness of causal algorithmic recourse. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 9584-9594, 2022.
Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harv. JL & Tech., 31:841, 2017.
Boxi Wu, Jinghui Chen, Deng Cai, Xiaofei He, and Quanquan Gu. Do wider neural networks really help adversarial robustness? Advances in Neural Information Processing Systems, 34, 2021.
Bing Xu. Naiyan Wang, Tianqi Chen, and Mu Li. Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv: 1505. 00853, 2015.
Fan Yang, Sahan Suresh Alva, Jiahao Chen, and Xia Hu. Model-based counterfactual synthesizer for interpretation. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery and Data Mining, KDD '21, pp. 1974, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 4703433333
doi: 10. 10.11457444548834477733 URL http://d/d org/
10.1145/34 341/344755444 .. 3467333.
I-Cheng Yeh and Che-hui Lien. The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients. Expert Systems with Applications, 36(2): 2473-2480, 2009.
Yixue Zhao, Marcelo Schmitt Laser, Yingjun Lyu, and Nenad Medvidovic. Leveraging program analysis reduce user- seer-perceive latency in mobile applications. In Proceedings of the 40th International Conference on Software Engineering, pp. 176-186, 2018.
13

---

Under review as a conference paper at ICLR 2023
Appendix
A Supplemental Proof
15
A.1 Proof Lemma 3.1
15
Proof Lemma 3.2
B Implementation Details
16
B.1 Software and Hardware Specification
16
B.2 Datasets for Evaluation
16
B.3 Evaluation Metrics
16
B.4 CounterNet Implementation Details Hyper-parameters Baselines
16
C Additional Experimental Results
17
C.I Additional Robustness Results
17
C.2 Training Time of CounterNet
18
D Additional Ablation Study
18
D.1 CounterNet under the Black-bor box Blatdhsssppppo.................. Ablations CounterNet's Training
18
E Experimental Evaluation Small-Sized Datasets
19
F Second-order Evaluation
21
F1 Experimental Results
21
G CounterNet under the Multi-class Settings
23
Training CounterNet Multi-Class Classification
diasiiccii......................
23
G.2 Experimental Evaluation
23
H Impact of Neural Network Structures
24
CounterNet on the Image Dataset
24
Woal-World Usage.
14

---

Under review as a conference paper at ICLR 2023
A
SUPPLEMENTAL PROOF
A.1 PROOF OF LEMMA 3.1
Proof. VoLi = Ve lly drl12. and VoL2 = Vo (1(1 -ba) - 0z Then, we have
VoLi Voy2 - 2y Vo 02 + Voyi
2y Voña + Vayi
-2y V 20 V Voux
=((xz y) Vouz
Since x² -> we expect CF example x' is closed to the original instance x, we can replace x to L2. Then, we have
VoL2 = Vo(1 - 2
= -44- Voûu + + Voye
--4- Voûr + +4.2 Us Voge
= 4- 1202g 1)Voùz
Hence,
VoLi VoL, = 2- (g 9) Voy. 4 (20. 1) (1- g) (2) o)())
Since (Voūa)²> > 0, only need to prove whether (gx y) (24 1) is positive negative. Given that 10 yl 0.5,
if y 1, we have 0.5 < 0æ Then, (je 9) 0, (2gz 1) > 0. y = 0, we have < 0x < 0.5. Then, (g- y) > 0, (20z - 1) < 0.
Therefore, (úx - y) (2) 1) < Hence, VoLi VoL2 <0.
A.2 PROOF OF LEMMA 3.2
Proof. Assuming fo(x) - y as we expect the predictor network produces accurate predictions, and {0. 1), we can replace fo (x) to y. Then, minimizing Le (in Lemma 3.2) indicates minimizing
[l - (1 fo(x'))l22- Since fo(-) 1, we have
min 1yy (1 fo(')))l max ly fo(x'))ls
By replacing y to fe(x), then minimizing L2 indicates maximizing [l.f(()) - (2)))l2 By definition, the lipschitz constant K is
[ fo('')11 ||-- x'll
K sup [[] sup z'EB(a.c)
x'E3(a,c)
where minimizing L2 increases [])((x) fo(x') (2)))) Therefore, the lipschitz constant K increases.
15

---

Under review as a conference paper at ICLR 2023
B IMPLEMENTATION DETAILS
Here we provide implementation details of CounterNet and five baselines on four datasets listed in Section 4. The code can be found in the supplemental material.
B.1 SOFTWARE AND HARDWARE SPECIFICATION
We use Python (v3.7) with Pytorch (v1.82), Pytorch Lightning (v1.10), numpy (v1.19.3), pandas (1.1.1) and scikit-learn (0.23.2) for the implementations. All our experiments were run on a Debian-10 Linux-based Deep Learning Image with CUDA 11.0 the Google Cloud Platform.
The CounterNet' network is trained on NVIDIA Tesla V100 with an 8-core Intel machine. CF generation of four baselines are generated from the same 16-core machine.
16-core Intel machine with 64 GB of RAM. The evaluation
B.2 DATASETS FOR EVALUATION
Here, we reiterate our used datasets for evalu- ations. Our evaluation is conducted eight widely-used tabular datasets. Our primary eval- uation uses four large-sized datasets (shown in Section 4), including Adult, Credit, HELOC, and OULAD, which contain least 10k data instances. In addition, we experiment with four small-sized datasets, including Student, Titanic, Cancer, and German. Table summa- rizes datasets used for evaluations.
Table 6: Summary of Datasets used for Evaluation
Dataset
Size
#Continuous
#Categorical
Adult Credit
32,561
30,000
20
HELOC
10,459
21
OULAD
32,593
23
Student
649
2
14
Titanic
891
2
24
B.3 EVALUATION METRICS
Cancer
569
30
German
1,000
7
13
Here, we provide formal definitions of the eval- uation metrics.
Predictive Accuracy is defined as the fraction of the correct predictions.
P i Accurray = #If(x) y
(2)
71
Validity is defined as the fraction of input instances on which CF explanation methods output valid CF examples.
Validity= #If(x') = =1 11-1
(3)
Proximity is defined the L norm distance between and x' divided by the number of features.
n
Proximity nd
x (
Sparsity is defined as the fraction of the number of feature changes between and z'.
1
Sparsity- = nd 1
x1
(5)
B.4 COUNTERNET IMPLEMENTATION DETAILS
Across all six datasets, we apply the following same settings in training CounterNet: We initialize the weights in He et al. (2016). We adopt the Adam with mini-batch size of 128. For each datasets,
16

---

Under review as a conference paper at ICLR 2023
trained the models for up to 1 10 iterations. To avoid gradient explosion, we apply gradient clipping by setting the threshold to 0.5 to clip gradients with norm above 0.5. We set dropout rate to 0.3 to prevent overfitting. For all six datasets, set A 1.0, X2 0.2, X 0.1 in Equation 1.
The learning rate is the only study, we find the training to CounterNet is sensitive to the learning rate, although good choice of loss function (e.g. choosing MSE over cross-entropy) can widen the range an "optimal" learning rate. We apply grid search tune the learning rate, and our choice is specified in Table 7.
that varies across six datasets. From our empirical
Additionally, we specify the architecture's details (e.g. dimensions of each layer in encoder, predictor and CF generator) in Table 7. The numbers in each bracket represent the dimension of the transformed matrix. For example, the encoder dimensions for adult dataset is [29, 50, 10], which means that the dimension of input R2 is 29 (e.g. 29); the encoder first transforms the input into 50 dimension matrix, and then downsamples it to generate the latent representation 2 E R where k 10.
Table Hyperparameters and architectures for each dataset.
Dataset
Learning Rate
Encoder Dims
Predictor Dims
CF Generator Dims
Adult
0,003
[29, 50, 10]
[10, 10, 2]
[20, 50, 29]
Credit
0.003
[33, 50, 10]
[10, 10, 2]
[20. 50, 33]
HELOC
0.005
[35, 100, 10]
[10, 10,
[20, 100, 35]
OULAD
0.001
[127, 200, 10]
[10, 10,
[20, 200, 127]
Student
0.01
[85, 100, 10]
[10, 10. 2]
[20, 100, 85]
Titanic
0.01
(57, 100, 10]
[10, 10, 2]
[20, 100, 57]
Cancer
0.001
[30, 50, 10]
[10, 10, 2]
[20, 50, 30]
German
0.003
(61,50] 10]
[10, 10, 2]
[20, 50, 61]
B.5 HYPER PARER--ARAMEERSS FOR BASELINES
Next, we describe the implementation of baseline methods. For VanillaCF and ProtoCF, we follow author's instruction much as we can, and implement them in Pytorch. For VanillaCF, DiverseCF and ProtoCF, we run maximum 1 10x steps. After CF generation, we convert the results to onc-hot-encoding for- mat for each categorical feature. For training the VAE-CF, we follow Mahajan et al. (2019)'s settings on running maximum 50 epoches and setting the batch size to 1024. use the same learning rate as in Table 7 for VAE training-
Table 8: Learning rate of the base predictive models on each dataset.
Dataset
Learning Rate
Adult
0.01
HELOC
0.005
OULAD
0.001
Student
0.01
For training predictive models for baseline algorithms, we apply grid search for tuning the learning rate, which is specified in Table 8. Similar to training the CounterNet, we adopt the Adam with mini-batch size of 128, and set the dropout rate to 0.3. We train the model for up to 100 iterations with early stopping to avoid overfittings.
Titanic
0.01
Cancer
0.001
C ADDITIONAL EXPERIMENTAL RESULTS
Here, we provide additional results of experiments Scction 4. These results further demonstrate the effectiveness of CounteNet.
C. ADDITIONAL ROBUSTNESS RESULTS
We provide supplementary results on evaluating the robustness of the predictor network on three large datasets (i.e,, Adult, HELOC and OULAD). In particular, we implement FSGM (Goodfellow et al., 2015) and PGD (Madry et al., 2018) attack for testing the robustness of the predictive models. Figure 5 illustrates that CounterNet achieves comparable perturbation stability (i.e., the robustness of
17

---

Under review as a conference paper at ICLR 2023
the predictive model) the base model. In addition, Figure 5 supports the findings in Lemma 3.2 since CounterNet-Nooreeer consistently achieves lower stability than base models and CounterNet.
0.8
Buc Medel
Penartatios Suaii Subale
Daterkiin Sucale
0.5
Countecee
80
Couneereet
8g &
03
0.10
0.15
0.05
Epsioo
Epoia
Spellon
(b) Robustness of f(-) under FSGM attack on the OULAD dataset.
Robustness of f(-) under FSGM attack on the HELOC dataset.
(a) Robustness of f()) under
FSGM attack on the adult dataset.
Poutiiio Sabite
0.5
& &
Connerthes
0.
0.4
Pestattaion 8 &
0.3
0.2
0.1
03
Epetion
Spation
Upuioo
(d) Robustness of f(-) under PGD attack adult dataset.
(c) Robustness of f(-) under PGD attack on the OULAD dataset.
(f) Robustness of f(-) under PGD attack on the HELOC dataset.
Figure 5: Robustness of f(.) under FSGM attack (5a-5c) (Goodfellow et al., 2015) and PGD (5d-5f) (Madry et al., 2018) attack.
C.2 TRAINING TIME OF COUNTERNET
Table 9 shows the training time of base model and CounterNet for each epoch in seconds. CounterNet takes roughly 3X times training time as compared to the base model (which is only trained for the predictive performance). Note that the training time secondary metric for evaluating the speed, as the training of the model only occurs once, whereas the inference time (i.e., runtime in Table 2) is more important metrie, as keeps increasing during the deployment stage.
Table 9: Training time of base model and CounterNet for each epoch (in second).
Dataset
Base Model
CounterNet
Adult
0.73
2.11
HELOC
0.22
0.65
OULAD
0.72
2.11
D ADDITIONAL ABLATION STUDY
D. COUNTERNET UNDER THE BLACK-BOX ASSUMPTIONS
We illustrate how CounterNet can be adapted to the post-hoc black-box setting, In this setting, CF explanation methods generate CF explanations for trained black-box model (with access to the model's 'ss output). CounterNet can also be used in this post-hoc setting by forcing the predictor network to surrogate the black-bon model. Specifically, let black-box model M X - y outputs the predictions, our goal of training the predictor is to ensure that the predictor model behaves like the black-box model (i.e., M(x) = fxx).. The training objective of CounterNet is
1
N
] (6)
argminn 8
X (M - (z.) )2 d2
(0z. (1- ún;)) + As
(x. (x1-2()²
N
Prediction Loss (C1)
Validity Loss (C2)
Proximity Loss (Cs)
18

---

Under review as a conference paper at ICLR 2023
Table 10: Evaluation CounterNet under the post-hoc setting. CFNET-BB represents the CounterNet evaluated under the black-box setting. CFNET-PH represents the CounterNet trained via post-hoc fashion, which in turn, demonstrates the importance of joint-training procedure in CounterNet.
Method
Aduit
Credit
HELOC
OULAD
Val.
Prox.
Spar. Man.
Val.
Prox.
Spar.
Man.
Val.
Prox.
Spar. Man.
Prox.
Spar.
Man.
CFNET- BB
0.99
.217
.716
0.73
1.99
.138
.861
0.64
00.98
.158
.758
0.58
0.99
.073
.641 0.96
CFNET- 1.00
.276
.663
1.26
1.00
.247
.804
1.36
1.00
.153
.815
0.83
0.99
.099
.731
1.64
CounterNet 1.00
.196
.644
0.64
1.00
.132
912
0.56
1.00
.125
.740
0.56
1.00
.075
.725
0.87
Table 11: Ablation analysis of CounterNet. Each ablation leads to degraded performance, which in turn, demonstrates the importance of different design choices inside CounterNet.
Ablation
Adult
Credit
HELOC
OULAD
Val.
Prox.
Val.
Prox.
Val.
Prox.
Val.
Prox.
CounterNet-Iy
.98
0.25
0.99
.163
.99
.155
0.99
.094
CounterNet
1.00
.196
1.00
.132
1.00
.125
1.00
.075
Note that Eq. 6 looks identical to Eq. 1. The only difference is that y in Eq. is replaced to M (x;).
Table 10 shows the performance of CounterNet under the black-box setting (CFNET-BB). CFNET- BB degrades slightly in terms of validity, average Ln to CounterNet. This is because approximating the black-box model leads degraded performance in the quality of generating CF explanations.
D.2 ABLATIONS ON COUNTERNET'S TRAINING
In addition, provide supplementary results on ablation analysis of three large datasets (Adult, HELOC, and OULAD) to understand the design choices of the CounterNet training, shown in Figure 6). This figure shows that compared CounterNet's learning curve for L2. CounterNet-BCE and CounterNer-NoSmoone importance MSE-based loss functions and label smoothing techniques. Moreover, CounterNer- SingleBP's learning curve for La performs poorly in comparison, which illustrates the difficulty of optimizing three divergent objectives using single BP procedure. turn, this also illustrates the effectiveness of our block-wise coordinate descent optimization procedure in CounterNet's training. These results show that all design choices made in Section 3 contribute to training the model effectively.
learning curves show significantly higher instability, illustrating the
In addition, we experiment with alternative loss formulations. We replace the MSE based L3 loss in Eq. with norm (CounterNet-I,). Table 11 shows that replacing Ls with formulation leads to a degraded performance.
E EXPERIMENTAL EVALUATION ON SMALL-SIZED DATASETS
In addition four large datasets in Section we experiment with four small-sized datasets: (i) Breast Cancer Wisconsin (Blake, 1998) which classifies malignant (Y=1) or benign (Y=0) tumors; (ii) Student Performance (Cortez. & Silva, 2008) which predicts whether student will pass (Y=1) or fail (X-0) the exam; (iii) Titanic (Kaggle, 2018) which predicts whether passengers survived (Y- 1) the Titanic shipwreck or not (Y-0); and (iv) German Credit (Asuncion & Newman, 2007) which predicts whether the credit score of eustomer good (Y=1) or bad (Y=0).
Table compares the validity, average L1 and sparsity achieved by CounterNet and baselines.
Similar to results in Table 2, CounterNet achieves perfect validity. In addition, CounterNet achieves the lowest proximity in three out four small datasets. This result further shows CounterNet's s ability in balancing the cost-invalidity trade-off.
19

---

Under review as a conference paper at ICLR 2023
Predictive Accunccy
Valkiny
Presinity (Cs)
0.500
DCC
Deidetiee Ketreey (2)
0.77
Sakaby (C3
0.723
Oonamofit 6000
0.637
100
2003
(001
909
3000
3000
100
200
2000
100
6000
Kpuck
Stpoch
(a) Learning of model ablations the Adult dataset.
Predicziv Arcuracy (66)
Validit (62)
Preciminy (Ra)
Pridetive Acunncy (C)
Saiai 160
Promuit
10
Fipoils
(b) Learning curves of model ablations the HELOC dataset.
Prediciio Accuracy (6.)
Validity (8x)
Preediming (Pa)
0.98
Nicta
Fredette ACEERIO (4))
0.50
Frouinty (63)
0.75
0.70
0.00
0.55
(c) Learning curves of model ablations on the OULAD dataset.
Figure 6: Learning curves C1 (left), C2 (mid), and L3 (right) of model ablations on the Adult (6a), HELOC (6b), and OULAD (6c) dataset.
20

---

Under review as a conference paper at ICLR 2023
Table 12: Evaluation of counterfactual explanations on four small-sized datasets.
Method
Student Prox.
Titanic Prox.
Cancer Prox.
German
Val.
Spar.
Val.
Spar.
Val.
Spar.
Val.
Prox.
Spar.
VanillaCF
0.80
0.101
0.762
0.91
0.289
0.381
1.00
0.135
0.278
0.86
0.384
0.967
DiverseCF
0.53
0.117
0.565
0.52
0.321
0.370
0.99
0.075
0.157
0.64
0.246
1.000
ProtoCF
0.32
0.107
0.754
0.76
0.305
0.383
1.00
0.070
0.167
0.82
0.369
0.983
UncertainCI
0.45
0.251
0.675
0.41
0.422
0,512
1.00
0.023
0.039
0.50
0.310
0.945
C-CHVAE
1.00
0.110
0.797
1.00
0,389
0.475
0.62
0.353
0.325
1.00
0.307
0.568
VAE-CF
0.50
0.115
0.586
0.38
0.356
0.460
0.39
0.202
0.293
0.34
0.310
0.577
CounterNet
1.00
0.075
0.725
1.00
0.257
0.354
1.00
0.121
0.259
1.00
0.222
0.626
F SECOND-ORDER EVALUATION
We define three additional second-order metrics which attempt to evaluate the usability of CF explanation techniques by human end-users. We posit that negligible feature differences (among continuous features) between instance x and CF example x' make difficult for human end-users to use CF example x' (as many of the recourse recommendations contained within x' may not be actionable due to negligible differences). For example, human end-users may find it impossible to increase their Daily Sugar __trs_mConddddee by 0.523 grams (if the value of Daily Sugar Consumed feature is 700 and 700.523 between x and z', respectively). As such, human end-users may be willing to ignore small feature differences between and x'.
To define our usability related metrics, we construct a user- friendlyy second-order CF example x" by ignoring small feature differences (i.e., |a; zz!1 is less than threshold b) between instance and CF example x'. Formally, let & = {xı, I2, .., -22 } and x' = {2í, 2i,.., x''} be the features of the input instance and the CF example, respectively. Then, we use threshold of b. and create new data point x" = (4: = 1 [1 + 1 ln, |n.- Vi E ....... i.e. we replace all features i E {1, d] in CF example x' with features in the original input instance for which [ x S b. Our metries for CF usability are defined in terms of z and x" as follows:
Second- Order Validity is defined as the fraction of input instances on which x" remains a valid CF example. High second-order validity is desirable, because it implies that despite ignoring small feature differences, the second-order CF example x" remains valid.
Second-Order Proximity is defined as the Ia norm distance between x and x". It is desirable to maintain low second-order proximity because it indicates fewer cumulative modifications in the input space.
Second-Oree Sparsity is defined as the number of feature changes (i.e., Lo norm) between x and x". High second-order sparsity enhances the interpretability of a CF explanation. Note that second-order sparsity is more important than the original sparsity metric, as the second-order CF example x" ignores small feature changes the continuous features, yielding fewer number of feature changes in the input space.
F.1 EXPERIMENTAL RESULTS
The evaluation of counterfactual usability measures the quality of the second-orded CF example z" which is created by ignoring negligible differences between input instance x and the CF example x'. We use a fixed threshold b = 2 to derive the "sparse" second-order CF example x", and compute the second-order evaluation metrics.
Second-order validity. Table 13 compares the second-order validity of CF examples generated by CounterNet and other baselines on all six datasets. Similar to results in Table 2, CounterNet performs consistently well across all six datasets the validity metric, as CounterNet is the only CF explanation method which achieves over 93.7% second-order validity on all six datasets. In particular, CounterNet achieves ~ 1% higher second-order validity than C-CHVAE (its closest competitor) on all six datasets. Further, CounterNet is the only CF method which achieves more than 90% second-order validity the Breast Cancer dataset, whereas all post-hoc baselines perform poorly
21

---

Under review as a conference paper at ICLR 2023
Table 13: Evaluation of Usability of Counterfactual Explanations
Datasets
Metrics
Methods
VanillaCF
DiverseCF
ProtoCF
C-CHVAE
VAE-CF
CounterNet
Validity
0.764
0,515
0.508
0.995
0.348
0.995
Adult
Proximity
5.843
8.007
7.261
8.139
8.319
7.170
Sparsity
4.445
5.297
5.181
5.771
5,869
5.148
Validity
1.000
0.906
1.000
0.986
1.000
0.988
HELOC
Proximity
5.350
5.202
6.131
5.841
6.725
4.289
Sparsity
20.304
9.979
18.514
18.166
20,546
17.020
Validity
1.000
0.701
0.999
0.886
0.969
0.980
OULAD
Proximity
12.469
14.751
13.183
13.569
13.335
11.740
Sparsity
23.618
17.516
23.360
24.696
18.162
22.472
Validity
0.669
0.528
0.307
0,982
0.485
0.982
Student
Proximity
11.919
18.392
13.606
21.406
21.336
19.758
Sparsity
6.840
9.313
7.896
10.847
10.951
10.043
Validity
0.987
0.570
0.785
1.000
0.386
0.978
Titanic
Proximity
17.282
16.809
17.039
21.145
20.278
15.056
Sparsity
9.906
9.632
9.960
12.359
11.964
9.215
Validity
0.699
0.196
0.329
0.615
0.210
0.937
Breast Cancer
Proximity
1.313
0.890
0.655
3.618
2.089
1.422
Sparsity
8.343
4.699
5,014
9.741
8.783
7.762
(none of them achieve second-order validity higher than 70%), despite the fact that three of these baselines (VanillaCI, DiverseCI, and ProtoCF) achieved more than 99% first-order validity on this dataset. This result demonstrates that CounterNet is much more robust against small perturbations in the continuous feature space.
Second-order Sparsity and Proximity. In terms of second-order sparsity, CounterNet outperforms two parametric CF explanation methods (C-CHVAE and VAE-CF), and maintains competitive performance against two non- non-parametric methods (VanillaCF and ProtoCF). Across all six datasets, CounterNet outperforms C-CHVAE and VAE-CF by ~00% on the this metrie. Moreover, the difference between the second-order sparsity achieved by CounterNet and VanillaCT (and ProtoCF) close 1%, which indicates that CounterNet achieves the same level of second-order sparsity these two non-parnnn con-paraettric methods. In terms of second-order proximity, CounterNet is highly proximal against baseline methods as it achieves the lowest proximity in HELOC, OULAD, and Titanic datasets (similar to results in Table 2).
cost-Invalidity Trade-off. Figure shows that Counter- Net positions the bottom left of this figure, which il- lustrates that CounterNet can balance the cost-invalidity trade-off in the counterfactual usability evaluation. No- tably, CounterNet outperforms all post-hoc methods in the second-orde invalidity metric, and maintains the same level of sccond-ordes sparsity as VanillaCF and ProtoCF (~1% difference). Moreover, although Diver- seCF achieves 10%% lower second-order sparsity value than CounterNet, it has ~00 higher second-order in- validity than CounterNet. This results from DiverseCF's inability to balance the the trade-off between second- order invalidity and sparsity. This high second-order in- validity of DiverseCF hampers its usability, even though it generates sparse explanations.
VanilaC
2.5
Doveee
ProluC
OCHVAL
CounerSia
Spucity
1.0
0.5
0.0
0.1
12.2
10.3
0.5
0.5
0.7
Invild22
22
Figure 7: Illustration of trade-off between invalidity and sparsity across six datasets (methods at the bottom left are preferable).

---

Under review as a conference paper at ICLR 2023
G COUNTERNET
UNDER THE MULTI-CLASS SETTINGS
prior CF explanation literature, counterfactual expla- nations are primarily evaluated under the binary classi- fication settings Mothilal et al. (2020); Mahajan et al. (2019); Upadhyay al. (2021). However, it is worth- noting that CF explanation methods (including CounterNet) can be adapted to the multi-class classifi- cation settings. This section first describes the problem setting of the CF explanations when dealing with multi-class classification. Next, we describe how to train CounterNet for multi-class predictions and CF explanations. Finally, we present the evaluation set-up and show the simulation results.
G.1 TRAINING COUNTERNET FOR MULTI-CLASS CLASSIFICATION
Given an input instance Rdd CounterNet aims to generate two outputs: (i) a prediction å.r TR for input instance z; and (ii) the CF example E Id as an explanation for input instance x. The prediction U.r E R is encoded as one-hot format as Ú.. E 13. where Σ. ye = 1, k denotes the number classes. Moreover, we assume that there desired outcome y for every input instances a. Then, it is desirable that a CF explanation Yz. needs to be predicted as the desired outcome y' (i.e.,
Yx = =').
The objective for CounterNet in the multi-class setting remains the same as in the binary setting. Specifically, we expect CounterNet to achieve high predictive accuracy, counterfactual validity and proximity. As result, adjust loss functions from Eq. follows:
C1 N
i=1 (yi )2.)²
1
L2 N
(ja; 3/
L3 N
(xi xx1)²
Same training CounterNet in the binary setting, we optimize the parameter 0 of the overall network by solving the minimization problem Eq. (except that we are switching to use loss functions in Eq. 7). Moreover, we adopt the same block-wise coordinate optimization procedure to solve this minimization problem by first updating for predictive accuracy
Vo(Ai L1), and then
updating for CF explanation 0" 0' Vo(d2 L2 + A3 Lz).
G.2 EXPERIMENTAL EVALUATION
Dataset. We use Cover Type dataset Blackard (1998) for evaluating the multi-class classification experiment. Cover Type dataset predicts forest cover type from cartographic variables. This dataset contains seven classes (e.g., Y-1, Y=2, Y- 7), with 10 continuous features. For CF explanation generation, we assume that cover type 5 (e.g., Y=5) the desired class. The original dataset highly imbalanced, so we equally sample data instances from each class.
Results. Table 14 compares the performance of counterNet and our two most competitive base- lines (i.c., VanillaCF and C-CHVAE) in the evaluation for binary datasets (as found in Table 2 & 13). This table shows that CounterNet can achieve competitive performance against post-hoc CF explanation techniques in the multi-class classification settings. In terms of predictive accuracy, CountenNet performs comparably as the baseline methods with only ~2% decrease (in average). In terms of validity and proximity, CounterNet can properly balance the cost-invalidity trade-off. Although CounterNet achieves higher proximity score than VanillaCF, it achieves 100% validity score. Compared to C-CHVAE, CounterNet achieves ~80% lower proximity. Finally, CounterNet runs order-of of magnitddess faster than our two baseline methods. CounterNet runs more than 1000X and 3000X faster than C-CHVAE and VanillaCH, respectively.
23

---

Under review as a conference paper at ICLR 2023
Table 14: Results for CF explanation methods on Forester Cover Type dataset.
Methods
Predictive Accuracy
Validity
Proximity
Running Time
VanillaCF 0.911
0.921
0.379
1679.676
C-CHVAE
0.911 CounterNet
1.000
1.503
734.625
0.887
1.000
0.800
0.566
H IMPACT OF NEURAL NETWORK STRUCTURES
We further study the impact of the different neural network blocks. In our experiment, we primarily use multi-layer perception as it is suitable baseline model for the tabular data. For comparison, We also implemented the CounterNet with Convolutional building blocks (i.e. replace the feed forward neural network with convolution layer). We implemented the convolutional CounterNet on the Adult dataset. To train the feed forward neural network with convolution layers, we set the learning rate as 0.03 and À 1.0, A2 = 0.4, A3 = 0.01. The rest of the configuration exactly the same as training CounterNet with MLP.
Table 15 shows comparison between CounterNet with convolutional building blocks (CounterNee-- Conv) and multi- perceptions (CounterNer-MLP), The results indicate that CounterNet-Com matches the performances of CounterNet-MBL In fact, CounterNet-Coud than CounterNet-Mil MMP because convolutional block is not well-suitable for tabular datasets. Yet, CounterNet-Conm outperforms the rest of our post-hoc baselines in validity (with reasonably good proximity score). This illustrates CounterNet's potential real-world usage in various settings is agnostic to the network structures.
performs slightly worse
Table 15: Results for the CounterNet with Convolution layers on Adult dataset.
Building Block
Predictive Accuracy
Validity Proximity
CounterNet-Coms CounterNet- MLP
0.823 0.828
0.980 0.994
7.554
7.156
I COUNTERNET ON THE IMAGE DATASET
CounterNet is designed to generate counterfactual explanations for tabular datasets (the most common case for CF explanations). We also experiment with CounterNet the image datasets. This experiment uses the MNIST dataset: class "7" is used as the positive label, and class "1" is used as the negative label. Next, apply the same CounterNet training procedure generate image counterfac- tuals. Table 16 demonstrates the results of CounterNet on the MNIST dataset. CounterNet achieves 52.4% validity with 0.059 average L1 distance. This result shows current limitation of CounterNet as applying CounterNet as-is is ill-suited for generating image counterfactual explanations.
Table 16: CounterNet on the Image Datasets.
Validity
Proximity
CounterNet
0.524
0.059
J REAL-WORLD USAGE
We illustrate how CounterNet generates interpretable explanations for end-users. Figure 8 show an actual data point z from the Adult dataset, and the corresponding CF explanation x' generated by CounterNet. This figure shows that and x' differ in three features. In addition, CounterNet generates z by ignoring feature changes that are less than threshold = 2 (in practice, domain experts can help identify realistic values of b). Note that due to CounterNer's high second-order
24

---

Under review as a conference paper at ICLR 2023
validity, x" also remains valid CF example. After this post-processing step, and x" differ in exactly two features, and the end-user is provided with the following natural-languuge explanation: "If you want the ML model to predict that you will earn more than USS50K. change your education from HS-Grat to Doctorate, and reduce the number of hours of work/week from 48 to 33.5." "
Age
week
workelass
Education
Marital
Job
Race
Gender
Predicted
Predictor
26
48
Private
HS-grad
Married
Sales
White
Male
< 50k
Explainer
Predictor
x"
27.3
33.5
Private
Doctorate
Married
Sales
White
Male
50k
Round-off
Predictor
26
33.5
Private
Doctorate
Married
Sales
White
Male
> 50k
50k
Figure 8: A counterfactual explanation from CounterNet.
25

---

Under review as a conference paper at ICLR 2023
26


Reviews:
[{"review_id": "iDL5HNbhMJs", "reviewer": "Reviewer_9tKf", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on generating counterfactual explanations for model predictions, and then proposes an end-to-end learning framework that learns an explanation generator while training the predictive model. They also analyze the challenges in jointly training the predictor and the explanation generator and propose an iterative training procedure.", "summary_of_the_review": "As the novelty is limited and the main argument(on cost-invalidity trade-off) is not well clarified, I am lean on the negative side.", "strength_and_weaknesses": "Strength:\n - It proposes a new learning framework to generate counterfactual explanations(CF) for ML models, which facilitates service providers to specialize CF explanation techniques that can leverage the knowledge of their particular ML model.\n- They theoretically analyze the joint objective and address two challenges in the jointly end-to-end training procedure. It may offer some insights into the reasonability of the iterative training scheme when jointly training two modules with different roles.\n\nWeaknesses\n- On the method:\n    - The idea of intrinsic explainability is not new, though may have not been applied in generating CF explanations directly.\n    - The so-called “block-wise coordinate descent procedure” is indeed an iterative training scheme, which is very common when handling two or more modules. \n    - While the paper emphasizes the proposed framework can achieve better cost-invalidity trade-off than the post-hot methods, it’s not clear why the framework can achieve that. The paper says “Second, in the post-hoc CF explanation paradigm, the optimization procedure that finds CF explanations is completely uninformed by the ML model training procedure (and the resulting decision boundary). Consequently, such a post-hoc procedure does not properly balance the cost-invalidity trade-off”, but it seems impossible and infeasible to obtain the decision boundary of the black-box model.\n- Others:\n    - A very similar paper seems to have been published on ICML2021.\n    - GitHub link provided in the paper seems to leak information about the author.\n    - There is a typo in the header of Table 5.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "dkd3AJpstz", "reviewer": "Reviewer_Ycp6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The goal of the paper is the provide counterfactual explanations (CFs) for neural classification models. The paper makes the point that existing methods for generating CFs are posthoc -- so they are not aware of the decision boundary of the model. As a result, the approximations taken by these models lead to subpar CFs. The posthoc search also means that the process of generating CFs is time consuming. To overcome these issues, the paper proposes to generate CFs as a part of the model forward pass. Specifically, the model architecture and the objective is modified to generate CFs. Experiments are shown comparing the performance with several existing methods.", "summary_of_the_review": "Overall, it is not clear if the problem solved in the paper (improving runtime and validity of CFs) is a relevant problem to begin with. The paper also could be improved in terms of conveying the design choices which seem quite adhoc. For these reasons, I don't think the paper is ready yet. ", "strength_and_weaknesses": "### Strengths \n\n1. The topic of the paper (generating CFs) is important and timely.\n\n### Weaknesses\n\nOverall, the paper makes a bunch of adhoc choices when setting up the architecture and the loss function. The added value over the other methods is not very clear. Also, the proposed architecture does not support important criteria like diversity. See detailed comments below:\n\n1. First, it is not very clear if improving the runtime is such an important aspect that requires training the model from scratch. From Table 3, it looks like Vanilla CF seems to be able to generate a CF within 1-2 seconds which seems quite reasonable. Also, adding the CF generation to model training seems to prolong the training time (due to multiple objective and alternative updates). How large was this additional training time?\n\n2. Some of the choices could be justified better. For instance, why consider the non-standard squared loss in the training objective. The paper provides some ablation analysis but the reasoning behind moving from the standard cross entropy loss is not very clear. Similarly, why should the change loss be the squared loss (is it the L2 norm)? Why not consider the sparsity inducing distance like in Wachter et al? Similarly, should we expect the proposed method to be sparse in Section 4.1 when the objective function contains no sparsity term itself?\n\n3. Section 3.3: Why is adversarial robustness is a special concern here? Aren’t usual NNs also prone to adversarial perturbations? I see the point that the addition of the validity loss might exacerbate the robustness. But then the paper should quantify the additional lack of robustness.\n\n4. Section 3.3: How does the alternate update strategy solve the divergent gradient problem? Since the models two objectives are in conflict with each other, is there an equilibrium point? How do you detect convergence?\n\n5. When measuring the manifold distance, it is not clear why the L1 metric was chosen. Why not L2 distance? Why only consider the 1-NN?\n\n6. Recent work on CFs seems to support more criteria than validity and proximity. For instance, Mothilal et al consider diversity. Schleich et al (https://arxiv.org/pdf/2101.01292.pdf) consider a combination of distance function to cap the number of feature changes and the max. change in the feature. Can these criteria be included in CounterNet?\n\n7. Since many of the gradient based methods highly depend on the init point (due to non-convexity of the CF objective), simply restarting with different random seeds could help with validity. It would be greatly helpful to add the restarting strategy to see how much the validity improves.\n\n8. Different CF methods come with different hyperparameters (e.g., balancing between sparsity and validity). How were these HPs selected?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "CjVOkeXfEA1", "reviewer": "Reviewer_MUfa", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The work proposes a new method for generating Countefactuals for neural networks. The method consists of designing the network by having a part dedicated to the creation of counterfactuals, which is trained together with the predictive part. Through reasoned design and a special training, this new network, called CounterNet, is able to achieve accuracy on a par with other models. It can also generate counterfactuals in much less time than current post-hoc methods and achieve competitive values of widely used metrics from prior literature.", "summary_of_the_review": "The strengths of the work far outweigh the weaknesses I have identified. Regarding these, they are mostly on the clarity of certain images or statements, which I strongly encourage the authors to follow, to make the work even more polished than it is now.\n\n## Minor remarks:\n1. In Introduction, I would advise some minor specifications:\n    * It should be specified that x2’ is an instance with different (or opposite) label.\n    * “More generally, CF examples with low (high) cost of change imply high (low) invalidities”. While usually true, this is not always the case. I would rephrase it, including “often”, “usually” or a synonym.\n    * “Next, we describe key limitations of prior CF techniques.” Seems redundant/ a repetition.\n    * “First, all prior methods” I’ll add, just once and here, “to our best knowledge” or similar.\n    * “Post-hoc explanation techniques are agnostic” Not every method is agnostic.\n    * “Consequently, such a post-hoc procedure does not properly balance the cost-invalidity trade-off (as explained above), causing shortcomings in the quality of the generated CF explanations” This would need a citation\n\n2. In Section 3:\n    * “output is binary-valued, then yˆx and yˆx′ should take on opposite values (i.e., yˆx + yˆx′ = 1)”. Since in a binary classification task, labels can also be (+1) and (-1), I would write “output is binary-valued (0, 1)” or similar\n    * While very well designed, Figure 2 hides two flaws. First, it does not include a yx’ used just for training, that is indeed cited in the text; probably adding a dotted line at the end of Predictor will fix this. Second, it should be highlighted that the connection from the Predictor to the CF Generator happens only once; otherwise, the network, in training regime, will continuously produce Counterfactuals of its own Counterfactuals in loop. Even in text, the “feedback loop” from x’ to input is mentioned, but not the px to CF Generator.\n    * “generate a latent vector z ∈ Rk (s.t. k < d)”, “and up-samples” While generally true, there’s no need to specify k < d, since, to my understanding, nothing prevents the network to work even if k ≥ d, unless you have experiments that prove otherwise.\n    * I think that three hyper-parameters are not actually needed for the loss (1), cause what’s important is the relative weight of the three losses. The additional parameter probably just ensure that there are multiple optimal configuration, fixing other hyperparameters (e.g. learning rate, decay rate, etc.). However, I don't think it should be changed.\n    * “Proof of Lemma 3.1 and 3.2 can be found in Appendix A.” It’s strange to mention Lemma 3.2 in the Lemma 3.1 section. I would move this sentence to the previous introductory paragraph, that of subsection 3.3\n    * “we update network weights θ = {θh, θf , θg} twice” This is in contrast with the sentence “we only update the weights in the CF generator θg”, cause you seem to update twice just the θg weights. But, if looking carefully at L1 loss, it doesn’t depend on θg, so each weight is actually updated just once, if you freeze the gradient updates.\n\n3. Experimental evaluation\n    * The Ablation Analysis section is really appreciated, but in my opinion needs slightly more clarity. Perhaps even just starting a new line every time a modification is mentioned would help.\n    * \"reaches the theoretical upper bound of robustness (i.e., the robustness of base model)\". This statement is incorrect for me. First of all, I do not understand why the upper bound of robustness should be the robustness of base model. Even if it were, CounterNet exceeds it, so it cannot be an upper bound. Secondly, there is no theoretical study of the bounds of robustness, so I would avoid such a mathematically specific statement.\n\n4. Appendix:\n    * In Section G, “methods with only 2 ̃% decrease (in average).” The tilde ~ should be before the 2%", "strength_and_weaknesses": "# Strengths:\n\n1. Clear and concise explanation of the Counterfactual Explainability setting.\n\n2.  The key limitations of previous methods are clearly presented, as well as how the new method is able to overcome them. The limitations of this new method are also explicitly stated.\n\n3. Much appreciated is the theoretical analysis of Convergence and Adversarial Examples, which gives depth to the paper.\n\n4. Experiments seems complete, with multiple datasets, many baselines and ablation analysis about design choices.\n\n# Weaknesses:\n\n1. Although some are well shown, I think some limitations of the model need to be made more explicit. W1a) First of all, this method is not suitable for already trained networks, which makes it useless if you cannot/will not replace the model you have with this one.\n     * Does the training run slower than that of a standard NN? If so, this should be highlighted as a (small) limitation. The time of just training the predictor could be added to Table 3.\n     * I also think it should be made explicit that, considering the manifold, VCNet is still very competitive, also managing to have perfect validity, but lesser manifold on 3 of the 4 datasets.\n     * While briefly stated in a footnote, there is no experimental proof that this architecture works well in domains where alternatives neuronal structures performs best. The authors try their method with a Convolutional block, but with tabular datasets; as they say \"convolutional block is not well-suitable for tabular datasets”. Likewise, is not explicit if the CounterNet used for the Image dataset is a convolutional one (Appendix I). Since this is not a good enough experimental proof to state that the method works well (as CF Generator) with other types of neural networks (as you say for images), it is necessary in my opinion to make this limitation explicit in the main paper and not only in the supplementary.\n\n2. “Note that passing px through the CF generator network is analogous to feeding information about the decision boundary to the CF example generation procedure, which leverages this knowledge to find high-quality CF examples x′” The reason why px contains boundary information is not clear. Likewise, is not theoretically explained why the CF generator needs both zx and px (apart from the experiments in the Ablation Analysis section).\n\n3. Some claims in the Experimental evaluation section should be sustained via statistical tests.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "7t19K9Tr8N", "reviewer": "Reviewer_Mece", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper tackles the problem is generating counterfactual (CF) explanations for neural networks. The authors propose a model that consists of an encoder, a predictor, and a CF generator. Correspondingly these models are trained by optimizing an objective function with three loss terms, prediction loss, validity loss, and change loss. Experiments show that CounterNet can generate CF explanations with reasonable performance. ", "summary_of_the_review": "This paper tackles the problem is generating counterfactual (CF) explanations for neural networks. This is an interesting and important problem in the ML community. Overall the model design makes sense to me. My major concerns include the absence of highly related state-of-the-art baselines, several false claims or over-claims in the abstract and introduction, and the limited technical merit, especially given a related work CounteRGAN. ", "strength_and_weaknesses": "Strength \n\n+ Generating counterfactual explanations is an important and interesting problem in the ML community. \n\n+ The design of the model with three terms, prediction loss, validity loss, and change loss, makes sense. \n\nWeaknesses\n\n- The proposed method has limited technical merit, especially compared to recent methods such as CounteRGAN [1]. \n\n- Some claims in the paper on major limitations of existing methods are not true. \n\n- Important and highly relevant baselines are missing.\n\n- Lemma 3.1 is incorrect or irrelevant to Eq (1) (see below for details). \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject