Gradient-based Counterfactual Explanations using Tractable Probabilistic Models

1 INTRODUCTION . Explaining decisions made by intelligent systems , especially black-box models , is important . First of all , a model that can explain their decisions is more likely to gain human trust , especially if there are significant consequences for incorrect results , or the problem has not been sufficiently studied and validated in real-world applications ( Simpson , 2007 ; Hoffman et al. , 2013 ; Doshi-Velez & Kim , 2017 ) . Secondly , explanations can promote fairness by exposing bias towards protected attributes in the system ( Prates et al. , 2019 ; Buolamwini & Gebru , 2018 ; Obermeyer et al. , 2019 ) . In addition , explanations are a helpful tool for debugging purpose ( Holte , 1993 ; Freitas , 2014 ; Rudin , 2019 ) . Counterfactuals are closely related to classes of explanation , which much of the early work in AI on explaining the decisions made by an expert or rule-based systems focused on ( Wachter et al. , 2017 ) . As the name implies , counterfactual is an alternative example counter to the fact which takes the form “ If X had not occurred , Y would not have occurred ” . Explicitly establishing the connection between counterfactuals and explanations , Wachter et al . ( 2017 ) concluded that counterfactual explanations can bridge the gap between the interests of data subjects and data controllers that otherwise acts as a barrier to a legally binding right to explanation . They further formulate the problem of finding counterfactual explanations as an optimization task that minimizes the loss of a desired outcome w.r.t . an alternative input combined with regularization for distance to the query example . In a similar vein , the dominant majority of recent counterfactual explanation methods are proposed based on their adapted objective functions for optimization . Unfortunately , these deep learning approaches have some downsides : First of all , they are very time-consuming for generating explanations due to the iterative optimization process , because for every optimization iteration the model needs to be evaluated at least once and it may take a lot of iterations until a candidate is found . Secondly , the objective function is usually highly non-convex , which makes it more tricky to find a satisfying solution in practice . Thirdly , the optimizer involves additional hyperparameters that need to be carefully selected , such as learning rate . Furthermore , although these methods optimize for the ” closest possible world ” by constraining the distance of the counterfactual to the query instance , they are still not aware of the underlying density and the data manifold . As a direct consequence , the counterfactuals often appear rather unnatural and unrealistic although they are as close as possible to the query instance . See figure 3 for example . This imposes difficulties for humans to understand the explanations . As Nickerson ( 1998 ) argues , humans exhibit confirmation bias , meaning that we tend to ignore information that is inconsistent with our prior beliefs . In analogy , unrealistic counterfactuals also have very limited effect for communicating explanations to humans . To improve upon these limitations , we propose a novel approach to generate counterfactual explanations . Unlike the overwhelmingly dominant deep learning approach , we decouple the goal of perturbing the prediction and maintaining an additional constraint on the perturbation . Specifically , we view the task as a two-step process , whereby the first step is to perturb a sample towards a desired class and the second step is to constrain the perturbed sample to be close to the data manifold so it is realistic and natural . The first goal aligns with adversarial attacks ( Goodfellow et al. , 2015 ; Brown et al. , 2017 ; Yuan et al. , 2019 ) , where a small perturbation is computed to perturb the prediction . In the literature there are gradient-based methods ( Kurakin et al. , 2016 ; Moosavi-Dezfooli et al. , 2016 ) and optimization-based methods ( Szegedy et al. , 2013 ; Carlini & Wagner , 2017 ) . Exploiting the fact that gradient corresponds to the direction of steepest ascent , gradient-based perturbation is not only very easy to implement with current deep learning frameworks , but also much faster to compute compared to optimization-based perturbation . Therefore we adopt a gradient-based approach for the first step . To ensure the quality and effectiveness of the second step , we learn a sum-productnetwork ( SPN ) ( Poon & Domingos , 2011 ) , a tractable density model , on the input , and we perturb the example further based on the gradient that directs to steepest ascent of likelihood , evaluated on the SPN . In summary , we make the following contributions : 1 . We propose the first approach for generating counterfactual examples using probabilistic circuits to ensure density-awareness in a tractable way . 2 . We experiment with complex and high-dimensional real-world dataset , which was not dealt with using domain-agnostic methods in the literature . 3 . We give empirical evidence to demonstrate the advantages and effectiveness of our approach : visually appealing examples with high density , fast computation , and high success rate . We proceed as follows : First , we give an overview of the literature and summarize the shortcomings of the existing approaches . Then we introduce RAT-SPNs , preparing for presenting our approach . Finally , we evaluate our approach via empirical evidence and conclude our work . 2 RELATED WORK ON COUNTERFACTUAL EXPLANATIONS . Research attention on counterfactual explanations has been increasingly raised since Wachter et al . ( 2017 ) presented the concept of unconditional counterfactual explanations as a novel type of explanation of automated decisions . Identifying the resemblance between counterfactual explanations and adversarial perturbations , Wachter et al . ( 2017 ) proposed to generate counterfactual explanations based on the optimisation techniques used in the adversarial perturbation literature . Specifically , a loss function is minimized w.r.t . its input , using standard gradient-based techniques , for a desired output and a regularizer penalizing the distance between the counterfactual and the query . Following Wachter et al . ( 2017 ) , Mothilal et al . ( 2020 ) augmented the loss with diversity constraint to encourage diverse solutions . However , these approaches do not have explicit knowledge of the underlying density or data manifold , which may lead to unrealistic counterfactual explanations . A bunch of methods counteract this issue by learning an auxiliary generative model to impose additional density constraints on the optimization process . As common choice for density approximator , Variational Autoencoder ( VAE ) ( Kingma & Welling , 2014 ) and its variants ( Klys et al. , 2018 ; Ivanov et al. , 2019 ) are used . For instance , Dhurandhar et al . 2018 proposed contrastive explanations method ( CEM ) for neural networks based on optimization . The objective function consists of a hinge-like loss function and the elastic net regularizer as well as an auxiliary VAE to evaluate the proximity to the data manifold . Ustun et al . 2019 defined the term recourse as the ability of a person to change the decision of a model by altering actionable input variables . Recourse is evaluated by solving an optimization problem . Joshi et al . 2019 provided an algorithm , called REVISE , to suggest a recourse , based on samples from the latent space of a VAE characterizing the data distribution . Pawelczyk et al . 2020 developed a framework , called C-CHVAE , to generate faithful counterfactuals . C-CHVAE trains a VAE and returns the closest counterfactual due to a Figure 1 : The SPN structure used to estimate a distribution as a mixture of classconditional densities . Each class-conditional density is in turn represented by a subSPN . Classification is done via Bayes ’ rule . nearest neighbour style search in the latent space . Downs et al . 2020 proposed another algorithmic recourse generation method , CRUDS , that generates multiple recourses satisfying underlying structure of the data as well as end-user specified constraints . Based on a VAE-variant , CRUDS uses a Conditional Subspace Variational Autoencoder ( CSVAE ) model ( Klys et al. , 2018 ) that is capable of extracting latent features that are relevant for prediction . Another method called CLUE Antorán et al . ( 2020 ) is proposed for interpreting uncertainty estimates from differentiable probabilistic models using counterfactual explanations , by searching in the latent space of a VAE with arbitrary conditioning ( VAEAC ) ( Ivanov et al. , 2019 ) . Poyiadzi et al . 2020 proposed FACE , a graph-based algorithm to generate counterfactuals that are coherent with the underlying data distribution by constructing a graph over all the candidate targets . Besides , several domain-specific approaches are also emerging Olson et al . ( 2021 ) ; Goyal et al . ( 2019 ) ; Chang et al . ( 2019 ) . The aforementioned techniques have the following shortcomings : 1 . The deep learning-like approach ( Wachter et al. , 2017 ; Mothilal et al. , 2020 ; Dhurandhar et al. , 2018 ; Ustun et al. , 2019 ; Joshi et al. , 2019 ) is too slow to generate explanations on the fly due to the iterative optimization process , which in turn comes with additional tuning parameters . 2 . Although highly expressive , neural density estimators such as VAEs are highly intractable , which makes explicit density constraint on counterfactual explanations infeasible ( Dhurandhar et al. , 2018 ; Ivanov et al. , 2019 ; Klys et al. , 2018 ; Joshi et al. , 2019 ; Pawelczyk et al. , 2020 ) . In addition , they come with the overhead of latent representation , which is not necessarily in good quality when maximum likelihood training is used to learn them ( Alemi et al. , 2018 ; Dai & Wipf , 2019 ) . We improve upon the aforementioned shortcomings by proposing the first counterfactual method using tractable probabilistic circuits , specifically sum-product networks ( SPNs ) ( Darwiche , 2003 ; Poon & Domingos , 2011 ) . 3 GRADIENT-BASED COUNTERFACTUAL EXPLANATIONS USING TRACTABLE PROBABILISTIC MODELS . Before explaining our idea , we first introduce the tractable probabilistic models we use . An sum-product network ( SPN ) S over X is a tractable probabilistic model for P ( X ) based on a directed acyclic graph ( DAG ) . This graph indicates computation for probabilistic inference and consists of three types of nodes : univariate leaf nodes , sum nodes , and product nodes . Let ch ( · ) denote the children of a node . A sum node S is weighted sum of its children , i.e . S = ∑ N∈ch ( S ) wS , NN where the weights wS , N are non-negative and sum to 1 , i.e . wS , N ≥ 0 , ∑ N wS , N = 1 . Sum nodes can be viewed as mixtures of their child distributions . A product node P is product of its children , i.e . P = ∏ N∈ch ( P ) N . The root node represents P ( X ) . Products are factorized distributions , implying independence assumption among their children . SPNs allow for fast , exact inference on high-treewidth models . Unlike most of the probabilistic deep learning approaches , SPNs permit exact and efficient inference . Specifically , they are able to compute any marginalization and conditioning query in time linear of the model ’ s representation size . By employing SPNs for deep learning , random and tensorized SPNs ( RAT-SPNs ) ( Peharz et al. , 2020 ) are proposed using a simple approach to construct random SPN structure and combine it with GPU-based optimization . It is worth noting that RAT-SPNs are not fooled by certain out-of-domain image detection tests on which VAEs , normalizing flows ( NFs ) , and auto-regressive density estimators ( ARDEs ) consistently fail ( Choi & Jang , 2018 ; Nalisnick et al. , 2018 ) . To learn the parameters ω of a given RAT-SPN structure S in generative setting to approximate a distribution P ∗ ( X ) , we assume i.i.d . samples X = { x1 , . . . , xN } are given . Then maximum likelihood estimation is employed , i.e . ω = argmax 1N ∑N n=1 logS ( xn ) , where S ( x ) is a distribution over X represented by the RAT-SPN S . Apart from the standard use for density estimation , RAT-SPNs can be used as a generative classifier as well . Consider a classification problem f : Rd → { 1 , . . . , C } with C labels , C roots are used to represent class-conditional densities Sc ( X ) = : S ( X|Y = y ) . The overall density distribution is then given by S ( X ) = ∑ y S ( X|y ) P ( y ) . See figure 1 for illustration . Bayes ’ rule is used to classify a sample x : S ( Y |x ) = S ( x|Y ) P ( Y ) S ( x ) . In other words , a RAT-SPN of this special structure has dual use : It is both a density estimator S ( X ) and a classifier S ( Y |X ) . We will use this model to demonstrate our approach because it can be used for classification and yield tractable density evaluation for free at the same time . However , our approach can be easily extended to deep neural classifiers by training an auxiliary RAT-SPN for density estimation . For more details on RAT-SPN , check out appendix . Our approach is defined as two serial perturbations : In the first step , we maximize log S ( y ′|x ) S ( y|x ) to induce desired outcome y′ , which is equivalent to maximizing log S ( x|y ′ ) S ( x|y ) since log S ( y′|x ) S ( y|x ) = log ( S ( x|y′ ) P ( y′ ) S ( x ) S ( x ) S ( x|y ) P ( y ) ) = log S ( x|y′ ) S ( x|y ) , when assuming a uniform class prior . Towards this end , we take a gradient step towards the steepest ascent of the counterfactual outcome . This results in the perturbed example u where u = x+ ∂ ( logS ( x|y′ ) − logS ( x|y ) ) ∂x ∗ 1 . This step is similar to generating an adversarial perturbation in that the sample u is expected to change the prediction to y′ from y with a very slight change . However , without additional constraints , the perturbed sample u is very likely to deviate from the underlying data manifold . In order to generate in-distribution counterfactuals , we maximize the density P ∗ ( u ) of the current sample u , which is approximated via the SPN S ( u ) . In contrast to the density estimators used primarily in the literature , SPNs have the merit that they allow for tractable density evaluation . To maximize the density S ( u ) , we take another gradient step towards its steepest ascent , i.e . x′ = u+ ∂S ( u ) ∂u ∗ 2. x′ is the final counterfactual explanation for x that yields y′ instead of y . See figure 2 for illustration .

Summary: This is a XAI paper presenting a method to find counterfactual examples (i.e., contrastive example leading to a different prediction from the one currently obtained for a given example). The approach is based on tractable generative probabilistic circuits. This makes the procedure considerably faster than the existing SOTA approach. This seems to be very valuable and extensive validations show good performances even in terms of interpretability, smoothness and effectiveness.