Gradient-based Counterfactual Explanations using Tractable Probabilistic Models

1 INTRODUCTION . Explaining decisions made by intelligent systems , especially black-box models , is important . First of all , a model that can explain their decisions is more likely to gain human trust , especially if there are significant consequences for incorrect results , or the problem has not been sufficiently studied and validated in real-world applications ( Simpson , 2007 ; Hoffman et al. , 2013 ; Doshi-Velez & Kim , 2017 ) . Secondly , explanations can promote fairness by exposing bias towards protected attributes in the system ( Prates et al. , 2019 ; Buolamwini & Gebru , 2018 ; Obermeyer et al. , 2019 ) . In addition , explanations are a helpful tool for debugging purpose ( Holte , 1993 ; Freitas , 2014 ; Rudin , 2019 ) . Counterfactuals are closely related to classes of explanation , which much of the early work in AI on explaining the decisions made by an expert or rule-based systems focused on ( Wachter et al. , 2017 ) . As the name implies , counterfactual is an alternative example counter to the fact which takes the form “ If X had not occurred , Y would not have occurred ” . Explicitly establishing the connection between counterfactuals and explanations , Wachter et al . ( 2017 ) concluded that counterfactual explanations can bridge the gap between the interests of data subjects and data controllers that otherwise acts as a barrier to a legally binding right to explanation . They further formulate the problem of finding counterfactual explanations as an optimization task that minimizes the loss of a desired outcome w.r.t . an alternative input combined with regularization for distance to the query example . In a similar vein , the dominant majority of recent counterfactual explanation methods are proposed based on their adapted objective functions for optimization . Unfortunately , these deep learning approaches have some downsides : First of all , they are very time-consuming for generating explanations due to the iterative optimization process , because for every optimization iteration the model needs to be evaluated at least once and it may take a lot of iterations until a candidate is found . Secondly , the objective function is usually highly non-convex , which makes it more tricky to find a satisfying solution in practice . Thirdly , the optimizer involves additional hyperparameters that need to be carefully selected , such as learning rate . Furthermore , although these methods optimize for the ” closest possible world ” by constraining the distance of the counterfactual to the query instance , they are still not aware of the underlying density and the data manifold . As a direct consequence , the counterfactuals often appear rather unnatural and unrealistic although they are as close as possible to the query instance . See figure 3 for example . This imposes difficulties for humans to understand the explanations . As Nickerson ( 1998 ) argues , humans exhibit confirmation bias , meaning that we tend to ignore information that is inconsistent with our prior beliefs . In analogy , unrealistic counterfactuals also have very limited effect for communicating explanations to humans . To improve upon these limitations , we propose a novel approach to generate counterfactual explanations . Unlike the overwhelmingly dominant deep learning approach , we decouple the goal of perturbing the prediction and maintaining an additional constraint on the perturbation . Specifically , we view the task as a two-step process , whereby the first step is to perturb a sample towards a desired class and the second step is to constrain the perturbed sample to be close to the data manifold so it is realistic and natural . The first goal aligns with adversarial attacks ( Goodfellow et al. , 2015 ; Brown et al. , 2017 ; Yuan et al. , 2019 ) , where a small perturbation is computed to perturb the prediction . In the literature there are gradient-based methods ( Kurakin et al. , 2016 ; Moosavi-Dezfooli et al. , 2016 ) and optimization-based methods ( Szegedy et al. , 2013 ; Carlini & Wagner , 2017 ) . Exploiting the fact that gradient corresponds to the direction of steepest ascent , gradient-based perturbation is not only very easy to implement with current deep learning frameworks , but also much faster to compute compared to optimization-based perturbation . Therefore we adopt a gradient-based approach for the first step . To ensure the quality and effectiveness of the second step , we learn a sum-productnetwork ( SPN ) ( Poon & Domingos , 2011 ) , a tractable density model , on the input , and we perturb the example further based on the gradient that directs to steepest ascent of likelihood , evaluated on the SPN . In summary , we make the following contributions : 1 . We propose the first approach for generating counterfactual examples using probabilistic circuits to ensure density-awareness in a tractable way . 2 . We experiment with complex and high-dimensional real-world dataset , which was not dealt with using domain-agnostic methods in the literature . 3 . We give empirical evidence to demonstrate the advantages and effectiveness of our approach : visually appealing examples with high density , fast computation , and high success rate . We proceed as follows : First , we give an overview of the literature and summarize the shortcomings of the existing approaches . Then we introduce RAT-SPNs , preparing for presenting our approach . Finally , we evaluate our approach via empirical evidence and conclude our work . 2 RELATED WORK ON COUNTERFACTUAL EXPLANATIONS . Research attention on counterfactual explanations has been increasingly raised since Wachter et al . ( 2017 ) presented the concept of unconditional counterfactual explanations as a novel type of explanation of automated decisions . Identifying the resemblance between counterfactual explanations and adversarial perturbations , Wachter et al . ( 2017 ) proposed to generate counterfactual explanations based on the optimisation techniques used in the adversarial perturbation literature . Specifically , a loss function is minimized w.r.t . its input , using standard gradient-based techniques , for a desired output and a regularizer penalizing the distance between the counterfactual and the query . Following Wachter et al . ( 2017 ) , Mothilal et al . ( 2020 ) augmented the loss with diversity constraint to encourage diverse solutions . However , these approaches do not have explicit knowledge of the underlying density or data manifold , which may lead to unrealistic counterfactual explanations . A bunch of methods counteract this issue by learning an auxiliary generative model to impose additional density constraints on the optimization process . As common choice for density approximator , Variational Autoencoder ( VAE ) ( Kingma & Welling , 2014 ) and its variants ( Klys et al. , 2018 ; Ivanov et al. , 2019 ) are used . For instance , Dhurandhar et al . 2018 proposed contrastive explanations method ( CEM ) for neural networks based on optimization . The objective function consists of a hinge-like loss function and the elastic net regularizer as well as an auxiliary VAE to evaluate the proximity to the data manifold . Ustun et al . 2019 defined the term recourse as the ability of a person to change the decision of a model by altering actionable input variables . Recourse is evaluated by solving an optimization problem . Joshi et al . 2019 provided an algorithm , called REVISE , to suggest a recourse , based on samples from the latent space of a VAE characterizing the data distribution . Pawelczyk et al . 2020 developed a framework , called C-CHVAE , to generate faithful counterfactuals . C-CHVAE trains a VAE and returns the closest counterfactual due to a Figure 1 : The SPN structure used to estimate a distribution as a mixture of classconditional densities . Each class-conditional density is in turn represented by a subSPN . Classification is done via Bayes ’ rule . nearest neighbour style search in the latent space . Downs et al . 2020 proposed another algorithmic recourse generation method , CRUDS , that generates multiple recourses satisfying underlying structure of the data as well as end-user specified constraints . Based on a VAE-variant , CRUDS uses a Conditional Subspace Variational Autoencoder ( CSVAE ) model ( Klys et al. , 2018 ) that is capable of extracting latent features that are relevant for prediction . Another method called CLUE Antorán et al . ( 2020 ) is proposed for interpreting uncertainty estimates from differentiable probabilistic models using counterfactual explanations , by searching in the latent space of a VAE with arbitrary conditioning ( VAEAC ) ( Ivanov et al. , 2019 ) . Poyiadzi et al . 2020 proposed FACE , a graph-based algorithm to generate counterfactuals that are coherent with the underlying data distribution by constructing a graph over all the candidate targets . Besides , several domain-specific approaches are also emerging Olson et al . ( 2021 ) ; Goyal et al . ( 2019 ) ; Chang et al . ( 2019 ) . The aforementioned techniques have the following shortcomings : 1 . The deep learning-like approach ( Wachter et al. , 2017 ; Mothilal et al. , 2020 ; Dhurandhar et al. , 2018 ; Ustun et al. , 2019 ; Joshi et al. , 2019 ) is too slow to generate explanations on the fly due to the iterative optimization process , which in turn comes with additional tuning parameters . 2 . Although highly expressive , neural density estimators such as VAEs are highly intractable , which makes explicit density constraint on counterfactual explanations infeasible ( Dhurandhar et al. , 2018 ; Ivanov et al. , 2019 ; Klys et al. , 2018 ; Joshi et al. , 2019 ; Pawelczyk et al. , 2020 ) . In addition , they come with the overhead of latent representation , which is not necessarily in good quality when maximum likelihood training is used to learn them ( Alemi et al. , 2018 ; Dai & Wipf , 2019 ) . We improve upon the aforementioned shortcomings by proposing the first counterfactual method using tractable probabilistic circuits , specifically sum-product networks ( SPNs ) ( Darwiche , 2003 ; Poon & Domingos , 2011 ) . 3 GRADIENT-BASED COUNTERFACTUAL EXPLANATIONS USING TRACTABLE PROBABILISTIC MODELS . Before explaining our idea , we first introduce the tractable probabilistic models we use . An sum-product network ( SPN ) S over X is a tractable probabilistic model for P ( X ) based on a directed acyclic graph ( DAG ) . This graph indicates computation for probabilistic inference and consists of three types of nodes : univariate leaf nodes , sum nodes , and product nodes . Let ch ( · ) denote the children of a node . A sum node S is weighted sum of its children , i.e . S = ∑ N∈ch ( S ) wS , NN where the weights wS , N are non-negative and sum to 1 , i.e . wS , N ≥ 0 , ∑ N wS , N = 1 . Sum nodes can be viewed as mixtures of their child distributions . A product node P is product of its children , i.e . P = ∏ N∈ch ( P ) N . The root node represents P ( X ) . Products are factorized distributions , implying independence assumption among their children . SPNs allow for fast , exact inference on high-treewidth models . Unlike most of the probabilistic deep learning approaches , SPNs permit exact and efficient inference . Specifically , they are able to compute any marginalization and conditioning query in time linear of the model ’ s representation size . By employing SPNs for deep learning , random and tensorized SPNs ( RAT-SPNs ) ( Peharz et al. , 2020 ) are proposed using a simple approach to construct random SPN structure and combine it with GPU-based optimization . It is worth noting that RAT-SPNs are not fooled by certain out-of-domain image detection tests on which VAEs , normalizing flows ( NFs ) , and auto-regressive density estimators ( ARDEs ) consistently fail ( Choi & Jang , 2018 ; Nalisnick et al. , 2018 ) . To learn the parameters ω of a given RAT-SPN structure S in generative setting to approximate a distribution P ∗ ( X ) , we assume i.i.d . samples X = { x1 , . . . , xN } are given . Then maximum likelihood estimation is employed , i.e . ω = argmax 1N ∑N n=1 logS ( xn ) , where S ( x ) is a distribution over X represented by the RAT-SPN S . Apart from the standard use for density estimation , RAT-SPNs can be used as a generative classifier as well . Consider a classification problem f : Rd → { 1 , . . . , C } with C labels , C roots are used to represent class-conditional densities Sc ( X ) = : S ( X|Y = y ) . The overall density distribution is then given by S ( X ) = ∑ y S ( X|y ) P ( y ) . See figure 1 for illustration . Bayes ’ rule is used to classify a sample x : S ( Y |x ) = S ( x|Y ) P ( Y ) S ( x ) . In other words , a RAT-SPN of this special structure has dual use : It is both a density estimator S ( X ) and a classifier S ( Y |X ) . We will use this model to demonstrate our approach because it can be used for classification and yield tractable density evaluation for free at the same time . However , our approach can be easily extended to deep neural classifiers by training an auxiliary RAT-SPN for density estimation . For more details on RAT-SPN , check out appendix . Our approach is defined as two serial perturbations : In the first step , we maximize log S ( y ′|x ) S ( y|x ) to induce desired outcome y′ , which is equivalent to maximizing log S ( x|y ′ ) S ( x|y ) since log S ( y′|x ) S ( y|x ) = log ( S ( x|y′ ) P ( y′ ) S ( x ) S ( x ) S ( x|y ) P ( y ) ) = log S ( x|y′ ) S ( x|y ) , when assuming a uniform class prior . Towards this end , we take a gradient step towards the steepest ascent of the counterfactual outcome . This results in the perturbed example u where u = x+ ∂ ( logS ( x|y′ ) − logS ( x|y ) ) ∂x ∗ 1 . This step is similar to generating an adversarial perturbation in that the sample u is expected to change the prediction to y′ from y with a very slight change . However , without additional constraints , the perturbed sample u is very likely to deviate from the underlying data manifold . In order to generate in-distribution counterfactuals , we maximize the density P ∗ ( u ) of the current sample u , which is approximated via the SPN S ( u ) . In contrast to the density estimators used primarily in the literature , SPNs have the merit that they allow for tractable density evaluation . To maximize the density S ( u ) , we take another gradient step towards its steepest ascent , i.e . x′ = u+ ∂S ( u ) ∂u ∗ 2. x′ is the final counterfactual explanation for x that yields y′ instead of y . See figure 2 for illustration .

Summary: Explaining in a human-understandable way the predictions made by machine learning models is an important research topic for the development of trustworthy systems. In this topic, the present paper focuses on counterfactual explanations, a specific class of explanations that provides a link between what could have happened if the input to a model had been changed.   The key idea of this paper is to exploit Sum-Product Networks (SPNs), a class of machine learning models capable of providing counterfactual explanations endowed with appealing computational properties. Based on those models, the authors present a conceptually simple two-stage gradient-based algorithm for computing counterfactual explanations. This approach is empirically compared with baseline techniques on several benchmarks.