CLEAR: Generative Counterfactual Explanations on Graphs
Anonymous Author(s)
AffiliationAddress email
Abstract
Counterfactual explanations promote explainability in machine learning models byanswering the question “ how should the input instance be perturbed to obtain adesired predicted label? ".
The comparison of this instance before and after perturbation can enhance human interpretation.
Most existing studies on counterfactualexplanations are limited in tabular data or image data.
In this paper, we studythe problem of counterfactual explanation generation on graphs.
A few studieshave explored counterfactual explanations on graphs, but many challenges of thisproblem are still not well-addressed: 1) optimizing in the discrete and disorganizedspace of graphs; 2) generalizing on unseen graphs; 3) maintaining the causalityin the generated counterfactuals without prior knowledge of the causal model.
Totackle these challenges, we propose a novel framework CLEAR which aims togenerate counterfactual explanations on graphs for graph-level prediction models.
Specifically, CLEAR leverages a graph variational autoencoder based mechanismto facilitate its optimization and generalization, and promotes causality by leveraging an auxiliary variable to better identify the causal model.
Extensive experimentson both synthetic and real-world graphs validate the superiority of CLEAR overstate-of-the-art methods in different aspects.
Introduction
To facilitate explainability in opaque machine learning (ML) models (e g., how predictions are madeby a model and what to do to achieve a desired outcome), explainable artificial intelligence (XAI)[1] has recently attracted significant attention in many communities.
Among existing work of XAI,a special class counterfactual explanation (CFE) [2] promotes model explainability by answeringthe following question: “ how should the input features X be slightly perturbed to new features X ′to obtain a different predicted label (often a desired label) for a specific instance? ".
The originalinstance whose prediction needs to be explained is called an explainee instance , and the generatedinstances after perturbation are referred to as “counterfactual explanations".
Generally, CFE promoteshuman interpretation through the comparison between X and X ′ .
With its intuitive nature, CFEscan be deployed in various real-world scenarios such as loan application and legal framework [3].
Different from traditional CFE studies [2, 3, 4, 5] on tabular or image data, recently, CFE on graphsis also an emerging field with applications in many graph structure related domains such as molecularanalysis [6] and career networking [7].
For example, consider a grant application prediction [8]model with each input instance as a graph representing a research team’s collaboration network, whereeach node represents a team member, and each edge represents a collaboration between them.
Teamleaders can improve their teams for next application by changing the original graph according to thecounterfactual with a desired predicted label (application being granted).
If the counterfactual is more dense than the original, the team leader may then encourage more team collaborations.
In this work,we investigate the problem of generating counterfactual explanations on graphs.
As shown in Fig.
1,given a ML prediction model f on graphs, for a graph instance G , we aim to generate counterfactuals(e.g. , G CF ) which are slightly different from G w.r.t.
their node features or graph structures to elicita desired model prediction.
Specifically, we focus on graph-level prediction without any assumptionsof the prediction model type and its model access, i.e., f can be a black box with unknown structure.
[Figure]
Teamculture
Grant [Figure] # of collaborations
[Figure]
# +'(
[Figure]
Causality GraphMLmodel
[Figure] [Figure] [Figure]
Original
[Figure]
×
[Figure]
Recently, a few studies [9, 6, 10, 11, 12, 13] exploreto extend CFEs into graphs.
However, this problem
[Figure]
still remains a daunting task due to the following key
[Figure]
challenges: 1) Optimization: Different from tradi-
[Figure]
tional data, the space of perturbation operations on
[Figure]
graphs (e.g., add/remove nodes or edges) is discrete,
[Figure]
disorganized, and extraordinarily vast, which bringsdifficulties for optimization in CFE generation.
Mostexisting methods [6, 11] search for graph counterfactuals by enumerating all the possible perturbationoperations on the current graph.
However, such enumeration on graphs is of high complexity, and it isalso challenging to solve an optimization problem in such a complex search space.
2) Generalization: The discrete and disorganized nature of graphs also brings challenges for the generalization of CFEmethods on unseen graphs, as it is hard to sequentialize the process of graph CFE generation andthen generalize it.
Most existing CFE methods on graphs [9, 12] solve an optimization problem foreach explainee graph separately.
These methods, however, cannot be generalized to new graphs.3)
Causality: It is challenging to generate counterfactuals that are consistent with the underlyingcausality.
Specifically, causal relations may exist among different node features and the graph structure.
In the aforementioned example, for each team, after establishing more collaborations, the teamculture may be causally influenced.
Incorporating causality can generate more realistic and feasiblecounterfactuals [4], but most existing CFE methods either cannot handle causality, or require toomuch prior knowledge about the causal relations in data.
To address the aforementioned challenges, in this work, we propose a novel framework — generative
C ounterfactua L E xpl
A nation gene R ator for graphs (CLEAR).
At a high level, CLEAR is a generative,model-agnostic CFE generation framework for graph prediction models.
For any explainee graphinstance, CLEAR aims to generate counterfactuals with slight perturbations on the explainee graphto elicit a desired predicted label, and the counterfactuals are encouraged to be in line with theunderlying causality.
More specifically, to facilitate the optimization of the CFE generator, wemap each graph into a latent representation space, and output the counterfactuals as a probabilisticfully-connected graph with node features and graph structure similar as the explainee graph.
Inthis way, the framework is differentiable and enables gradient-based optimization.
To promotegeneralization of the CFE generator on unseen graphs, we propose a generative way to constructthe counterfactuals.
After training the CFE generator, it can be efficiently deployed to generate(multiple) counterfactuals on unseen graphs, rather than retraining from scratch.
To generate morerealistic counterfactuals without explicit prior knowledge of the causal relations, inspired by therecent progress in nonlinear independent component analysis (ICA) [14] and its connection withcausality [15], we make an exploration to promote causality in counterfactuals by leveraging anauxiliary variable to better identify the latent causal relations.
The main contributions of this work canbe summarized as follows: 1) Problem.
We study an important problem: counterfactual explanationgeneration on graphs.
We analyze its challenges including optimization, generalization, and causality.
To the best of our knowledge, this is the first work jointly addressing all these challenges of thisproblem.
2) Method.
We propose a novel framework CLEAR to generate counterfactual explanationsfor graphs.
CLEAR can generalize to unseen graphs, and promotes causality in the counterfactuals. 3)
Experiments.
We conduct extensive experiments on both synthetic and real-world graphs to validatethe superiority of our method over state-of-the-art baselines of graph CFE generation.
Preliminaries
A graph G =
( X, A ) is specified with its node feature X and adjacency matrix A .
We have a graphprediction model f : G → Y , where G and Y represent the space of graphs and labels, respectively.
In this work, we assume that we can access the prediction of f for any input graph, but we do notassume the access of any knowledge of the prediction model itself.
For a graph G ∈ G , we denote theoutput of the prediction model as Y = f
( G ) .
A counterfactual G CF =
( X CF , A CF ) is expectedto be similar as the original explainee graph G , but the predicted label for G CF made by f (i.e.,
Y CF =
f ( G CF ) ) should be different from f ( G ) .
With a desired label Y ∗ (here Y ∗ ̸ = Y ), thecounterfactual G CF is considered to be valid if and only if Y ∗ =
Y CF .
In this paper, we mainlyfocus on graph classification, but our framework can also be extended to other tasks such as nodeclassification, as discussed in Appendix D.
Suppose we have a set of graphs sampled from the space G , and different graphs may have differentnumbers of nodes and edges.
A counterfactual explanation generator can generate counterfactualsfor any input graph G w.r.t.
its desired predicted label Y ∗ .
As aforementioned, most existing CFEmethods on graphs have limitations in three aspects: optimization, generalization, and causality.
Next,we provide more background of causality.
The causal relations between different variables (e.g., nodefeatures, degree, etc.) in the data can be described with a structural causal model (SCM):
Definition 1.
(Structural Causal Model)
A structural causal model (SCM) [15] is denoted by atriple ( U, V, F ) : U is a set of exogenous variables, and V is a set of endogenous variables.
Thestructural equations F = { F 1 , ..., F | V | } determine the value for each V i ∈ V with V
i
=
F i ( PA i , U i ) ,here PA i ⊆
V \ V i denotes the “parents" of V i , and U i ⊆
U .
Definition 2.
(Causality in CFE)
For an explainee graph G , a counterfactual G CF satisfies causalityif the change from G to G CF is consistent with the underlying structural causal model.
GraphMLmodel !
(#) =( (# '( ) =
Differentpredicted labels
Original
Counterfactualexplanation × √ GraphMLmodel !
[Figure]
Samepredictionmodel
Teamculture
[Figure] collaborations
[Figure]
GraphMLmodel !
(#) =(
(# '( ) =
Differentpredicted labels
Original
Counterfactual × √ GraphMLmodel !
Counterfactual explanationgenerator * Samepredictionmodel
GraphMLmodel !
(#) =(
(# '( ) =
Differentpredicted labels
Original
Counterfactual × √ GraphMLmodel !
Counterfactual explanationgenerator * Samepredictionmodel
Example:
In the aforementioned grant applicationexample, for a research team which has been rejectedfor an application before (as the graph G in Fig.
2),to get the next application approved, a valid counterfactual may suggest this team to improve the numberof collaborations between team members.
Based onreal-world observations, we assume that an additionalcausal relation exists: the number of team collaborations causally affects the team culture.
For example, for the same team, if more collaborationshad been established, then the team culture should have been improved in terms of better memberengagement and respect for diversity.
The SCM is illustrated in Fig.
2, where we leave out theexogenous variables for simplicity.
Although the team culture usually does not affect the result ofgrant application, the counterfactuals with team culture changed correspondingly when the numberof collaborations changes are more consistent with the ground-truth SCM.
G CFin Fig.
2 showsan example of such counterfactuals.
In contrast, if a counterfactual improves a team’s number ofcollaborations alone without improving the team culture (see G CFin Fig. 2), then it violates thecausality.
As discussed in [4], traditional CFE methods often optimize on a single instance, and areprone to perturb different features independently, thus they often fail to satisfy the causality.
The Proposed Framework — CLEAR
In this section, we describe a novel generative framework — CLEAR, which addresses the problemof counterfactual explanation generation for graphs.
First, we introduce its backbone CLEAR-VAEto enable optimization on graphs and generalization on unseen graph instances.
This backbone isbased on a graph variational auto-encoder (VAE) [16] mechanism.
On top of CLEAR-VAE, we thenpromote the causality of CFEs with an auxiliary variable.
3.1 CLEAR-VAE: Backbone of Graph Generative Counterfactual Explanations
Different from most existing methods [9, 12] for CFE generation on graphs which focus on a singlegraph instance, CLEAR is based on a generative backbone CLEAR-VAE which can efficiently [Figure] Perturbed [Figure]
KL (8 ∥ :)
[Figure] Encoder Decoder [Figure]
Sampling
Blackbox 8 :
Inserted edges
Edges
Deleted edges generate CFEs for different graphs after training, even for the graphs that do not appear in the trainingdata.
As shown in Fig.
3, CLEAR-VAE follows a traditional graph VAE [16] architecture with anencoder and a decoder.
The encoder maps each original graph G =
( X, A ) into a latent space as arepresentation Z , then the decoder generates a counterfactual G CF based on the latent representation
Z .
Following the VAE mechanism as in [16, 17] and its recent application in CFE generation fortabular data [4, 18], the optimization objective is based on the evidence lower bound (ELBO), whichis a lower bound for the log likelihood ln P ( G CF | Y ∗ , G ) .
Here, P ( G CF | Y ∗ , G ) is the probabilityof the generated counterfactual G CF conditioned on an input explainee graph G and a desired label
Y ∗ .
The ELBO for CLEAR-VAE can be derived as follows:
[Equation]
where Z is the graph representation in the bottleneck layer for G , which can be considered as a set oflatent variables.
Q refers to the approximate posterior distribution Q ( Z | G, Y ∗ ) , and KL ( ·||· ) meansthe Kullback-Leibler (KL) divergence.
The first term P ( G CF | Z, Y ∗ , G ) denotes the probability ofthe generated counterfactual conditioned on the representation Z and the desired label Y ∗ .
Due tothe lack of ground-truth counterfactuals, it is hard to directly optimize this term.
But inspired by [4],maximizing this term can be considered as generating valid graph counterfactuals w.r.t.
the desiredlabel Y ∗ , thus we replace this term with − E Q [ d ( G, G CF ) + α · l ( f ( G CF ) , Y ∗ )] , where d ( · , · ) is asimilarity loss, which is a distance metric to measure the difference between G and G CF , l ( · ) is thecounterfactual prediction loss to measure the difference between the predicted label f
( G CF ) and thedesired label Y ∗ .
In summary, these two terms encourage the model to output counterfactuals whichare similar as the input graph but elicit the desired predicted label.
α is a hyperparameter to controlthe weight of the counterfactual prediction loss.
Overall, the loss function of CLEAR-VAE is:
[Equation]
Encoder.
In the encoder, the input includes node features X and graph structure A of the explaineegraph G , as well as the desired label Y ∗ , the output is latent representation Z .
The encoder learns thedistribution Q ( Z | G, Y ∗ ) .
We use a Gaussian distribution P ( Z | G, Y ∗ )
=
N ( µ z ( Y ∗ ) , diag ( σ 2 z ( Y ∗ )))as prior, and enforce the learned distribution Q ( Z | G, Y ∗ ) to be close to the prior by minimizingtheir KL divergence.
Here, µ z ( Y ∗ ) and diag ( σ 2 z ( Y ∗ )) are mean and diagonal covariance of theprior distribution learned by a neural network module.
Z is sampled from the learned distribution
Q ( Z | G, Y ∗ ) with the widely-used reparameterization trick [17].
Decoder.
In the decoder, the input includes Z and Y ∗ , while the output is the counterfactual
G CF =
( X CF , A CF ) .
Different counterfactuals can be generated for one explainee graph bysampling Z from Q ( Z | G, Y ∗ ) for multiple times.
The adjancency matrix is often discrete, andtypically assumed to include only binary values ( A ( i,j )
= 1 if edge from node i to node j exists,otherwise A ( i,j ) = 0 ).
To facilitate optimization, inspired by recent graph generative models [9, 16],our decoder outputs a probabilistic adjacency matrix ˆ A CF with elements in range [0 , 1] , and thengenerates a binary adjancency matrix A CF by sampling from Bernoulli distribution with probabilitiesin ˆ A CF .
We calculate the similarity loss in Eq.
(2) as:
[Equation]
where d A and d X are metrics to measure the distance between two graphs w.r.t.
their graph structuresand node features, respectively.
β controls the weight for the similarity loss w.r.t.
node features.
More details of model implementation are in Appendix B.
3.2 CLEAR: Improving the Causality in Counterfactual Explanations
To further incorporate the causality in the generated CFEs, most existing studies [4, 19, 20] leveragecertain prior knowledge (e.g., a given path diagram which depicts the causal relations among variables)of the SCM.
However, it is often difficult to obtain sufficient prior knowledge of the SCM in real-worlddata, especially for graph data.
In this work, we do not assume the access of the prior knowledge of
SCM, and only assume that the observational data is available.
However, the key challenge, as shownin [19], is that it is impossible to identify the ground-truth SCM from observational data withoutadditional assumptions w.r.t.
the structural equations and the exogenous variables, because different
SCMs may result in the same observed data distribution.
Considering that different SCMs cangenerate different counterfactuals, the identifiability of SCM is an obstacle of promoting causality in
CFE.
Fortunately, enlightened by recent progress in nonlinear independent component analysis (ICA)[14, 21], we make an initial exploration to promote causality in CFE by improving the identifiabilityof the latent variables in our CFE generator with the help of an auxiliary observed variable.
This CFEgenerator is denoted by CLEAR.
In nonlinear independent component analysis (ICA) [14, 21, 22, 23], it is assumed that the observeddata, e.g., X , is generated from a smooth and invertible nonlinear transformation of independentlatent variables (referred to as sources )
Z .
Identifying the sources and the transformation are the keygoals in nonlinear ICA.
Similarly, traditional VAE models also assume that the observed features Xare generated by a set of latent variables Z .
However, traditional VAEs cannot be directly used fornonlinear ICA as they lack identifiability, i.e., we can find different Z that lead to the same observeddata distribution p ( X ) .
Recent studies [14] have shown that the identifiability of VAE models canbe improved with an auxiliary observed variable S (e.g., a time index or class label), which enablesus to use VAE for nonlinear ICA problem.
As discussed in [22, 23], a SCM can be consideredas an nonlinear ICA model if the exogenous variables in the SCM are considered as the sourcesin nonlinear ICA.
Similar connections can be built between the structural equations in SCM
andthe transformations in ICA.
Such connections shed a light on improving the identifiability of theunderlying SCM without much explicit prior knowledge of the SCM.
With this idea, based on the backbone CLEAR-VAE, CLEAR improves the causality in counterfactualsby promoting identifiability with an observed auxiliary variable S .
Intuitively, we expect the graph
VAE can capture the exogenous variables of the SCM in its representations Z , and approximatethe data generation process from the exogenous variables to the observed data, which is consistentwith the SCM.
Here, for each graph, the auxiliary variable S can provide additional information for
CLEAR to better identify the exogenous variables in the SCM, and thus elicit counterfactuals withbetter causality.
To achieve this goal, following the previous work of nonlinear ICA [14, 21], wemake the following assumption:
Assumption 1.
We assume that the prior on the latent variables P ( Z | S ) is conditionally factorial.
With this assumption, the original data can be stratified by different values of S , and each separateddata stratum can be considered to be generated by the ground-truth SCM under certain constraints(e.g.
, the range of values that the exogenous variables can take).
When the constraints become morerestricted, the space of possible SCMs that can generate the same data distribution in each datastratum shrinks.
In this way, the identification of the ground-truth SCM can be easier if we leveragethe auxiliary variable S .
Here, with the auxiliary variable S , we infer the ELBO of CLEAR:
Theorem 1.
The evidence lower bound (ELBO) to optimize the framework CLEAR is:
[Equation]
the detailed proof is shown in Appendix A.
Loss function of CLEAR.
Based on the above ELBO, the final loss function of CLEAR is:
[Equation]
Encoder and Decoder.
The encoder takes the input G , S , and Y ∗ , and outputs Z as the latentrepresentation.
We use a Gaussian prior P ( Z | G, S, Y ∗ ) =
N ( µ z ( S, Y ∗ ) , diag ( σ 2 Z ( S, Y ∗ )) with itsmean and diagonal covariance learned by neural network, and we encourage the learned approximateposterior Q ( Z | G, S, Y ∗ ) to approach the prior by minimizing their KL divergence.
Similar tothe backbone CLEAR-VAE, the decoder takes the inputs Z and Y ∗ to generate one or multiplecounterfactuals G CF for each explainee graph.
More implementation details are in Appendix B.
Experiment
In this section, we evaluate our framework CLEAR 1 with extensive experiments on both syntheticand real-world graphs.
In particular, we answer the following research questions in our experiments:
RQ1: How does CLEAR perform compared to state-of-the-art baselines?
RQ2: How do differentcomponents in CLEAR contribute to the performance?
RQ3: How can the generated CFEs promotemodel explainability?
RQ4: How does CLEAR perform under different settings of hyperparameters?
4.1 Baselines
We use the following baselines for comparison: 1) Random :
For each explainee graph, randomlyperturb the graph structure for at most T steps.
Stop if a desired predicted label is achieved. 2)
EG-IST :
For each explainee graph, randomly insert edges into it for at most T steps. 3) EG-RM :
For each explainee graph, randomly remove edges from it for at most T steps. 4) GNNExplainer :
GNNExplainer [12] is proposed to identify the most important subgraphs for prediction.
We applyit for CFE generation by removing the important subgraphs identified by GNNExplainer.
5) CFGNNExplainer : CF-GNNExplainer [9] is proposed for generating counterfactual ego networks innode classification tasks.
We adapt CF-GNNExplainer for graph classification by taking the wholegraph (instead of the ego network of any specific node) as input, and optimizing the model until thegraph classification (instead of node classification) label has been changed to the desired one. 6) MEG : MEG [6] is a reinforcement learning based CFE generation method.
In all the experiments,we set T = 150 .
More details of the baseline setup can be referred in Appendix B.
4.2 Datasets
We evaluate our method on three datasets, including a synthetic dataset and two datasets with realworld graphs.
(1) Community.
This dataset contains synthetic graphs generated by the Erdös-Rényi(E-R) model [24].
In this dataset, each graph consists of two 10 -node communities.
The label Y isdetermined by the average node degree in the first community (denoted by deg 1 ( A ) ).
According to thecausal model (in Appendix B), when deg 1 ( A ) increases (decreases), the average node degree in thesecond community deg 2 ( A ) should decrease (increase) correspondingly.
We take this causal relationdeg 1 ( A ) → deg 2 ( A ) as our causal relation of interest, and denote it as R .
Correspondingly, we definea causal constraint for later evaluation of causality: “ ( deg 1 ( A CF ) > deg 1 ( A ))
⇒ ( deg 2 ( A CF ) <
deg 2 ( A ))”
OR “ ( deg 1 ( A CF ) <
deg 1 ( A ))
⇒ ( deg 2 ( A CF ) > deg 2 ( A )) ".
(2) Ogbg-molhiv.
In thisdataset, each graph stands for a molecule, where each node represents an atom, and each edge is achemical bond.
As the ground-truth causal model is unavailable, we simulate the label Y and causalrelation of interest R .
(3) IMDB-M.
This dataset contains movie collaboration networks from IMDB.
In each graph, each node represents an actor or an actress, and each edge represents the collaborationbetween two actors or actresses in the same movie.
Similarly as the above datasets, we simulate thelabel Y and causal relation of interest R , and define causal constraints corresponding to R .
It is worthmentioning that the causal relation of interest R in the three datasets covers different types of causalrelations respectively: i) causal relations between variables in graph structure A ; ii) between variablesin node features X ; iii) between variables in A and in X .
Thereby we comprehensively evaluate theperformance of CLEAR in leveraging different modalities (node features and graph structure) ofgraphs to fit in different types of causal relations.
More details about datasets are in Appendix B.
4.3 Evaluation Metrics
Validity: the proportion of counterfactuals which obtain the desired labels.
[Equation]
[Table]
where N is the number of graph instances, N CF is the number of counterfactuals generated for eachgraph.
G CF ( i,j ) =
( X CF ( i,j ) , A CF ( i,j ) ) denotes the j -th counterfactual generated for the i -th graph instance.
Here, y ∗ i is the realization of Y ∗ for the i -th graph. 1 ( · ) is an indicator function which outputs 1when the input condition is true, otherwise it outputs 0 .
Proximity: the similarity between the generated counterfactuals and the input graph.
Specifically,we separately evaluate the proximity w.r.t.
node features and graph structure, respectively.
[Equation]
where we use cosine similarity for sim X ( · ) , and accuracy for sim A ( · ) .
Causality: As it is difficult to obtain the true SCMs for real-world data, we focus on the causalrelation of interest R mentioned in dataset description.
Similarly as [4], we measure the causality byreporting the ratio of counterfactuals which satisfy the causal constraints corresponding to R .
Time: the average time cost (seconds) of generating a counterfactual for a single graph instance.
4.4 Setup
We set the desired label Y ∗ for each graph as its flipped label (e.g., if Y = 0 , then Y ∗ = 1 ).
For eachgraph, we generate three counterfactuals for it ( N CF = 3 ).
Other setup details are in Appendix B.
4.5 RQ1: Performance of Different Methods
To evaluate our framework CLEAR, we compare its CFE generation performance against the stateof-the-art baselines.
From the results in Table 1, we summarize the main observations as follows:
1) Validity and proximity.
Our framework CLEAR achieves good performance in validity and proximity.
This observation validates the effectiveness of our method in achieving the basic target
of CFE generation.
a) In validity, CLEAR obviously outperforms all baselines on most datasets.
Random, EG-IST, and EG-RM perform worst due to their random nature; GNNExplainer can onlyremove edges and nodes, which also limits its validity; CF-GNNExplainer and MEG perform wellas their optimization is designed for CFE generation; b) In Proximity A , CLEAR outperforms allnon-random baselines.
EG-RM performs best in Proximity A because most graphs are very sparse,thus only removing edges can change the graph relatively less than other methods.
As the baselineseither cannot perturb node features, or their perturbation approach on node features cannot fit well inour setting, we do not compare Proximity X with them.
2) Time.
CLEAR significantly outperformsall baselines in time efficiency.
Most of the baselines generate CFEs in an iterative way, and MEGneeds to enumerate all perturbations at each step.
GNNExplainer and CF-GNNExplainer optimize
4 degree of community 1    d e g r ee o f c o mm un i t y      (a) Original data  4 degree of community 1    d e g r ee o f c o mm un i t y      (b) CFEs from CLEAR-VAE  4 degree of community 1    d e g r ee
o f c o mm un i t y      (c) CFEs from CLEAR on every single instance, which limits their generalization.
All the above reasons erode their timeefficiency.
While in our framework, the generative mechanism enables efficient CFE generation andgeneralization on unseen graphs, thus brings substantial improvement in time efficiency.
3) Causality.
CLEAR dramatically outperforms all baselines in causality.
We contribute the superiority of ourframework w.r.t.
causality in two key factors: a) different from some baselines (e.g., GNNExplainer)optimized on each single graph, our framework can better capture the causal relations among differentvariables in data by leveraging the data distribution of the training set; b) our framework utilizes theauxiliary variable to better identify the underlying causal model and promote causality.
4.6 RQ2: Ablation Study
[Figure]
To evaluate the effectiveness of different compo- [Figure] nents in CLEAR, we conduct ablation study with [Figure]
the following variants: 1) CLEAR-NC .
In thisvariant, we remove the counterfactual predictionloss; 2) CLEAR-NPA , we remove the similarityloss w.r.t.
graph structure; 3) CLEAR-NPX , weremove the similarity loss w.r.t.
node features; 4)
CLEAR-NP , we remove all the similarity loss; 5) CLEAR-VAE , the backbone of our framework.
As shown in Fig.
5, we have the following observations: 1)
The validity of CLEAR-NC degradesdramatically due to the lack of counterfactual prediction loss; 2)
The performance w.r.t.
proximity isworse in CLEAR-NPA, CLEAR-NPX, and CLEAR-NP as the similarity loss is removed.
Besides,removing the similarity loss can also hurt the performance of causality when the variables in thecausal relation of interest R are involved.
For example, in Community, CLEAR-NPA performsmuch worse in causality (as R in Community involves node degree in graph structure), while in
Ogbg-molhiv, the performance in causality of CLEAR-NPX is eroded (as R on Ogbg-molhiv involvesnode features); 3)
The performance w.r.t.
causality is impeded in CLEAR-VAE.
This observationvalidates the effectiveness of the auxiliary variable for promoting causality.
Similar observations canalso be found in the ablation study on the IMDB-M dataset, which is shown in Appendix C.
4.7 RQ3: Explainability through CFEs
To investigate how CFE on graphs promote model explainability, we take a closer look in the generatedcounterfactuals.
Due to the space limit, we only show our investigation on the Community dataset.
Fig. 4(a) shows the distribution of two variables: the average node degree in the first communityand in the second community in the original dataset, i.e., deg 1 ( A ) and deg 2 ( A ) .
Fig.
4(b) shows thedistribution of these two variables in counterfactuals generated by CLEAR-VAE.
We observe thatthese counterfactuals are distributed close to the decision boundary, i.e., deg 1 ( A ) =
ADG 1 , where
ADG 1 is a constant around 2.
This is because that the counterfactuals are enforced to change theirpredicted labels with perturbation as slight as possible.
Fig. 4(c) shows the distribution of these twovariables deg 1 ( A ) and deg 2 ( A ) in counterfactuals generated by CLEAR.
Different colors denotedifferent values of the auxiliary variable S .
Notice that based on the causal model (in Appendix
B), the exogenous variables are distributed in a narrow range when the value of S is fixed, thus thesame color also indicates similar values of exogenous variables.
We observe that compared with thecolor distribution in Fig. 4(b), the color distribution in Fig. 4(c) is more consistent with Fig. 4(a).
This indicates that compared with CLEAR-VAE, CLEAR can better capture the values of exogenousvariables, and thus the counterfactuals generated by CLEAR are more consistent with the underlying causal model.
To better illustrate the explainability provided by CFE, we further conduct case studiesto compare the original graphs and their counterfactuals in Appendix C.
[Figure]
To evaluate the robustness of our method,
[Figure]
we test the model performance under different settings of hyperparameters.
We vary [Figure] α ∈ { 0 . 01 , 0 . 1 , 1 . 0 , 5 . 0 , 10 . 0 } and β ∈ [Figure]
Due to the space [Figure] limit, we only show the parameter study on Ogbgmolhiv, but similar observations can be found inother datasets.
As shown in Fig.
6, the selection of α and β controls the tradeoff between differentgoals in the objective function.
But generally speaking, the proposed framework is not very sensitiveto the hyparameter setting.
More studies regarding other parameters are in Appendix C.
Related Work
Counterfactual explanations on tabular data.
Counterfactual explanations have attracted increasingattention [2, 3, 5, 25].
Especially, recent works also consider more aspects in CFEs, such asactionability [26, 27], sparsity [28, 29, 30], data manifold closeness [31, 32], diversity [29, 33],feasibility [4, 27], causality [4, 19, 20], and amortized inference [4].
These methods include modelagnostic [27, 30, 32] methods and model-accessible [2, 25] methods.
Recently, a few studies [4, 10]develop generative CFE generators based on variational autoencoder.
However, most of current CFEapproaches only focus on tabular or image datasets, and can not be directly grafted for the graph data.
Counterfactual explanations on graphs.
There have been a few studies related to CFEs on graphs[9, 6, 10, 11, 12, 13].
Among them, GNNExplainer [12] identifies subgraphs which are importantfor graph neural network (GNN) model prediction [34, 35, 36, 37, 38, 39, 40].
It can be adapted togenerate counterfactuals by perturbing the identified important subgraphs.
Similarly, RCExplainer[10] generates CFEs by removing important edges from the original graph, but it is based on anassumption that GNN prediction model is partially accessible.
CF-GNNExplainer [9] studies oncounterfactual explanations for GNN models in node classification tasks.
Besides, a few domainspecific methods [6, 11] are particularly designed for certain domains such as chemistry and braindiagnosis.
However, all these above methods are either limited in a specific setting (e.g., the predictionmodel is accessible), or heavily based on domain knowledge.
Many general and important issues inmodel-agnostic CFE generation on graphs still lack exploration.
Graph generative models.
Many efforts [16, 41, 42] have been made in graph generative modelsrecently.
Among them, GraphVAE [16] develops a VAE-based mechanism to generate graphsfrom continuous embeddings.
GraphRNN [41] is an autoregressive generative model for graphs.
It generates graphs by decomposing the graph generation process into a sequence of node andedge formations.
Furthermore, a surge of domain-specific graph generative models [43, 44, 45] aredeveloped with domain knowledge incorporated.
Although different from CFE methods in their goals,graph generative models can serve as the cornerstone of our work for CFE generation on graphs.
Ourframework can be compatible with techniques in different graph generative models.
Conclusion
In this paper, we study on an important problem of counterfactual explanations on graphs.
Morespecifically, we aim to facilitate the optimization, generalization, and causality in CFE generationon graphs.
To address this problem, we propose a novel framework CLEAR, which use a graphvariational autoencoder mechanism to enable efficient optimization in graph data, and generalizationto unseen graphs.
Furthermore, we promote the causality in counterfactuals by improving the modelidentifiability with the help of an auxiliary observed variable.
Extensive experiments are conducted tovalidate the superiority of the proposed framework in different aspects.
In the future, more propertiesof the counterfactuals in graphs, such as diversity, data manifold closeness can be considered.
Besides,incorporating different amount and types of prior knowledge regarding the causal models into CFEgeneration on graphs is also an interesting direction.