Explaining Graph Neural Networks with Large Language Models: A Counterfactual Perspective on Molecule Graphs (2024)

Abstract: In recent years, Graph Neural Networks (GNNs) have become successful in molecular property prediction tasks such as toxicity analysis. However, due to the black-box nature of GNNs, their outputs can be concerning in high-stakes decision-making scenarios, e.g., drug discovery. Facing such an issue, Graph Counterfactual Explanation (GCE) has emerged as a promising approach to improve GNN transparency. However, current GCE methods usually fail to take domain-specific knowledge into consideration, which can result in outputs that are not easily comprehensible by humans. To address this challenge, we propose a novel GCE method, LLM-GCE, to unleash the power of large language models (LLMs) in explaining GNNs for molecular property prediction. Specifically, we utilize an autoencoder to generate the counterfactual graph topology from a set of counterfactual text pairs (CTPs) based on an input graph. Meanwhile, we also incorporate a CTP dynamic feedback module to mitigate LLM hallucination, which provides intermediate feedback derived from the generated counterfactuals as an attempt to give more faithful guidance. Extensive experiments demonstrate the superior performance of LLM-GCE. Our code is released on https://github.com/YinhanHe123/new_LLM4GNNExplanation.

## introduction molecular property prediction has attracted increasing attention in recent years, where graph neural networks (gnns) have achieved significant success in the related downstream tasks, such as drug discovery @xcite and toxicity analysis @xcite . however, gnns are typically considered as black-box models, making it difficult for users to understand how a given prediction is derived. such a lack of explainability brings obstacles against their broader real-world applications to understand molecular properties. facing such an issue, a series of approaches have been proposed to explain the predictions of gnns, where graph counterfactual explanations (gce) have become a prevalent approach in recent years @xcite @xcite . specifically, gce aims to identify the minimum modification over a given graph, such that a trained gnn yields a desired prediction for the post-modified graph. here, the graph with the identified modification is called the counterfactual graph, or simply counterfactual for short. the identified modifications may involve adding or removing nodes and/or edges, as well as altering the node/edge attributes. for instance, given an undesired (non-aids drug) molecule as in fig. 1(a) , a gce method may generate modifications to produce a molecule as a "desired" graph (i.e., predicted as an aids-drug by the gnn model) as shown in fig. 1(b) . however, existing gce models have two significant limitations: (i) incomprehensible counterfactual optimization. most gce models are optimized to generate counterfactuals either through heuristic methods, such as random walk @xcite , or black-box deep learning methods @xcite @xcite @xcite , which fail to involve any human-interpretable knowledge in optimizing the counterfactuals. (ii) lack of domain knowledge. most current gce methods do not consider any domain-specific knowledge @xcite @xcite @xcite thus the generated counterfactuals may not be realistic in realworld contexts. continuing the example shown in fig. 1 , although the generated counterfactual fig. 1(b ) is classified as in the desired class, it is not chemically stable since it violates the valence bond theory @xcite . to handle the above limitations, large language models (llms) @xcite are ideal for addressing these limitations due to their ability to (i) generate comprehensible natural language texts, (ii) make the counterfactual optimization process human-interpretable, and (iii) leverage inherent domain knowledge from extensive pretraining to produce realistic counterfactuals. however, harnessing llms to improve counterfactual explanation generation faces challenges: (i) there exists a natural mismatch between texts (sequential data) and graph structures @xcite ; and (ii) llms may hallucinate, i.e., generate seemingly plausible while incorrect information @xcite . to handle these challenges, we propose a novel framework: llm-gce (large language models guided graph counterfactual explainer). specifically, to mitigate the first challenge, instead of directly generating the counterfactual graphs by llms, we utilize a counterfactual autoencoder (ca) to construct counterfactual graph structures based on the the text pairs (tps) and counterfactual text pairs (ctps) given by llms. to tackle the second challenge, the hallucination, we design a ctp dynamic feedback module enlightened by @xcite to update ctps iteratively based on previously generated counterfactuals. our main contributions are summarized as follows: (i) dataset construction. we collect llmgenerated text pairs over five molecule datasets. they not only support our empirical evaluations but also facilitate future studies for researchers in this field. (ii) algorithmic design. we propose a novel llm-gce framework that learns to generate graph counterfactual explanations under the guidance of an llm. llm-gce unlocks llm's strong reasoning ability in gce by addressing hallucinations and graph structure inference limitations. (iii) experimental evaluation. we conduct extensive experiments on multiple real-world datasets, validating the effectiveness of llm in generating more feasible counterfactuals while providing a compre-hensive optimization trajectory. ## preliminaries in this section, we introduce the problem settings for gce and the evaluation metrics used. we denote a molecule graph of m nodes (atoms) as @xmath0, a, e), where @xmath1, 1} m×m is the adjacency matrix, @xmath2, and @xmath3. note that the real-world molecules have 3-d structures; we leave the gce for 3-d molecule graphs for future work. furthermore, to determine if a generated counterfactual is classified as desired, we assume that there exists a ground-truth graph neural network (gt-gnn) represented as ϕ : @xmath4, which provides label predictions for graphs in the input graph domain d. this assumption is widely adopted in current literature @xcite . we define the problem of gce: definition 1. (graph counterfactual explanation). let ϕ : @xmath5, and let @xmath6, a, e) be an input graph with ϕ(g) = 0. the aim of graph counterfactual explanation is to find a model f : @xmath7. here, perturbations on the input graph g may include node/edge insertions and removals as well as changes to node/edge features. we refer to ĝ as the counterfactual of the original graph g. for an input graph dataset g sampled from the input graph domain d, we evaluate the performance of a gce model with the following metrics: we then define the validity metric of g as this molecule contains a cyclohexane ring, a dithiane ring, a ketone group, and a thiocarbonyl group, in which the ketone group may be the most influential for aids treatment. figure 2 : prompt-answer pair in text pairs generation. in alignment with this, we also provide the validity and proximity results with the feasibility check, where we only calculate the two metrics to the set of feasible counterfactuals ĝs, i.e., those counterfactuals that are chemically stable according to valence-bond theory @xcite . ## data construction there exist multiple datasets of molecules paired with text descriptions @xcite @xcite ; however, most of these datasets only have text pairs describing the graph labeling information. to support the evaluation of gce methods, a text pair should contain at least three aspects: graph structure information, graph label information, and the significant subgraphs that contribute the most to the graph's label. generating satisfying text pairs for gce is expensive since it requires the most advanced llms, such as gpt-4 and customized prompts, so we release five molecule datasets with our generated high-quality text pairs for the convenience of the community. ## conclusion in this work, we explore the ability of llms in guiding the gce for molecule properties prediction. specifically, we propose a novel model called llm-gce, comprised of a contrastive pre-training module, a counterfactual autoencoder, and a dynamic feedback module. extensive experiments validate the superior performance of llm-gce. firstly, the effectiveness of llm-gce relies heavily on the quality and relevance of the pretraining data used for the large language models. if the pretraining data is biased or lacks sufficient coverage of the target domain, it may lead to less accurate or relevant counterfactuals being generated. ensuring the llms are trained on high-quality, domainspecific data is crucial for optimal performance. additionally, the computational cost associated with using large language models can be a drawback. training and inference with llms are more time-consuming and resource-intensive than other graph counterfactual explanation methods. although we have shown that the time execution time is comparable for our methods with other baselines on the adopted datasets, llm-gce can be time consuming when dealing with extremely largescale graphs, such as some big proteins. furthermore, while the experiments demonstrate the effectiveness of llm-gce on several realworld datasets, the evaluation is still limited to specific domains, such as molecular property prediction. the generalizability of the proposed framework to other types of graphs and application areas remains to be investigated. further research is needed to assess the performance and adaptability of llm-gce across a wider range of graph structures and problem domains. lastly, the potential for hallucinations or inconsistencies in the generated counterfactuals remains a solid challenge. although the dynamic feedback module aims to mitigate this issue, there may still be cases where the llms produce counterfactuals that are not entirely faithful to the original graph or the desired properties.