GraphPINE: Graph importance propagation Neural Network for interpretable drug response prediction

Authors: Yoshitaka Inoue, Tianfan Fu, Augustin Luna
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: Graph Neural Networks, Information Propagation, Drug Response Prediction, Explainability

Abstract:
Explainability is necessary for tasks that require a clear reason for a given result such as finance or biomedical research. Recent explainability methodologies have focused on attention, gradient, and Shapley value methods. These do not handle data with strong associated prior knowledge and fail to constrain explainability results by relationships that may exist between predictive features.

We propose a GraphPINE, a novel graph neural network (GNN) architecture that leverages domain-specific prior knowledge for node importance score initialization. Use cases in biomedicine necessitate generating hypotheses related to specific nodes. Commonly, there is a manual post-prediction step examining literature (i.e., prior knowledge) to better understand features. While node importance can be obtained for gradient and attention-based methods after prediction, these node importances lack complementary prior knowledge; GraphPINE seeks to overcome this limitation. GraphPINE differs from other GNNs with gating methods that utilize an LSTM-like sequential format such that we introduce an importance propagation layer that unifies 1) updates for feature matrix and node importances, jointly and 2) uses GNN-based graph propagation of feature values.  This initialization and updating mechanism allows for more informed feature learning and improved graph representation.

We apply GraphPINE to cancer drug response prediction using pharmacogenomics data (i.e., both drug screening and gene data collected by several assays) for ~5K gene nodes included in a gene-gene input graph with drug-target interaction (DTI) knowledge graph as initial importance. The gene-gene graph and DTIs were taken from literature curated prior knowledge sources and weighted by the literature information. GraphPINE demonstrates competitive performance and achieves a PR-AUC of 0.894 and ROC-AUC of 0.796 across 952 drugs. To highlight the interpretability aspect of our work, we provide the ability to generate sub-graphs of node importances. While our use case is related to biology, our work is generally applicable to tasks where information is separately known about feature relationships. Code: https://anonymous.4open.science/r/GraphPINE-40DE

Decision: reject
Reviewer scores: 5, 6, 5

Reviews and discussion:
<thread>
creation date:
1729847693098

thread_type:
official_review

<thread_object>
creator: reviewer_rcqp

<content>
summary:
This work proposes GraphPINE to handle the problem of drug response prediction. GraphPINE uses a graph neural network incorporating biomedical prior knowledge from various resources. As for the model architecture, GraphPINE utilizes Graph Transformer and GAT to handle features, and designs Importance Propagation Layer to provide understandings for nodes and their relations. In experiments, GraphPINE outperforms other baseline methods and provide interpretable results for drug response prediction examples.

strengths:
1. This work utilizes different kinds of biomedical information to tackle drug response prediction, including gene-gene network, drug-target interaction and methylation. 
2. GraphPINE introduces Importance Propagation Layer, which is good at processing information from multi sources.

weaknesses:
1. This work lacks a problem formulation for the problem of drug response prediction, which makes the input and the output of the learning problem unclear. 
2. The model architecture design is generally lack of novelty. For the feature processing part, Graph Transformer and GAT are existing works widely used. While the Importance Propagation Layer is similar to LSTM/GRU gates. 
3. In the experiment part, all baselines are learning methods proposed several years ago and none of them is specially designed for drug response prediction. This makes the experimental comparison unreasonable, as the drug response prediction methods introduced in related work part are not compared.

questions:
1. What are the differences between drug response prediction and other drug-related prediction problems (e.g. drug-target interaction prediction and drug-drug interaction prediction)?
2. In section 2.2 “graph neural network in computational biology”, why molecular property prediction methods are not mentioned? GNN is widely used for molecular property prediction problems. 
3. Why the title of section 3.3.2 is “GraphPINE model”? It seems that the title and contents do not match. 
4. In Figure 2, 3, 4, why there are color bars on the right? There are only two types of colors for nodes in the figure.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730569302520

thread_type:
official_review

<thread_object>
creator: reviewer_1hic

<content>
summary:
The authors present GraphPINE, an interpretable GNN for drug response prediction. This methodology is able to deliver accurate results in terms of drug response prediction and, at the same time, provide interpretable outcomes in terms of important genes in the input graph. The authors properly described their methodology and compared their results with other methods. Finally, they show how it is possible to obtain and visualize interpretations for the predictions obtained.

strengths:
The main strengths of the paper are the following:

1) The work is original and relevant since it touches on an important aspect of GNNs (interpretability). Instead of relying on external explainers (which can deliver biased results), the authors propose a way to render GNN interpretable by using importance score propagation.

2) The domain of application is of extreme importance and relevance since it could help facilitate the detection of drug response/resistance, speeding up drug development and clinical trial phases.

3) The interpretability results are in line with the knowledge present in the literature.

4) Overall, the work is well-presented and also the Appendix provides useful information.

5) The authors provided an anonymous repository for reproducibility

weaknesses:
The main weaknesses of the paper are the following:

1) It is not clear to me how the initial importance scores are assigned. The authors say these scores are obtained using the weights of the edges of a knowledge graph. How is this done? My doubt is that if we start from consolidated importance scores, the result's final propagated importance will present a bias since it will be strongly dependent on the initial weights.

2) It is not clear to me if the authors compared their results against GAT, GT, and GINE. They present three GraphPINE versions based on those architectures, but a direct comparison with them is not provided. I am puzzled since, in the Appendix, they describe the hyperparameter tuning for those models, but no result from them is present in Table 1.

3) The improvement brought by GraphPINE in terms of evaluation metrics is marginal with respect to other methods. In particular, it would be interesting to see how GAT, GT, and GINE perform when used as standalone techniques.

questions:
My questions are related to the weak points I described. 

1) Can the authors better describe how the initial importance scores are obtained? This should be carefully described in the main paper and not in the Appendix.

2) If one used the initial importance scores to build a ranking of important nodes, would the outcome be different? If yes, then the methodology is effective; if no, probably the resutls are strongly affected by the initial importance scores.

3) Can the authors show how GAT, GT, and GINE perform when used as standalone strategies?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730728429158

thread_type:
official_review

<thread_object>
creator: reviewer_kmoq

<content>
summary:
The manuscript introduces GraphPINE, a Graph Neural Network (GNN) architecture aimed at enhancing interpretable drug response prediction. By leveraging prior biological knowledge through a knowledge graph with weighted edges, GraphPINE generates initial importance scores for nodes. The core innovation of the work is the Importance Propagation (IP) layer, which facilitates the propagation of node importance throughout the GNN, thereby promoting biological interpretability.

strengths:
1. The manuscript addresses an important challenge in drug response prediction, highlighting the need for interpretable models in biomedical applications.

2. The integration of a knowledge graph to inform initial importance scores is a compelling approach that could enhance the interpretability of GNNs in drug response contexts.

weaknesses:
1. The methodology lacks novelty, as graph convolutional networks (GCN) and importance gating have been previously employed in similar contexts. The claim of novelty in this paper is undermined by the existence of other interpretable GNN-based methods for drug response prediction.

2. The ablation study is missing, and important baseline models are not included in the comparisons. The results show only marginal improvements over baseline models. For instance, ROC-AUC results show GraphPINE at 0.7955 compared to LightGBM at 0.7901, and PR-AUC results indicate 0.8939 vs 0.8917. These limited improvements weaken the overall impact of the claims.

3. The authors should provide comprehensive comparative analyses with and without drug-target interaction (DTI) information to demonstrate the importance of DTI. If predictive performance is comparable to models that exclude DTI, this raises questions about the effective utilization of the interaction information.

questions:
see above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732682364855

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
We thank all reviewers for their review and constructive feedback. In this general response, we summarize our main updates and the additional experiments we performed. In the updated paper we highlight major changes in blue.

## Updates to our main results

We have made important updates to our main results that are detailed in reviewer responses. 
1. Added ablation study and drug response prediction baseline methods.
2. Clarify the novelty of GNN with gating methods comparison with previous methods.
3. Evaluate the importance propagation between the initial and propagated score. 

#### Minor updates
- We explain more the use case being examined (i.e., the interplay between drug response and drug target interaction prediction)
- We add more related works
- We move details about generating our DTI dataset from appendix to main text
- We edit section titles for clarity
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper addresses drug response prediction using a novel GNN-based model, GraphPINE. Reviewers raised concerns about the novelty of the approach, as the methods employed (e.g., GCN, importance gating, and GAT) are well-established. The lack of an ablation study, insufficient baseline comparisons, and marginal improvements over existing models were also highlighted. The authors have clarified the novelty of their work, emphasizing the unique integration of domain-specific prior knowledge and importance propagation. They also addressed the absence of molecular property prediction methods and expanded experimental comparisons. Despite these clarifications, the overall contributions lack substantial innovation and the experimental validation is not convincing enough to justify publication. The paper’s incremental advances and limited empirical support led to the decision for rejection.