MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
Authors: yeyunchen
Jiangming Shi
Venue: ICML 2025 poster (2025)
Keywords: graph prompt
molecule property prediction
Primary area: applications->chemistry_physics_and_earth_sciences

Abstract: The progress in materials science and drug discovery is impeded by the availability of labeled data and the high costs of manual annotation, driving the need for efficient strategies to capture molecular representations and enable accurate predictions. Pretrained Graph Neural Networks have shown promise in capturing universal molecular representations, but adapting them to task-specific applications remains challenging. In this paper, we propose Multilevel Informed Prompt-Tuning (MIPT), a novel framework for effectively tailoring pretrained models to molecule-related tasks. MIPT utilizes a lightweight, multi-level prompt learning module to capture node-level and graph-level task-specific knowledge, ensuring adaptable and efficient tuning. Additionally, a noise penalty mechanism is introduced to address mismatches between pretrained representations and downstream tasks, reducing irrelevant or noisy information. Experimental results show that MIPT surpasses all baselines, aligning graph space and task space while achieving significant improvements in molecule-related tasks, demonstrating its scalability and versatility for molecular tasks.

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
Yeyun Chen 2 Jiangming Shi 1.2
Abstract
driven interest in strategies that efficiently capture molecu- lar properties and enable accurate predictions with minimal supervision. Pretrained Graph Neural Networks (GNNs), designed to learn universal molecular representations, offer a promising solution. Yet, adapting these representations to task-specific applications remains a key challenge, necessi- tating efficient and flexible tuning methods. The pretrain- and-finetune paradigm (Hu et al., 2019) has been proposed to address this challenge, but the gap between pre-training objectives and downstream tasks often hinders effective knowledge transfer, leading to suboptimal performance.
The progress in materials science and drug discov- cry is impeded by the availability of labeled data and the high costs of manual annotation, driving the need for efficient strategies to capture molec- ular representations and enable accurate predic- tions. Pretrained Graph Neural Networks have shown promise in capturing universal molecular representations, but adapting them to task-specific applications remains challenging. In this paper, we propose Multilevel Informed Prompt-Tuning (MIPT), a novel framework for effectively tailor- ing pretrained models to molecule-related tasks. MIPT utilizes a lightweight, multi-level prompt learning module to capture node-level and graph- level task-specific knowledge, ensuring adapt- able and efficient tuning. Additionally, a noise penalty mechanism is introduced to address mis- matches between pretrained representations and downstream tasks, reducing irrelevant or noisy in- formation. Experimental results show that MIPT surpasses baselines, aligning graph space and task space while achieving significant improve- ments in molecule-related tasks, demonstrating its scalability and versatility for molecular tasks.
Prompt-based learning (Sun et al., 2023b; et al., 2024) has recently emerged as a powerful method for bridging this semantic gap. By aligning downstream tasks with pre-trained feature spaces, prompts facilitate knowledge transfer, improving accuracy and efficiency in applications such as molecular property prediction. Graph prompts, in particular, have shown efficacy in narrowing this gap. Some approaches use manually customized prompts (Wang et al., 2024), incorporating task-specific substructures or functional groups, while others explore universal prompts, such GPF (Fang et al., 2024), which are applicable across all nodes, enhancing task adaptability and gener- alization while reducing computational costs. Additionally, in-context learning methods like PRODIGY (Huang et al., 2024) leverage label nodes to construct prompt graphs, trans- forming downstream tasks into link prediction problems and enhancing contextual learning. These approaches col- lectively provide more flexible and efficient solutions for downstream applications.
1. Introduction
Machine learning (ML) has demonstrated significant po- tential to revolutionize the discovery and development of new materials (Chibani & Coudert, 2020), drugs (Shen & Nicolaou, 2019), and chemical processes (Taskinen & Yliru- usi, 2003). However, the rapid growth in these fields is hindered by the scarcity of labeled molecular data and the high cost of manual annotation. These challenges have
Despite these advances, significant challenges remain in applying prompts to pre-trained GNNs. Many existing graph prompts are unable to optimize for specific tasks and exhibit limited scalability. While manually designed prompts (Wang al., 2024) offer task-specific flexibility, their reliance on predefined templates limits adaptability, Similarly, universal prompts, such GPF-plus (Fang et al., 2024), struggle to capture the diversity of all nodes, and as- signing distinct prompts to each node incurs high computa- tional costs (Sun et al.). Some studies address these issues by introducing label nodes construct graph prompt (Huang et al., 2024), but this approach can disrupt node depen-
'Institute of Artificial Intelligence, Xiamen University, Xi- amen Fujian, China "Shanghai Innovation Institute, Shang- hai, China. Correspondence to: Jiangming Shi <jiang- ming shi@ wot@@uuuoooocom...
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
dencies and alter graph topology, potentially reducing pre- diction accuracy. Moreover, the large number of tunable parameters in pre-trained GNNs, combined with the limited availability of labeled molecular data, makes downstream fine-tuning less effective. These limitations highlight the need for adaptable, scalable, and efficient prompt designs to fully leverage the potential of pre-trained GNNs.
We introduced novel framework called Multilevel Informed Prompt Tuning (MIPT), which leverages lightweight multi-level prompt learning module to effec- tively capture task-specific knowledge at both the node and graph levels, enabling efficient adaptation of pre- trained GNNs to molecular tasks.
We incorporates a noise penalty mechanism to address mismatches between pretrained representations and down- stream tasks, effectively reducing irrelevant or noisy in- formation and enhancing task performance.
On the other hand, while prompts are effective in enhanc- ing generalization, they may inadvertently introduce task- irrelevant noise, which can hinder the model's ability to prioritize critical features, ultimately leading to suboptimal performance in specific tasks. For instance, in graph classi- fication, prompts may overemphasize edge or node details while neglecting the overall graph structure. Similarly, pre- trained GNNs, despite capturing broad feature distributions, may fail to align with task-specific requirements, such mismatches between node-level pre-trained features and subgraph-level features needed downstream, causing overfit- ting and reduced accuracy. Additionally, downstream tasks often exhibit biases, prioritizing high-probabilitg features while ignoring low-probability tail samples. Effectively bridging the gap between generalized pre-trained knowl- edge and domain-specific needs presents critical challenge, necessitating highly adaptive approaches to mitigate noise, address feature mismatches, and align with task-specific biases for enhanced performance.
Experimental results demonstrate that MIPT surpasses baseline model, excelling in aligning graph and task space modalities, while showcasing scalability and versatility across a wide range of molecule-related tasks.
2. Related Works
2.1. Pre-trained GNNs
Pretraining and fine-tuning are widely used to transfer knowledge from related tasks and enhance model gener- alization. Pretraining trains on large-scale data via self- supervised or supervised tasks, followed by fine-tuning on smaller labeled datasets. In molecular GNNs, various self-supervised tasks capture chemical rules and patterns at node, subgraph, and graph levels (Xia et al., 2022b). However, pretrained GNNs face challenges in extracting task-relevant knowledge and often suffer from overfitting during fine-tuning (Xia et al., 2022a). Unlike NLP and CV, pretrained GNNs do not consistently improve downstream performance (Sun et al., 2022b), partly due to limited re- search on selecting effective self-supervised tasks. Most studies adopt only a few tasks for pretraining, resulting in GNN models excelling in specific downstream tasks but lacking consistency (Sun et al., 2022b). Moreover, recent findings suggest self-supervised pretraining sometimes fails to outperform non-pretrained methods (Sun et al., 2022b), Overall, pretrained GNNs offer limited advantages over non non-pretrained models. Recent approaches have attempted to improve fine-tuning using regularization (Xuhong et al., 2018) or update constraints (Houlsby et al., 2019; Xia et al., 2022a; Zhang et al., 2022).
Based on the above observations, this paper aims to address the following challenges: (i) While prompts are designed to bridge the gap between pre-trainnn pre-training and downstream tasks, universal prompts frequently exhibit limited intuitive interpretability and poor scalability. Consequently, effec- tive prompt design necessitates integrating graph-specific information, capturing their adaptability to graph structures, and ensuring efficient scalability. (ii) Although prompts are intended to guide models towards task adaptation, their ap- plication may inadvertently introduce task-irrelevant noise due to suboptimal feature modeling, thereby compromising performance. Therefore, it is crucial to effectively extract salient features, mitigate noise interference, and enhance the model's robustness and adaptability to downstream tasks.
In this paper, we propose a novel framework, Multilevel Informed Prompt Tuning (MIPT), to enhance pretrained GNNs for molecule-related tasks. The framework employs a lightweight multi- prompt learning network to cap- ture task-specific knowledge at both node and graph levels, enabling effective adaptation to diverse scenarios. Further- more, a noise penalty mechanism is introduced to align distributions and facilitate knowledge transfer across graph spaces. Experiments on public datasets demonstrate that MIPT achieves superior performance in molecular property prediction while requiring fewer trainable parameters.
2.2. Graph Prompt Learning
Despite the advances in pre-trained models, bridging this gap between pre-trained and fine-tuned models has become critical focus in recent research. GPPT (Sun et al., 2022a) introduced learnable prompt mechanism for graphs, specif- ically targeting node classification tasks. While effective in this specific domain, its design lacks generalizability and cannot adapt to other downstream tasks. Building upon this, Graph Prompt (Liu et al., 2023) proposed a novel task-specific learnable prompt, which guides the ReadOut
Our contributions can be summarized as below:
2

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
operation of each downstream task using appropriate aggregation scheme. However, this approach faces chal- lenges in multitasking scenarios, as differences across tasks make it difficult to identify universal prompt pattern, ul- timately limiting its effectiveness. All in One (Sun et al., 2023a) addressed the issue of multitasking by designing a unified training framework for joint multi-task prompts. While this approach leverages meta-learning to accommo- date large variety of tasks, it is computationally expensive and exhibits limited generalization capabilities across di- verse tasks. GPF and its extended version, GPF-plus (Fang et al., 2024), introduced learnable prompts at the node level, aiming for versatility across different downstream tasks. This approach effectively balances adaptability and compu- tational efficiency, presenting a more practical solution for a broad range of task requirements.
nents: a GNN encoder f: 9 - IR, which generates molec- ular representations, and a projection layer g: IR - y. In this context, the projection layer serves downstream classifier, predicting labels from the generated representa- tions. Specifically, the encoder / operates in two stages. The first stage utilizes a GNN to learn node-level representations he formulated as follows:
h -0(*) (n.. (n. (hhht"",. (0 (fn!-") : w(e))) (2)
where (k) represents the node embedding of node at the k-th layer, ((v) is the set of neighbors of node v, ()() is the aggregation function at the k-th iteration, and (0()) is the combination function at the k-tth iteration. The node embeddings are initialized as hū X.., where X,, denotes the input features of node v.
To ensure effective feature updates, the feature transforma- tion at layer k can be expressed as:
3. Preliminaries
3.1. Problem Formulation
(1+c h (1-+))
h MLP(K)
Σ
h(t
Notations. Let G = (V,8) E G denotes a graph, where V ==( Un) represents the set of nodes, and & S V x V represents the set of edges. G is associated with a node fea- ture matrix X NxF where F is the feature dimension, and an adjacency matrix A R N where Aij 1 if and only if (us, 11)) E E and Aij 0 otherwise.
MENTO
(3)
where (a) is a learnable parameter, MLP(() represents multi-layer perceptron applied at the k-th layer, and M[c] includes the neighbors of node v along with the node itself.
Secondly, at the final layer K, the GNN aggregates the learned node embeddings using a READOUT function:
Problem Definition. Given pre-trained GNN f. a learn- able projection layer g. and a downstream task dataset D = {(G1, y1), (92, v2)..... (an..mm).... our objective is to fine-tune the parameters of the prompt Py and the pro- jection layer g to effectively bridge the gap between the trained GNN and the downstream task. Specifically, the aim is to maximize the likelihood of accurately predict- ing the ground truth labels y associated with the downstream task, formulated as follows:
h (EADOUT(( : e VJ)
(4)
where READOUT can be implemented as sum, mean, or maximum over all node embeddings in the graph.
4. Method
This section provides thorough explanation of the pro- posed MIPT framework, detailing its technical design and the motivations behind its development. MIPT is specifi- cally designed to address two primary challenges observed in current prompt-tuning approaches: (i) Universal prompts often lack the necessary flexibility and interpretability to adapt to diverse task-specific requirements, and (ii) prompts can inadvertently introduce task-irrelevant noise, which can significantly degrade overall model performance.
max P Po Ly (
pg.
3.2. Molecule Representation Learning.
We adopt the Graph Isomorphism Network (GIN) (Xu et al., 2018) as the backbone for our models, leveraging its strong expressive power and effectiveness in graph-related tasks. In fact, our framework is model-agnostic, enabling seamless integration with various existing molecular pre- diction model as downstream models. The pre-training process designed learn general-pe general-purpose -purpse graph repre- sentations by optimizing auxiliary objectives large-scale, unlabeled graph data. This process primarily follows two paradigms: self- supervuee learning and contrastive learn- ing (Zhou et al., 2024).
To tackle these issues, we present a comprehensive overview of the framework's 's core components: the multi-level graph- informed prompts and denoising prompt mechanisms. The overall architecture is illustrated in Figure 1.
4.1. Parameter-efficient Tuning for Node Encoder
Efficient Tuning with LoRA Update. To address the gap between pre-training objectives and downstream task re- quirements, we propose an efficient fine-tuning approach
Consider a molecule graph G = (V,E). Typically, the pre- dictor p can be expressed as f o g comprising two compo-
3

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
Node Embedding
Graph Embedding
Frozen
Pre-trained GNN
Readout
max (r)
Tuned
7A
Prompt
LNPP
LoRA-Tuning
I.cont
Ground Truah Label
Low-rank Adapter
dlean
Mask
Lets
Modeling
LoRA-Tuning for Node Encoder
Graph Informed Prompt
Noise Penalty Mechanism
Mosked Node
Low-ronk
Graph
Node space
Embedding
Adopter
Prompt
Confidence
Graph Encoder
Boadout
Mack
-Poofing
Node Fmbraiing
Graph Embrddinng
Graph space
Graph Embedding
Initial Graph Embedddings
Figure 1. Comprehensive overview of the MIPT framework. Our proposed framework synergistically combines efficient node fine-tuning and a unified distribution denoising mechanism, thereby providing an efficient robust solution for local-global prompt-tuning across varied scenarios.
that integrates a low-rank adaptation matrix for learning task-specific embeddings. This method prioritizes features pertinent to downstream tasks while preserving the expres- siveness of the pre-trained model. For the frozen pre-trained GNN weight matrix Wo B4xk the parameter update A W is represented using a low-rank decomposition. The feature update process, as described in Eq. 5, is reformulated as:
4.2. Multilevel Graph-Informed Prompt
For downstream prediction tasks, obtaining task-specific graph representations hg is essential, as graph-level em- beddings directly influence model performance. However, existing pooling operations fail to satisfy the expressiveness conditions (Bianchi & Lachi, 2023). To address this, pro- pose Multilevel Graph- Infoph-nnformed Prompt (MGIP), novel method that extracts features from node-level embeddings and constructs low-rank graph prompt after aggregation. By encoding domain-specift information directly into hy, our approach enhances the adaptability and efficiency of graph representations.
4 (W + AW(R) Σ (((+c(t) 1(6-1)++
(1++())))
Σ h (--1)
here, Wo denotes the weight matrix of the pre-trained MI.P layer, and AW represents the low-rank adaptation. The parameter AW is further expressed as:
Random Node Masking. Let h; E R be the node embeddings with LoRA tuning initialized using Kaiming initialization (He et al., 2016). Subsequently, we employ a random masking method to process hü, aiming to enhance the model's robustness and its ability to identify critical subgraphs. This method involves three main steps: segment- ing, filling, and random masking, ensuring both structural and representational variability initialize binary matrix M E {0, 1 1 where b is the batch size and Smax is the maximum number of nodes in the batch. To ensure all samples in the batch share the same input dimensions, we apply zero sero-padding operation to the node representations of each sample. The padding defined as:
(W(K) ((A))((0
where A R and E Rd are low-rank matrices with r d. The low-rank structure of AW ensures that the additional parameter number is minimal, thus enhancing computational efficiency. These trainable low-rank matrices are injected into the node MLP layer to approximate weight updates effectively.
effectively captured. We
For a pretrained GNN model f. with pa- rameters W, node features X, there satisfies that incor- porating LoRA parameters into the linear transformation during node updates does not alter the underlying graph structure.
Proposition 4.1.
M = {
if < 8;
otherwise
detailed proof for proposition 4.1 refers to Appendix A.
where 8; represents the size of the i-th graph. A random

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
masking matrix R. E {0, 1 jox8mm is generated, with each element sampled from Bernoulli distribution: R Bernoulli(1-P) (1-p), p). where is the masking probability (e.g.. p = =0.2). The final mask matrix is updated as: M t M @ R, where @ denotes element-wiss multiplication. The updated mask is applied to the node v to produce the masked embed-
final output:
ύ= g=g(h))
(12)
where g is the prediction function. This modification enables downstream models to efficiently leverage the prompts Pa for adaptation to new data and tasks.
dings hyz
4.3. Optimization
ho: if M =1
R1 -1000
In the prompt-tuning phase, we specifically update the pa- rameters of the prompt and downstream by performing limited training on an unseen dataset. This approach enables downstream models to efficiently adapt to new data.
[0, otherwise
where j in M., is the i-th graph of the j-th node. By com- bining deterministic segmentation with random masking, this approach balances structural consistency with variabil- ity, encouraging the model to identify key subgraphs and improving generalization. This process ensures the graph representations retain critical information while being robust to structural noise.
Training Procedures. To train the model, a task-specific projection head is used.
L. = log P (v' g (ha))
(13)
For graph classification, the loss is typically the binary cross- entropy loss:
Global-Local Information Maximization. To obtain the global representation of the entire graph, we utilize a readout function P M, E R/Vxdd + R/g E Rd (i.e., h2 = T(K,). hg = )),,)), which aggregates the local features into global context. To improve discriminative power, con- trastive learning (CL) is introduced to maximize mutual information between original and masked features. The contrastive learning loss function is formulated as:
N (ys log(ús) + (1 - yz) log(1 - (:))
Las((9)) = Σ
(14)
where N is the number of training samples, y/ is the true label, and it is the predicted label of i-th sample.
Noise Penalty Mechanism. Building on Arpit et al. (2017), it has been demonstrated that deep neural networks (DNNs) initially learn simpler and more generalizable patterns dur- ing the early stages of training, before gradually overfitting to noisy or degenerate samples. Inspired by this observation, model the confidence distribution of samples based on their classification loss (Shi et al., 2024). Specifically, we employ a two-component Gaussian Mixture Model (GMM) to represent the loss distribution derived from Eq. (13):
Leant
exp (g (bg. ))
log exp (0 (hg hn.)) + E)= exp (s (h2 has) (9)
(()) = ( T TT) (hg.))/T
(10)
where 7 the temperature parameter. CL enhances the cor- relation between local node representations and global graph representations, improving the multi-level expressiveness of graph embeddings.
T
Graph Informed Prompt Generation. Then, we propose the initialization of graph prompt is guided by graph masked feature. To capture domain-specific graph features, we in- troduce a lightweight prompt function, denoted as (b-), to provide implicit prompts Pg- Formally, this prompt function, is defined as follows:
p(Cots 10)
Σ t=1
Ty T |t)
(15)
where hgg is pretrained molecular features, g(-) is down- stream task with parameters 0, and n. and o( 1) repre- sent the mixture coefficient and the probability density of the 1-th Gaussian component, respectively.
(hkg) = WB K2)
(11)
The GMM, (C ( - p. is utilized to model the pos- terior probability, denoted as w; S w; S 1), which is computed to quantify the confidence of the -th sample:
The trainable parameters of this prompt function are de- noted as WA E Rd' and WB E IR" where r < d' The domain-specifhe prompt is combined with the graph features from pretrained encoder, creating rich information that enables the effective identification of context-specific changes in critical structures. This, in turn, facilitates pre- cise modeling of structural correlations.
w, p (t Cats)
(16)
where k is the Gaussian component with the smaller mean, and p (t C.ss) represents the posterior probability of C. under the t-th component. The confidence d; is then up- dated as follows:
Task-specific Projection Head. After that, the refined graph representation, hg = CONCAT(h,,P)) Py), is subse- quently passed through a projection head to produce the
1, if >
W
(17)
0, otherwise
5

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
Here, j is the average of wi, i.e. mean( With the confi- dence i effectively capturing high-relevant representation, we introduce the Noise Prompts Penalty (NPP) loss, LNPP, to suppress the influence of redundancy noise features. The loss is defined as:
RQ2: Does MIPT effective in generalizing pre-trained
models to new molecular property prediction datasets and tasks?
What impact do the key components of MIPT have on the performance of downstream models?
RQ3:
CNPP = - N Σ w; log P (g'1g (ha))
(18)
RQ4: How do hyperparameters influence the perfor- mance of MIPT on molecular property prediction?
This mechanism leverages & to penalize noisy features, ensuring that the model focuses on clean and reliable rep- resentations, thereby improving its robustness and overall performance.
5.1. Experiment Setup
Datasets, We employ eight common datasets from Molecu- leNet (Wu al., 2018) as our benchmark datasets: BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV and BACE. Random splits and scaffold splits for these datasets adopted. More dataset and training details are available in the Appendix C.I.
Final Objectives. The overall training objective for the downstream task is illustrated as follows:
Liotal = LNPP + Loont
(19)
The model parameters 0 are updated by minimizing Liotal using gradient- based optimization.
Pretraining strategies. We take into account four pre- trained models for comparison: Deep Graph Infomax (Info- max), Attribute Masking (AttrMasking), Context Prediction (ContextPred), and Edge Prediction (EdgePred).
Compute the gradient of Liotat with respect to the model parameters:
Baselines. We consider three tuning strategies for compari- son is used to evaluate: FT 2024), GPF-plus (Fang et al., 2024). In the case of freezing the pre-trained model, our prompt method aims to learn the input graph and reconstruct the downstream task to fit the pre-training strategy.
VoCrotal = VoLNPP + VoCcot
(20)
GPF (Fang et al.,
8- V frotal,
(21)
where is the learning rate. The parameters of GMM
are updated in alternating steps using the Expectation- Maximization (EM) algorithm, with the pseudocode pro- vided in the Appendix 1.
A variety of baselines are included for in-depth compar- ison. Supervised GNN methods include D-MPNN (Yang et al.. 2019), MGCN (Lu et al., 2019), and Atten- tiveFP (Xiong et al., 2019). Meanwhile, pretraining approaches consist N-gram (Liu et al., 2019), Pre- trainGNN (Hu et al., 2019), GROVER (Rong et al., 2020), 3D-Infomax (Stärk et al., 2022), GraphMVP (Liu al., 2021), MoICLR (Wang al., 2022), Uni-Mol (Zhou et al., 2023), and InstructMol (Wu et al.).
The training process alternates between updating the model parameters 0 and the GMM parameters until convergence. By progressively refining both components, the model learns to focus on reliable samples, while reducing the impact of noisy prompts, ultimately improving the robustness and accuracy of predictions.
Proposition 4.2. (Robustness of NPP) Given a classifica- rion task where prompts Py are used to adapt a pretrained model f(-), the incorporation of the NPP loss Expp im- proves robustness by selectively reducing the influence of noisy prompt.
Metric. ROC-AUC scores are evaluated on the test set. We report the mean and standard deviation of the results from three random seeds.
A detailed proof for proposition 4.2 is given in Appendix B.I.
5.2. Overall Performance (RQ1)
We evaluated the performance of downstream tasks across various pre-training and tuning strategies, with results sum- marized Table 1. Our method consistently outperforms all baselines, demonstrating its effectiveness and robustness. It achieves significant improvements over fine-tuning strate- gies across all datasets. While universal learnable prompts in baseline approaches yield limited gains and lack con- sistency, our method achieves the highest ROC-AUC (%) scores on most datasets, as highlighted in bold.
5. Experiments
In this section, present extensive set of experiments to evaluate the performance of our model. The analysis is structured around addressing the following key research questions:
RQ1: How does the efficiency of MIPT compare to that of prompt-tuning and end-to-end training approaches?
For example, on the BBBP dataset, our method achieves
6

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
Table 1. GGaaph classification (ROC-AUC, scores %, higher is better 1) results across 8 datasets. The best results are highlighted in boldface, while the second- best results marked with underline.
Datasets
BBBP
Tox211
ToxCast
SIDER1 1427
ClinToxt 1477
MUVI 41127
HIVI 93087
BACE!
Pretrained strategy
# Molecules
2039
7831
8576
1513
Tasks
1
12
617
27
17
+ FT GPF GPF-plus
67.55(2.06)
78.57(0.51)
65.16(0.53)
70.06(1.45) 63.34(0.45)
81.42(2.65)
81.32(1.255 (7.71(0.45)
66.83 (0.86)
79.09 (0.25)
66.10 (0.53)
66.17 (0.81)
73.56 (3.94)
80.43 (0.53)
76.49 (0.18)
83.60 (1.00)
Infomax
67.17 (0.36)
79.13(0.70)
66.35(0.37)
65.62 (0.74)
75.12(2.45)
81.33 (1.52)
83.67(1.08) 77.73(1.14)
+ Ours
69.24(0.84)
80.18(0.25)
67.10 (0.38)
66.40(0.36)
80.02 (1.3)
78.77(0.60) 82.84(0.58)
(3.55
+FT
66.33 (0.55)
78.28 (0.05)
65.34 (0.42)
66.77 (1.02)
74.46 (2.82)
81.78 (1.95)
77.90 (0.18)
80.94 (1.17)
+GPF
68.09 (0.38)
79.04 (0.90)
66.32 (0.42)
69.13 (1.16)
75.06 (1.02)
82.17 (0.65)
78. 86 (1.42)
84.33 (0.54)
AurMasking
GPP-pluus
67.71 (0.64)
87 (0.31)
66.58 (0.13)
68.65 (0.72)
76.17 (2.98)
81.12 (1.32)
78.13 (1.12)
85.76 (0.36)
+ Ours
69.56 (0.82)
80.77 (0.43)
67.91 (0.38)
69.15(0.40)
85.18 (0.12)
85.01(0.61)
79.48(0.64)
83,92 (1.46)
+FT
69.65 (0.87)
78.29 (0.44)
66.39 (0.57)
64.45 (0.60)
73.71 (1.57)
82.36 (1.22)
79.20 (0.51)
84.66 (0.84)
+GPF
68.48 (0.88)
79.99 (0.24)
67.92 (0.35)
66.18 (0.53)
74.51 (2.72)
84.34 (0.25)
78.62 (1.46)
85.32 (0.41)
ContextPred
+GPF-plus
69.15 (0.82)
80.05 (0.46)
67.58 (0.54)
66.94 (0.95)
75.25 (1.88)
84.48 (0.78)
78.40 (0.16)
85.81 (0.43)
+ Ours
73.62 (0.11)
80.60 (0.27)
68.81 (0.6)
68.52 (0.3)
81.21 (0.86)
84,96 (1.2)
81.76 (0.10)
86.57 (0.49)
+FT
66,56 (3.56)
78.67 (0.35)
66.29 (0.45)
64.35 (0.78)
69.07 (4,61)
79.67 (1.70)
77.44 (0.58)
80.90 (0.92)
GPF +GPF-plus
69.57 (0.21)
79.74 (0.03)
65.65 (0.30)
67.20 (0.99)
69.49(5.17)
82.86 (0.23)
77.00 (1.08)
81.57 (1.08)
EdgePred
69.06(0.68)
80.04 (0.06)
65.94 (0.31)
67.51(0.59)
68.80 (2.58)
83.13(0.42)
77.00 (0.82)
81.75(2.09)
+ Ours
72.11(0.50)
81.26(0.80)
67.05(0.60)
67.25(0.30)
77.68(1.10)
86.30(1.20)
(9.38(0.01)
(1.79(0.11)
an ROC-AUC of 73.62 under the ContextPred pretraining strategy, reflecting an 8.98% improvement over FineTTune. Similarly, on Clin it attains 85.18 under AutrMasking, surpassing GPF-plus by 11.83%. On MUV, despite FT's strong baseline performance, our method reaches 86.30, yielding 6.00% improvement. These results highlight its superior performance and generalizability, consistently sur- passing alternative approaches across diverse datasets and pre-training strategies. Its strong performance on complex or low-quality datasets further underscores its robustness and adaptability in graph-related tasks.
the test dataset compared to other models. This suggests that the model is more sensitive to the data during training, allowing it to flexibly adjust and capture complex patterns in the dataset. The increased fluctuations indicate more exploratory optimization process, which facilitates finding better global solution. Additionally, this behavior demon- strates strong adaptability and robustness to complex data distributions and noise. More training curves are provided in Appendix D.
Ablation Study (RQ3). To assess the effect of each compo- nent in our method, we conduct an ablation study evaluating LoRA, MGIP, and the loss functions (LNPp and Leont). The results in Table 3 highlight the significance of MGIP over LoRA when used independently, achieving an average im- provement 6.9% on SIDER and 1.6% on BACE, demon- strating its superior ability to capture task task-specific features. Integrating Leomt with LoRA further enhances performance, yielding a 6.5% gain on SIDER, indicating that contrastive learning effectively aligns global and local representations. Similarly, incorporating CNPP into MGIP boosts ClinTox performance by 1.2% underscoring the importance of node- to-graph alignment in refining global representations.
We also compared our model with other SOTA models for molecular property prediction. As shown in Table 2, our method outperforms SOTA models in 5 out of 8 tasks while using fewer parameters, demonstrating its efficiency and effectiveness.
Further analysis reveals that our approach offers distinct ad- vantages over prompt-tuning models in molecular property prediction. Specifically, the MIPT framework effectively captures multi-level structural features in molecular data, Its novel prompt-leaming paradigm facilitates knowledge trans- fer from pre-training to downstream tasks, mitigating distri- butional and semantic discrepancies between pre-training and target scenarios.
When combining all components. the complete model achieves the best results across all datasets, with improve- ments of 2.9% on SIDER, 1.3% on ClinTox,, and 0.9% on BACE compared to the best individual configurations.
5.3. Additional Experiments
These results confirm the complementary nature of the com- ponents and the effectiveness of the proposed approach in addressing multi-scale representation challenges in graph learning. Additional ablation results on various datasets are presented in Appendix D Table 6.
Transfer Performance Analysis (RQ2). We analyzed the training process of the molecular dataset and the GNN model using different tuning methods. Figure 2 (a) shows the testing curves during the tuning phase. From the curve, it is evident that our method exhibits greater fluctuations on
7

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
Table Performance comparison with SOTA models on molecular property prediction datasets(ROC-AUCC %, higher better f)
Pretraining
Datasets
BBBP
BACE! 1513
ClinToxt
Tox211
ToxCast
SIDER
MUV1 93087
HIV1 41127
Molecules
2039
1477 2
7831
8576
1427
# Tasks
1
12
617
27
17
D-MPNN
71.0 (0.3)
80.9 (0.6)
90.6 (0.6)
75.9 (0.7)
65.5 (0.3)
57.0 (0.7)
77.1 (0.5) 75.7 (1.4)
78.6 (1.4)
Attentive FP
(1.8)
(0.02)
84.7 (0.3)
76.1 (0.5)
63.7 (0.2)
60.6 (3.2)
MGCN
(0.5
(0.8)
(0.4)
(4.1
58.7
N-GramRF
(0.6)
(7.5 (1.5
(0.3)
66.8
77.2(0.))
76.9(0.7)
N-Gram XCB
(0.8
(1.1
(2.7)
75.8 (0.9)
65.5 (0.7)
78.7(0.4)
74.8(0.2)
PretrainGINN
(1.3
(0.7)
(2.6(1.5)
(8.6
65.7 (0.6)
62.7 (0.8)
79.9(0.7)
81.3(21)
GROVER
(0.1)
(2.6
83.2
(0.1)
(0.4)
64.8 (0.6)
62.5(0.9)
67.3(18)
GROVER lorge
(0.1)
(1.4
(3.7
(3.1)
65.3 (0.5)
(0.4
68.2(1.1)
67.3(1.8)
3D-Infomax
(1.1
(1.4
(3.2)
74.5
(1.0)
(3.4)
GraphMVP
(1.6)
(1.2
(2.1
75.9 (0.5)
63.1 (0.4)
(3.9
77.0(1.)
77.7(0.6)
MoICLR
(2.1)
82.4
(3.5
(0.2)
(1.4)
78. 78.1(0.5)
79.6(1.9)
Uni-Mol
(0.6)
(0.7
91.9
79.6 (0.5)
69.6 (0.I)
65.9 (1.3)
80.8(0.3)
82.1(1.))
GEM
72.4 (0.4)
81.6
90.1
(0.1)
69.2 (0.4)
67.2 (0.4)
80.6(0.9)
81.7(0.5)
InstructMol
73.3(0.8)
85.9(1.3)
92.5(2.1)
79.9(0.6)
70.8(0.4)
67.4(0.9)
Ours
73.6(0.1)
86.6(0.5)
85.2(0.1)
80.6 (0.3)
68.8(0.6)
68.5 (0.3)
85.0 (1.2)
81.8 (0.1)
different guning stratogies
Pyper parameters
Parametors
13
775
32.10
32
225
ROCALIC
ROGAUC
76.5
19.0
16-54
7623
65
C4-24
Cooche
Number of Tunabli Parametters 1003)
Figure 2. (a) Test curves for various tuning methods. (b) Impact of different hyperparameters. where the y-axis denotes the ROC-AUC score (%) and the x-axis represents the LoR A rank of node encoder hyperparameters. (c) Comparison of ROC-AUC (%) and trainable parameter count between MIPT with varying rank values and full fine-tuning on the Tox21 dataset.
LoRA ranks (8, 16, 32, 64), showing that lower ranks main- tain high performance while reducing parameter overhead, highlighting the efficiency of low-rank adaptation. Figure 2 (c) illustrates the trade-off between trainable parameters and performance, confirming that our approach significantly reduces computational costs without sacrificing accuracy compared to full fine-tuning. These results demonstrate that low-rank prompts provide a scalable and efficient alterna- tive traditional fine-tuning. Additionally, analysis of label distributions across datasets (Appendix C.I) underscores method's robustness to noisy data and adaptability to real-world scenarios.
Table Ablation analysis different configurations on BBBP,
ClinTox, BACE, and SIDER datascts.
Module
Loss
Datusets
LoRA
MGIP
ExPP
Lemi
bbbp
ClinTox
BACE
Sider
68.72
67.76
86.41
60.75
v
72.45
80.19
85.22
67.17
V
72.88
81.18
86.42
67.76
4
/
V
69.43
70.99
85.51
61.26
/
V
72.69
85.46
67.23
V 73.18
80.70
85.51
66.17
V
73.62
81.21
86.52
68.52
Hyperparameter Analysis (RQ4). The curves in Fig- ure 2 (a) show that method achieves higher ROC-AUC scores and remains stable across training epochs, outper- forming FT, GPF, and GPF-plus, demonstrating the effec- tiveness of low-rank adaptation for generalization and ef- ficiency. Figure/ (b) compares the performance of four pre-trained models on the Tox21 dataset under different
6. Conclusion
In this work, we propose Multilevel Informed Prompt Tun- ing (MIPT), novel framework for enhancing pre-trained molecular encoders in molecular property prediction tasks. MIPT employs lightweight, multi-level prompt learning
8

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
network and noise penalty mechanism to bridge the gap between pre-training and downstream tasks, while mitigat- ing noise at both node and graph levels. Experiments public and real-world datasets demonstrate MIPT's supe- rior performance, highlighting its potential to advance drug discovery and materials science.
Kipf, T. N. and Welling, M. Variational graph auto-encoders. arXiv preprint 161 2016.
Li, J., Sun, X., Li, Y., Li, Z., Cheng, H., and Yu, J. X. Graph intelligence with large language models and prompt learn- ing. In Proceedings of the 30th ACM SIGKDD Con- ference on Knowledge Discovery and Data Mining, PP. 6545-6554, 2024.
learm-
Impact Statement
Liu, S., Demirel, M. F., and Liang, Y. N-gram graph: Simple unsupervised representation for graphs, with applications to molecules. Advances in neural information processing systems, 32, 2019.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Liu, S., Wang, H.,, Liu, W., Lasenby, J., Guo, H., and Tang, J. Pre-training molecular graph representation with 3d geometry. arXiv preprint arXiv:21 10.0772, 2021.
References
Tox21 challenge.
http://tripod.niin g Cox21/challenge// Accessed: 2017-09-27.
Liu, Z., Yu, X., Fang, Y., and Zhang, X. Graphprompt: Uni- fying pre-training and downstream tasks for graph neural networks. In Proceedings of the ACM Web Conference 2023, pp. 417-428, 2023.
Arpit, D., Jastrzebski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., al. closer look at memorization in deep networks. In International conference on machine learning, pp. 233-242. PMLR, 2017.
Lu, C., Liu, Q. Wang, C., Huang, Z.,, Lin, P., and He, L., Molecular property prediction: A multilevel quantum interactions modeling perspective. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 1052-1060, 2019.
Bianchi, F. M. and Lachi, V. The expressive power of pooling in graph neural networks. Advances in neural information processing systems, 36:71603-7161 2023.
Martins, I. F., Teixcira, A. L., Pinheiro, L., and Falcao, A. O. A bayesian approach to in silico blood-brain barrier penetration modeling. Journal of chemical information and modeling. 52(6):668-1167 1697, 2012.
Chibani, S. and Coudert, F-X. Machine learning approaches for the prediction of materials properties. Apl Materials, 8(8), 2020.
Richard, A. M., Judson, R. S., Houck, K. A., Grulke, C. M.,
Fang. T., Zhang, Y., Yang, Y., Wang, C., and Chen, L.. Uni- versal prompt tuning for graph neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Volarath, P., Thillainadarajah, I., Yang, C., Rathman, J., Martin, M. T., Wambaugh, J. F., et al. Toxcast chemical landscape: paving the road to 21st century toxicology. Chemical research in toxicology, 2905):225511255 2016.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual leamn ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Rong, Y., Bian, Y., Xu, T., Xie, W., Wei, Y., Huang, W., and Huang, J. Self-supervised graph transformer on large- scale molecular data. Advances in neural information processing systems, 13:12559-12577 2020.
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q. Gesmundo, A., Attariyan, M., and Gelly, S. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pp. 2790- 2799. PMLR, 2019.
Shen, and Nicolaou, C. A. Molecular property prediction: recent trends in the era of artificial intelligence. Drug Discovery Today: Technologies, 32:29-36 -3, 2019.
Shi, J., Yin, X., Chen, Y., Zhang, Y., Zhang, Z., Xie, Y., and Qu, Y. Multi-memory matching for unsupervised visible- infrared person re-identification. In European Conference on Computer Vision, pp. 456-474. Springer, 2024.
Hu, W., Liu, B., Gomes, J., Zitnik, M., Liang, P., Pande, V., and Leskovec, J. Strategies for pre-training graph neural networks. arXiv preprint 19X5:: 12000....22...... 2019.
Stärk, H., Beaini, D., Corso, G., Tossou, P., Dallago, C., Günnemann, S., and Liò. P. 3d infomax improves gnns for molecular property prediction. In International Con- ference on Machine Learning, pp. 20502. PMLR, 2022.
Huang, Q., Ren, H., Chen, P., Kržmanc, G., Zeng, D., Liang, P. S., and Leskovec, J. Prodigy: Enabling in-context learning over graphs. Advances in Neural Information Processing Systems, 36, 2024.

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
Subramanian, G., Ramsundar, B., Pande, V., and Denny, R. A. Computational modeling of 8-secretase (bace- 1) inhibitors using ligand based approaches. Journal of chemical information and modeling, 16(10):1936-199, 2016.
Xia, J., Zheng, J., Tan, C., Wang, G., and Li, S. Z. Towards effective and generalizable fine-tuning for pre-trained molecular graph models. bioRxiv, pp. 2022-02, 2022a.
Xia, J., Zhu, Y., Du, Y., and Li, S. Z. Pre-training graph neural networks for molecular representations: Retro- spect and prospect. In ICMI. 2022 2nd Al for Science Workshop, 2022b.
Sun, M., Zhou, K., He, X., Wang, Y., and Wang. X. Gppt: Graph pre-training and prompt tuning to gener- alize graph neural networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1717-1727, 2022a.
Xiong, Z.., Wang, D., Liu, X., Zhong, F., Wan, X., Li, X., Li, Z., Luo, X., Chen, K., Jiang, H., et al. Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. Journal of medicinal chemistry, 63(16):8749987660 2019.
Sun, R., Dai, H., and Yu, A. W. Does gnn pretraining help molecular representation? Advances in Neural Informa- tion Processing Systems, 35: 12096-12109, 2022b.
Xu, K., Hu, W., Leskovec, J., and Jegelka, S. How powerful are graph neural networks? arXiv preprint arXiv:1810.0080 2018
Sun, X., Zhang, J., Wu, X., Cheng, H., Xiong, Y., and Li, J. Graph prompt learning: A comprehensive survey and beyond. arxiv 2023, arXiv preprint arXiv:231 1.
Xuhong, L.,, Grandvalet, Y., and Davoine, F. Explicit induc- tive bias for transfer learning with convolutional networks. In International Conference on Machine Learning, pp. PMLR, 2018.
Sun, X., Cheng, H., Li, J., Liu, B., and Guan, J. All in one: Multi-task prompting for graph neural networks. In Proceedings of the 29th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 2120-2131, 2023a.
Yang, K., Swanson, K., Jin, W., Coley, C., Eiden, P., Gao, H., Guzman- A., Hopper, T., Kelley, B., Mathea, M., et al. Analyzing learned molecular representations for property prediction. Journal of chemical information and modeling, 59(8):3370-3388 2019.
Sun, X., Zhang, J.. Wu, X., Cheng, H., Xiong, Y., and Li, J. Graph prompt learning: A comprehensive survey and beyond. arXiv preprini arXiv:2311.16534 2023b.
Taskinen, and Yliruusi, J. Prediction physicochemical properties based on neural network modelling. Advanced drug delivery reviews, 15(9):1163-11 2003.
Zhang, J., Xiao, X., Huang. L.-K., Rong, Y., and Bian, Y. Fine-tuning graph neural networks via graph topology induced optimal transport. arXiv preprint arXiv: M2Xiv:2203.. 2tii:2220311553. 2022.
Veličković, P., Fedus, W., Hamilton, W. L., Lió, P., Bengio, Y., and Hjelm, R. D. Deep graph infomax. arXiv preprint arXiv: 10341, 2018.
Zhou, G., Gao, Z., Ding, Q., Zheng, H., Xu, H., Wei, Z., Zhang, L., and Ke, G. Uni-mol: A universal 3d molecular representation learning framework. 2023.
Wang, J., Deng, Z., Lin, T., Li, W., and Ling, S. A novel prompt tuning for graph transformers: Tailoring prompts to graph topologies. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3116-3121 2024.
Zhou, G., Wang, Z., Yu, F., Ke, G., Wei, Z., and Gao, Z. S-molsearch: 3d semi-supervised contrastive learn- ing for bioactive molecule search. arXiv preprint arXiv:2499.07402. 2024.
Wang, Y., Wang, J., Cao, Z., and Barati Farimani, A. Molec- ular contrastive learning of representations via graph neu- ral networks. Nature Machine Intelligence, 4(3):279-27, 2022.
Wu, F., Jin, Li, and Li, S. Z. Instructor-inspired machine learning for robust molecular property predic- tion. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Ge- niesse, C., Pappu, A. S., Leswing, K., and Pande, V. Moleculenet: a benchmark for molecular machine learn- ing. Chemical science, 9(2):513-330, 2018.
10

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
A. Proof: Effectiveness of LoRA Updates in GNNs
Consider a GNN layer, the feature update at layer k + for node is expressed as:
h a W A AGG (( (( e((o))))
(22)
where (A) Raxd represents the node embeddings at layer and W E Rdxd is trainable weight matrix. The function a+)) denotes a non-linearity such as ReLU.
Let M E Raxd stack all aggregated messages, and let Raxd be the gradient of the loss witt. outputs. Then:
g V w C = 8
(23)
Since M results from summing over neighbors in sparse graphs, typically low-rank or approximately so. Thus, is approximately low-rank. From the Eckart- Young-Mirshe theorem, the best rank-r approximation of under Frobenius norm
AW Σ Couu,
(24)
where y UoVT is the SVD of gradient, and o; the top singular values. This can be written as AW AB with:
A A= E Rdxr, B B E Rxxd
(25)
which is precisely the LoRA parameterization. Following PAC- PAC-Bayes or norm-based generalization analysis:
ED (outt S (urain) O |AWWI?
(26)
n
where is the number of training samples and || || denotes the Frobenius norm. Since ||WI| = ||AB] < | the
low-rank constraint imposed by LoRA acts as an implicit regularizer, effectively reducing model complexity and improving generalization performance.
Then, the updated node feature equation becomes:
h (+++)
(WWo + AB) 1(0) + AGG (fn00:.. : ))N(())))
(27)
The key point is that the neighborhood (v) is solely determined by the static graph (V, E), which can be described via the adjacency matrix Aadj:
1, 0, otherwise
i(ee
Andi (u, v)
(28)
and does not depend on the model parameters. LoRAA modifies only the linear transformation term and does not alter the adjacency matrix neighborhood relations. N unaffected by the addition of AB.
B. Proof: Robustness of NPP
B.1. Derivation of Posterior Probability p(l ((s
Let Lels = (C) ,,. CN) a Gaussian Mixture Model (GMM) distribution. For a GMM with T components, the parameters include:
represent the observed classification loss values, and assume that each sample 4 follows
:े: The mixture coefficient for the t-th component.
Mr: The mean of the t-th Gaussian component.
af: The variance of the 1-th Gaussian component.

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
The posterior probability p(( Cix,) can be computed using Bayes' rule:
p(Cas (t) Te
p(t Cass)
(29)
Σj=1 P( (
For the k-th Gaussian component, the likelihood term given by:
1
(C u1)2 )
p( ())
exp
(30)
v /
20
Combining the likelihood and the prior, the posterior probability becomes:
(- (-1-iz77 (Ci1-11)
exp
20;
p(t C(b))
(31)
Σド= Tj 2200
cxp
( (C)s -1;;²²
203
B.2. EM Updates
E-STEP: COMPUTE POSTERIOR PROBABILITIES
It aims to estimate the posterior distributions of the loss, i.e. P( 14, 1,,0) by using the current estimated parameters
Bold: : (aooty J. Hence, we compute the posterior probabilities p(t (2n)) using the current estimates of n. 14 and of [
(m)
Tt pl p( (n) ,a! Σラ= T3
(t) Cdna)
(32)
(n) p( Mi Mi .0j
(x)
M-STEP: PARAMETER UPDATES
Using the posterior probabilities (t 2.,,,, we update the parameters as follows:
Mixture Coefficient:
NN (( (1s)
Ei
(n+1)
(33)
N
Mean:
N N ((m) Loba) . Lids
11
(x+1)
ii=
(34)
E p(a)(1 Liaa)
Variance:
Dim p( (./))= (C - Hi 1
2(++
(35)
Σ p(m)(( (2,
B.3. Proof NPP Robustness
The Noise Prompt Penalty (NPP) framework leverages the posterior probability p(t (ij)) to compute the weight w; for each sample:
Wi p( C.r))
(36)
By thresholding the weights w; based the mean value, we define:
1,
if w. > mean(w,),
b = 0, otherwise.
(37)
This filtering ensures that low-confidence (likely noisy) samples are excluded from the loss computation. The final NPP loss
12

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
Algorithm 1 Noise Penalty Sample Training Procedure
Input: Downstream task model ge, labeled data D, classification loss Cols, noise prompts penalty loss LNPP, learning rate i/, noise weight X, prompt-related parameters Py. LoRA parameters 6. Initialize model parameters GMM parameters
Tt, Pt, dt-
Output: Optimized model parameters
1: for t = 1,2
2: for epoch=1.max epoch do
3: for each sample (G,; Me) D do
4: Compute prediction: D; < (o:):
5: Compute classification loss using Eq. 14;
6: end for
7: GMM Modeling: wi; + GMM
8: while not converged or max iterations reached do
9: Update wi using EM algorithm
end while
11: Update confidence vi; using Eq. 17 Compute noise penalty loss using Eq. 13;
12:
13:
for iteration from to T do
14:
Liotal = Ccont + Nppi optimize the model parameters by Ltotal
15:
end for end for
Return 0, Pg. 8
is computed as:
Expp
Σ w log Ply 9(ho.)))
(38)
N
This framework achieves robustness by:
Utilizing the EM algorithm to iteratively refine the estimation of GMM parameters (me, Me, of), leading to improved separation of noise and trustworthy samples.
Filtering out low-confidence samples (ú;; 0), ensuring that only high-confidence samples contribute to the model optimization.
The robustness of NPP is thus guaranteed by the iterative EM updates and selective loss computation, effectively mitigating the impact of noisy samples. Pseudocode presented in Algorithm
C. More Experiment Settings.
C.1. Details of Datasets.
Pre-training datasets. The dataset includes 2 million unlabeled molecules drawn from the ZINCI5 database, used for node-level self-supervised pre-training. For graph-level multi-task supervised pre-training, the preprocessed ChEMBL dataset is utilized, consisting of 456K molecules spanning 1310 distinct biochemical assays.
Downstream Task. To evaluate model performance, we utilized eight binary graph classification datasets from Molecu- leNet (Wu et al., 2018), described as follows:
BBBP (Martins et al., 2012): Evaluates blood-brain barrier penetration based membrane permeability.
Tox21 (tox): Provides toxicity data for 12 biological targets, including nuclear receptors and stress response pathways.
13

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
ToxCast (Richard et al., 2016): Contains toxicological measurements derived from over 600 in vitro high-throughput screening assays.
SIDER: A comprehensive database of marketed drugs and their associated adverse drug reactions (ADR), categorized into 27 system organ classes
Includes qualitative data distinguishing FDA-approved drugs from those that failed clinical trials due toxicity.
Clin'Tx:
A subset of PubChem BioAssay data, specifically curated using a refined nearest-neighbor analysis for validating virtual screening techniques.
MUV:
HIV: Contains experimental data assessing the inhibitory efficacy of compounds against HIV replication.
BACE (Subramanian et al., 2016): Provides qualitative binding data for inhibitors targeting human B-secretase 1.
Table 4. Dataset statistics.
Dataset
BBBP
Tox21
ToxCast
SIDER
Clintox
MUV
HIV
BACE
# Molecule
2039
7831
8575
1427
1478
93127
41127
1513
# Property
12
617
27
2
17
% Positive Label
76.44
6.24
12.60
56.76
50.61
0.31
96.49
45.67
% Negative Label
23.56
76.71
72.43
43.24
49.39
15.76
1.18
54.33
Unknown Label
17.05
14.97
0
84.21
2.33
C.2. Details of pre-training strategies.
We utilize five widely recognized strategies for pre-training GNN models, detailed as follows:
Initially proposed by Velickovic et al. (Veličković et al., 2018), this method acquires expressive representations for graphs or nodes by maximizing mutual information between graph-level and substructure-Ievel representations across various granularities.
Deep Graph Infomax:
Edge Prediction: A common graph reconstruction task employed by many models, such as GAE (Kipf & Welling, 2016), where the goal is to predict the existence of edges between pairs of nodes.
Attribute Masking: Introduced by Hu al. (Hu al., 2019), this technique involves masking node/edge attributes and then using GNNs to predict these attributes based the neighboring structure.
Context Prediction: Also proposed by Hu et al. (Hu et al., 2019), this approach uses subgraphs to forecast their surrounding graph structures, aiming to map nodes similar structural contexts to nearby embeddings.
In our model pre-training phase, adhere to the methodologies described by Hu et al. (2019) for the tasks of Infomax, EdgePred, AttrMasking, and ContextPred. Subsequently, we engage in supervised graph-level property prediction to further boost the efficacy of our pre-trained models.
Training Details. We perform five rounds of experiments with different random seeds for each experimental setting and report the average results. The projection head 0 selected from range of [1, 2, 3J-l MLPs with equal widths. The hyper-parameter of GPF-plus is chosen from the range 10,20. Further details on the hyper-parameter settings can be found in the appendix.
Implementation Details. All experiments were conducted on high-performance computing server equipped with an NVIDIA 3090 GPU (24 GB memory). The implementation was based on Python 3.9, PyTorch 1.12, and the torc geometric library. For the GNN architecture, we utilized Graph Isomorphism Network (GIN), configured with a hidden dimension of 300, 3 graph convolutional layers, ReLU activation, and batch normalization. The optimizer was Adam with a learning rate 0.001, dropout rate is 0.5, the mask probability 0.2. Pretraining tasks included node-level attribute prediction and
14

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
Sepalie sender
Irored Promoa waje turater
0.7
3.5-
34-
6/
24
3.5
14 Prekee
Figure 3. Test ROC-AUC of molecule property prediction using different tuning strategies with different pre-training strategics. Each point represents particular individual downstream task from (Fang et al., 2024) (left) and method (right).
graph-wise classification, with data augmentation strategies such as node masking, edge sampling, and feature perturbation applied during training. The experiments were conducted on MoleculeNet datasets, running for 100 epochs with batch size of 32. Performance evaluated using ROC-AUC, and each experiment was repeated five times to ensure reproducibility, with the mean and standard deviation reported.
D. Additional Experiments
D.1. More comparison experiments
Figure 3 compares the test ROC-AUC performance of molecular property prediction across different tuning strategies with various pre-training approaches. Each point represents the performance of individual downstream task, with GPF shown on the left and our method on the right. The red diagonal line indicates parity, where points above the line represent improved performance compared to fine-tuning.
The left plot reveals that the method from GPF exhibits negative transfer in several tasks, as some points fall below the diagonal, indicating degradation in performance due to ineffective knowledge transfer. In contrast, the right plot demonstrates that our proposed Instruct Prompt effectively mitigates negative transfer, resulting a more consistent improvement across tasks. This highlights the robustness of our approach in enhancing molecular representation learning and improving generalization across diverse downstream tasks.
D.2. More training cases.
We provide more training curves in Figure a supplement to Section 5.3. Our method achieves considerable performance improvement over other approaches during training. We note that some smaller datasets, such as Clintox and BBBP, the training curves exhibit greater fluctuations. This behavior is hypothesized to stem from the increased number of parameters in our model coupled with the constrained data availability. Crucially, despite these fluctuations, overfitting is not observed, suggesting that the model remains robust. In stark contrast, training on larger datasets yields much smoother performance curves, with fluctuations being significantly less prominent.
Table The number of tunable parameters for different tuning strategies. is frozen parameters.
FT
GPF
GPF-plus
Ours
Size of GNN(Encoder)
1.86M
1.86M*
1.86M
1.86M
Size of Prompt
0
0.3K
3-12K
0.115M
Size of Graph Linear Layer
0.6K-0. 18M
0.6K-0. 18M
0.6K-0. 18M
0.6K-0.18M
Size of Total Tunable Part
1. 12M
0.9K-0.2IM
3.6K-0.192M
0.175M-0.295M
# Params(%)
100
0.046-0.98
0.187-8.97
9.1-13.78
15

---

MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction
Hiv
Toxcast
80
ROC AUCOS)
64-
62
50-
60-
GFF
GPF-pkus
Durs
FT
GPF
CiFF-plus
Ours
10
20
30
40
50
10
20
30
40
50
Epochs
Epochs
bbbp
Clintox
80-
70-
ROC AlCC
75
85
60
55
GPF
GPF-pkus
Ours
FT
GPF
CFF plus
Ours
10
20
30
40
50
20
30
40
50
Epochs
Epochs
Figure 4. Test curves of different tuning strategies pretrained GINs. Solid and dashed lines indicate training and validation curves, respectively.
Table Ablation analysis of different configurations Tox21, Toxcast, HIV, MUV, BBBP, Clintox, BACE and SIDER datasets.
MODULE
LOSS
DATASETS
LORA
MGIP
LNPP
LEONT
Tox21
TOXCAST
HIV
MUV
BBBP
CLINTOX
BACE
SIDER
75.31
64.34
75.95
82.44
68. 72
67.76
86.41
60.75
4
80.57
68.78
79.97
84.47
72.45
80.19
85.22
67.17
1
80.19
67.70
78.50
83.07
72.88
81.18
86.42
67.76
76.40
68.55
75.03
79.48
69.43
70.99
85.51
61.26
V
V
V
80.29
68.27
80.42
84.71
72.69
80.27
85.46
67.23
V
V
80.45
68.52
79.84
82.45
73.18
80.70
85.51
66.17
V
80.60
68.81
81.76
84.96
73.62
81.21
86.52
68.52
D.3. More ablation study
Here, Table 6 shows the comprehensive ablation results eight datasets. A comparison of the parameters of the different high-efficiency fine-tuning methods is shown in Table 5. From the experimental results, can conclude that our method can achieve the optimal performance with fewer parameters, showing superiority.
E. Limitations and Future Work
As discussed in Section while our method significantly outperforms state-of-the-art baselines, has certain limitations in few-shot learning. This is primarily due to the constraints of GMM modeling, which requires at least two samples per class for effective learning, limiting its applicability in few-shot scenarios. Additionally, GMM introduces some computational overhead, though its resource consumption remains modest. Furthermore, our method exhibits higher standard deviations across multiple runs with different ranks, likely due to model uncertainty in handling diverse graph structures. Addressing the integration of node-level and graph-level features remains an open challenge for future research.
16


Reviews:
[{"review_id": "eVYcqzOoSs", "reviewer": "Reviewer_Swha", "summary": "This paper addresses the challenge of prompt tuning for pretrained models in molecular property prediction. It introduces a multi-level prompt learning module to enhance task adaptation and a noise penalty mechanism to improve robustness, adaptability, and efficiency. Extensive experimental evaluations demonstrate the effectiveness and superiority of the proposed approach across various molecular prediction tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer Swha:\n\nThanks for your positive review of our paper and for your thoughtful comments.\n\n**For W1 model versatility.**  \n\n- Thank you for your positive feedback on our approach. We appreciate your recognition of the potential our MIPT framework for other graph-level applications. \n\nBelow is a comparison of the performance based on *RMSE* of different tuning strategies on regression tasks.\n|  Tuning Strategy  | ESOL | Lipo |\n|-------|-------|-------|\n| FT | 1.1262 | 0.6942|\n| GPF | 1.1125 | 0.6832 |\n| GPF-plus | 1.1104 | 0.6789 |\n| Ours | 0.6788 | 0.6702 |\n\n- In our initial research, we focus on molecular property prediction, where MIPT demonstrated exceptional performance. To explore its broader applicability, we extended our method to regression tasks according to your constructive comments. The experimental results indicate that our approach maintains superior performance in these tasks. We remain committed to further investigating and validating the framework's performance across a wider range of graph-level tasks.\n\n**For W2 model complexity.** Since fine-tuning time is often difficult to compare fairly due to differences in hardware and system environment, we analyze the training costs based on the number of tunable parameters.\n\nTable I. The number of tunable parameters for different tuning strategies. \\* is the frozen parameters.\n\n| Tuning Strategy | Size of GNN (Encoder) | Size of Prompt | Size of Graph Linear Layer | Size of Total Tunable Part |\n|----------------|---------------------|---------------|----------------------|----------------------|\n| FT            | 1.86M               | 0             | 0.6K-0.18M          | 1.92M-2.14M         |\n| GPF           | 1.86M*               | 0.3K         | 0.6K-0.18M          | 0.9K-0.21M          |\n| GPF-plus      | 1.86M*               | 3-12K        | 0.6K-0.18M          | 3.6K-0.192M         |\n| Ours          | 1.86M*               | 0.115M       | 0.6K-0.18M          | 0.175M-0.295M       |\n\n* **FT** requires the highest number of tunable parameters (1.92M-2.14M), leading to the highest training cost.\n* **GPF** and **GPF-plus** significantly reduce the number of tunable parameters by freezing the GNN encoder. This makes them more parameter-efficient compared to FT.\n* **Our method** also freezes the GNN encoder but introduces a larger prompt size (0.115M). Despite this, the total number of tunable parameters (0.175M-0.295M) remains significantly lower than FT, suggesting a balance between efficiency and effectiveness.\n\nIn summary, ​**GPF and GPF-plus are the most parameter-efficient**​, while ​**our approach achieves a middle ground**​, tuning more parameters than GPF but significantly fewer than FT, likely leading to a moderate training cost.\n\n**For Q3. the datasize of fine-tune:** In our study, we use the entire training set during the fine-tuning phase. This approach, which fine-tunes only a subset of model parameters rather than the entire network (see our discussion in ​**Q2**​), is a common practice in this field. Prior works [1,2,3,4] also employ the full training dataset for model fine-tuning, ensuring that the model benefits from as much labeled information as possible.\n\nWe appreciate the reviewer for spending time reviewing our paper and offering valuable suggestions. If you have any further questions, please tell us and we are willing to address your concerns.", "claims_and_evidence": "Yes, this paper is well-written and claims are well-supported", "methods_and_evaluation_criteria": "Yes, the evaluation is comprehensive and the experimental setup is reasonable, while I'm curious about the effectiveness of the proposed method on other graph-related applications.", "theoretical_claims": "Yes.", "experimental_designs_or_analyses": "The experimental setup is well-structured and includes multiple baselines", "relation_to_broader_scientific_literature": "prompt learning", "essential_references_not_discussed": "no", "other_strengths_and_weaknesses": "Please refers to the above parts and also the questions.", "other_comments_or_suggestions": "Please refers to the above parts and also the questions.", "questions_for_authors": "1. This method does not appear to be specifically designed for molecular property prediction. Is the prompt-tuning approach also effective for other tasks?\n\n2. How does the fine-tuning time compare across different fine-tuning strategies?\n\n3. How much data is required to fine-tune the model effectively?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2sdDTnmddT", "reviewer": "Reviewer_kVKq", "summary": "The paper introduces a novel framework called Multilevel Informed Prompt Tuning (MIPT) aimed at enhancing the performance of pretrained GNNs in molecular property prediction tasks. MIPT employs a lightweight multilevel prompt learning module to capture task-specific knowledge at both node and graph levels, while incorporating a noise penalty mechanism to mitigate the impact of irrelevant information. Experimental results demonstrate that MIPT outperforms baseline models across various molecular tasks, showcasing its effectiveness, scalability, and broad applicability, while also highlighting areas for future research, particularly in few-shot learning and stability across different graph structures.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer kVKq:\n\nThank you for your valuable and constructive comments. We have revised the paper according to your suggestions. \n## Weakness\n\n**W1: In this paper, only the benchmark of classification tasks is studied, and it is necessary to further increase the generality of other types of tasks to illustrate tasks, such as regression tasks.**\n\nIn our paper, while the primary focus was on benchmark classification tasks, we recognize the importance of demonstrating the generality of our approach across other types of tasks, such as regression. We extended our experiments to regression tasks on Table I. The supplementary results indicate that our method maintains its superiority even in the regression setting.\n\nTable I. Comparison of the performance of different tuning strategies on regression tasks based on MAE.\n\n|  Tuning Strategy  | ESOL | Lipo |\n|-------|-------|-------|\n| FT | 1.1262 | 0.6942|\n| GPF | 1.1125 | 0.6832 |\n| GPF-plus | 1.1104 | 0.6789 |\n| Ours | 0.6788 | 0.6702 |\n\n**W2: The SOTA performance in Table 2 is not available on all datasets.** Thank you for pointing this out. Indeed, in molecular representation learning, the scope and design of different models can vary significantly, often incorporating specialized features or data. For example, UniMol leverages 3D information, and InstructMol adopts pseudo-labeling strategies to enhance confidence. While these SOTA methods focus on addressing certain limitations in molecular representation, our primary goal is to bridge the gap between pre-training and fine-tuning. We believe that even if not all molecular properties achieve SOTA performance, they still hold significant value.\n\n**W3: The author does not have open source code.** We will make the code publicly available after the submission is published.\n\n**W4: The two LoRA module in Sec.4.1 and Sec.4.2 should provide more details on how they differ.**\nWe emphasis that the node-level LoRA is used for the node features of the adaptive learning molecule, and the LoRA on the graph-level features is a variant of LoRA, which is used as a prompt for adaptive learning of graph-level features.\n\n**W5: More benchmarks about Graph Prompt-based Methods.**\nWe have conducted a comprehensive comparison with existing graph prompt benchmarks. As shown in the Table II, our method outperforms both GPPT and GraphPrompt on several key datasets, particularly on BBBP, Toxcast, SIDER, HIV, and BACE.\n\nTable II.  Comparison of benchmarks about graph prompt for the models pre-trained by Edge Prediction based on RUC-AOC(%).\n\n|    | BBBP | Tox21| Toxcast| SIDER| Clintox| MUV| HIV| BACE|\n|-------|-------|-------|-------|-------|-------|-------|-------|-------|\n| GPPT | 64.13 | 66.41 | 60.34 | 54.86 | 59.81 | 63.05 | 60.54 | 70.85 |\n| GraphPrompt | 69.29 | 68.09 | 60.54 | 58.71 | 55.37 | 62.35 | 59.31 | 67.70 |\n| Ours | 72.73 | 80.82 | 67.44 | 79.46 | 79.29 | 80.02 | 78.68 | 82.91 |\n\n**W6: The LoRA have already been used in many tasks, such as NLP tasks. The technical novelty of this paper is limited.**\n\nWe agree that LoRA is an excellent and widely-used technique in many domains, including NLP. However, our paper does not claim novelty for the LoRA component itself. Instead, our main contribution lies in multilevel graph informed prompt and noise penalization mechanisms. This combination uniquely bridges the gap between pre-training and fine-tuning in molecular graph learning.\n\n## Questions\n\n**Q1: What is the difference between $L_{NPP}$ and $L_{cls}$ in Figure 1?**\n\n$L_{cls}$ (Eq. 14) is the standard classification loss (binary cross-entropy loss) used to train the model on the downstream task. In contrast, $L_{NPP}$ (Eq. 18) adds a Noise Penalty Mechanism to $L_{cls}$.\n\n**Q2: The author emphasizes the multi-level graph prompt, essentially using the LoRA module twice, so the increase in performance could be due to an increase in the amount of training parameters?**\n\nTable III. The number of tunable parameters for different tuning strategies. \n| Tuning Strategy | Size of Total Tunable Part |\n|----------------|---------------------|\n| FT            | 1.92M-2.14M         |\n| GPF          | 0.9K-0.21M          |\n| GPF-plus      | 3.6K-0.192M         |\n| Ours          | 0.175M-0.295M       |\n\nWhile it is true that our method employs the LoRA module at multiple levels, the increase in performance cannot be solely attributed to a higher number of tunable parameters. We had provided the parametric analysis in tab. 5 in the appendix. Although our approach uses more parameters than GPF/GPF-plus, it is still far below FT, which utilizes 1.92M-2.14M parameters yet performs worse. This clearly indicates that our performance gains stem from the unique integration of the multilevel graph-informed prompt and the noise penalization mechanism, which effectively bridges the gap between pre-training and fine-tuning in molecular graph learning.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "When comparing the fine-tuning methods, the authors only compared the FT and GPF/GPF-plus strategies, and the results of other PEFT strategies should be added to illustrate the superiority of the methods. In addition, the authors should add more benchmarks on graph prompts to compare.", "theoretical_claims": "I carefully examined the theory and its proof and found no major errors.", "experimental_designs_or_analyses": "This paper presents a comprehensive evaluation of multiple benchmarks, comparing not only the SOTA model that has been pre-trained and fine-tuned but also the SOTA model specifically for molecular property prediction. The experimental design is well-conceived and effectively addresses the research objectives.", "relation_to_broader_scientific_literature": "This paper introduces the innovative concept of molecular prompt tuning, which effectively bridges the gap between pre-training and fine-tuning without altering the molecular structure, while also enhancing interpretability.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "**Strengths**\n-This paper proposes MIPT, including multi-level prompt tuning and noise penalty mechanism, which provide a new solution for molecular knowledge transfer.\n- This paper is well-motivated and easy-to-follow.\n- The contributions are significant and somewhat new.\n\n**Weaknesses**\n- In this paper, only the benchmark of classification tasks is studied, and it is necessary to further increase the generality of other types of tasks to illustrate tasks, such as regression tasks.\n- The SOTA performance in Table 2 is not available on all datasets.\n- The author does not have open source code.\n\nI am willing to consider raising the score based on your rebuttal to the following questions:\n- The two LoRA module in Sec.4.1 and Sec.4.2 should provide more details on how they differ.\n- Authors should compare their work with other benchmarks about graph prompt.\n- The LoRA have already been used in many tasks, such as NLP tasks. The technical novelty of this paper is limited.", "other_comments_or_suggestions": "**Minor Weaknesses**\n- The molecular graph provided in the upper and lower sections of Figure 1 is not identical.\n- “MIT” in L430 should be “MIPT”.\n- L696, Implementation Details are misaligned.", "questions_for_authors": "- What is the difference between $L_{NPP}$ and $L_{cls}$ in Figure 1?\n- The author emphasizes the multi-level graph prompt, essentially using the LoRA module twice, so the increase in performance could be due to an increase in the amount of training parameters? Please specify.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VWBkuB1F4P", "reviewer": "Reviewer_Cxne", "summary": "This manuscript introduces a novel Multilevel Informed Prompt Tuning (MIPT) framework designed to enhance pre-trained molecular encoders for molecular property prediction tasks. The key contributions include a multi-level prompt learning network and a noise penalty mechanism. The proposed prompt learning network effectively mitigates the gap between pre-training and downstream tasks, while the noise penalty mechanism addresses potential mismatches between pre-trained representations and task-specific representations. Extensive experiments on both public and real-world datasets demonstrate that MIPT achieves significant improvements in molecule-related tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer Cxne:\n\nThank you for your valuable and constructive comments. We have revised the paper according to your suggestions. \n\n**W1: LoRA, as a parameter-efficient fine-tuning method, has been widely adopted in the LLM domain. The manuscript should more clearly clarify LoRA's unique advantages in molecular property prediction tasks and how it has been specifically optimized for molecular graph data.** \n\nOur work is the first to adopt LoRA for fine-tuning GNNs in molecular property prediction. In NLP, LoRA is typically employed to cut down training costs. However, our goal is to construct graph prompts that capture the relationships between graph structure and node features, thereby optimizing LoRA specifically for molecular graph data while maintaining parameter efficiency.\n\n\n**W2: Different from  Pin-tuning.** Pin-tuning is designed to be based on adaptor fine-tuning strategy for few-shot learning. Unlike Pin-Tuning, our method bridges pre-training and fine-tuning via multilevel graph prompts that align node/graph structures with task requirements. Additionally, we incorporate a noise penalty to mitigate mismatches between pretrained representations and downstream tasks.\n\n**W3: What is the role of $\\epsilon^{(k)}$ in Eq. (5)?**\nIn Eq. (5), $\\epsilon^{(k)}$ is a learnable parameter that modulates the contribution of the node's own features relative to its neighbors during the update at the $k$-th layer. Essentially, it scales the self-feature term, allowing the network to adaptively balance the importance of a node's current representation with the aggregated information from its neighbors.\n\n**W4: In Table 2, the performance improvement on some datasets is not significant. Additional analysis should be provided.**\n\nWe acknowledge that the performance improvement of our method is modest on some datasets. We are aware that public molecular datasets are limited, and different SOTA methods tackle these challenges in various ways. For instance, Uni-Mol incorporates 3D information, while InstructMol uses pseudo-labels to enhance model confidence. In contrast, our approach mainly aims to bridge the gap between pre-training and fine-tuning. Although our method may not achieve SOTA performance on every molecular property, the enhancements it brings in transferability and robustness are still of great significance.\n\n**W5: Algorithm proofreading.**\nWe have thoroughly proofread and revised Algorithm 1 to ensure its clarity and correctness.\n\n**W6: The manuscript does not provide the code.**\nWe will make the code publicly available after the submission is published.\n\n**Minor issues.** For typos, we’ve corrected error and updated the description in the paper.\n\nWe appreciate the reviewer for spending time reviewing our paper and offering valuable suggestions. If you have any further questions, please tell us and we are willing to address your concerns.", "claims_and_evidence": "Yes, this manuscript provides extensive experimental results, including comparisons with baseline models, ablation studies, and hyperparameter experiments.", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "The manuscript includes a theoretical proof related to the Noise Prompt Penalty (NPP) to demonstrate its robustness in mitigating the impact of noisy samples.", "experimental_designs_or_analyses": "In Table 2, the performance improvement on some datasets is not significant. Additional analysis should be provided.", "relation_to_broader_scientific_literature": "The manuscript presents a clear and well-structured motivation, precisely identifying the key challenges of pre-trained GNNs in molecular property prediction.", "essential_references_not_discussed": "[1]  Pin-Tuning: Parameter-Efficient In-Context Tuning for Few-Shot Molecular Property Prediction.\n[2] MMGNN:AMolecularMerged Graph Neural Network for Explainable Solvation Free Energy Prediction", "other_strengths_and_weaknesses": "Strengths \n\n1) The manuscript presents a clear and well-structured motivation, precisely identifying the key challenges of pre-trained GNNs in molecular property prediction. It highlights the limitations of existing methods and naturally introduces its core solution。\n\n2) The manuscript proposes multi-level prompt learning to extract task-specific information at both the node and graph levels, enabling the model to better adapt to downstream tasks and effectively bridging the gap between pre-training and real-world applications.\n\n3) The manuscript employs a Gaussian Mixture Model (GMM) to model the confidence distribution of samples, introducing a noise penalty mechanism to suppress irrelevant noise, allowing the model to focus on truly meaningful features and enhancing overall performance.\n\n4) The proposed method achieves significant performance improvements in molecular property prediction tasks across multiple pre-trained models.\n\nWeaknesses\n\n1) LoRA, as a parameter-efficient fine-tuning method, has been widely adopted in the LLM domain. The manuscript should more clearly clarify LoRA's unique advantages in molecular property prediction tasks and how it has been specifically optimized for molecular graph data.\n\n2) What is this work different from Pin-tuning [1]?\n\n3) What is the role of \\(\\epsilon^{(k)}\\) in Eq. (5)?\n\n4) In Table 2, the performance improvement on some datasets is not significant. Additional analysis should be provided.\n\n5) Algorithm 1 needs careful proofreading\n\n6) The manuscript does not provide the code.\n\n[1] Pin-Tuning: Parameter-Efficient In-Context Tuning for Few-Shot Molecular Property Prediction.", "other_comments_or_suggestions": "The manuscript requires more careful proofreading to avoid some nitpicks. For example:\n1. In line 24, \"MIT\" should be corrected to \"MIPT\".  \n2. In line 216, there is a missing space after \"conditions\".  \n3. In line 341, the last numerical value should not be in boldface.", "questions_for_authors": "Please refer to Other Strengths And Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2qicKDHc5n", "reviewer": "Reviewer_dDZn", "summary": "The paper introduces Multilevel Informed Prompt Tuning (MIPT), a framework that enhances pretrained Graph Neural Networks for molecular property prediction. MIPT significantly outperforms existing methods, achieving higher ROC-AUC scores while reducing the number of trainable parameters. Key contributions include a multilevel prompt learning module for capturing task-specific knowledge, a noise penalty mechanism to mitigate irrelevant information, and low-rank adaptation for efficient tuning.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer dDZn:\n\nThank you for your valuable and constructive comments. We have revised the paper according to your suggestions. \n\n## Experimental Designs Or Analyses\n\n**1. Why choose LoRA**: To learn the features of both node-level and graph-level, we used multi-level fine-tuning, but in order not to increase the training cost, we chose lightweight LoRA. Although there are alternative methods such as Adapter tuning [1] and BitFit [2], previous study [3] have shown that LoRA achieves the best balance between efficiency and performance.\n\n[1] Parameter-Efficient Transfer Learning for NLP.\n\n[2] Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models.\n\n[3] LoRA: Low-Rank Adaptation of Large Language Models.\n\n**2. Not all datasets were evaluated:** We apologized for the mistake in title. Due to space, we did not include all the data, which we will add to the appendix. Here we supplement the performance results of other datasets. \n\nTable. Ablation analysis of different configurations on Tox21, Toxcast, HIV, and MUV datasets based on RUC-AOC(%).\n|   Module |  Loss  | Tox21 | Toxcast | HIV | MUV |\n|-------|-------|-------|-------|-------|-------|\n| LoRA | $L_{cont}$ | 75.31 | 64.34 | 75.95 | 82.44 |\n| MGIP | $L_{cont}$ | 80.57 | 68.78 | 79.97 | 84.47 |\n| LoRA+MGIP| $L_{cont}$ | 80.19 | 67.70 | 78.50 | 83.07 |\n| LoRA | $L_{NPP}$+$L_{cont}$ | 76.40 | 68.55 | 75.03 | 79.48 |\n| MGIP | $L_{NPP}$+$L_{cont}$ | 80.29 | 68.27 | 80.42 | 84.71|\n| LoRA+MGIP | $L_{NPP}$ | 80.45 | 68.52 |  79.84 | 82.45 |\n| LoRA+MGIP | $L_{NPP}$+$L_{cont}$ | 80.60 | 68.81 | 81.76 | 84.96 |\n\n\n**3. The ablation experiment does not include the random noise mask.** Two parts of our approach mention noise. The random node mask discussed in the method is used to augment the original features, which are included in the GIP and the ablation experiment (MGIP). The noise penalty mechanism, aimed at reducing the uncertainty of the method and increasing confidence, has been applied in the ablation analysis (NPP).\n\nWe appreciate the reviewer for spending time reviewing our paper and offering valuable suggestions. If you have any further questions, please tell us and we are willing to address your concerns.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes.Mainly experiments.", "experimental_designs_or_analyses": "1. LoRA has proven its effectiveness through ablation experiments, but why weren't other feature extraction methods used? Was there a comparison with other methods to make the final decision?\n2. The title of Table 3 is inconsistent with its content. The title mentions a comparison on Tox21, but the final results are based on BBBP. Additionally, why wasn't the comparison conducted across all datasets?\n3. Both the abstract and the method section mention the random noise mask, so why wasn't this discussed in the ablation study?", "relation_to_broader_scientific_literature": "1. Advances in GNNs\nMIPT builds on recent advancements in GNN architectures, which have been shown to effectively capture complex relationships in molecular data.\n2. Prompt Tuning Techniques\nMIPT extends this prompt tuning to LoRA, contributing to the body of work that explores how prompt-based methods can enhance performance across various domains.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "1. The H_v(k) in the line above Eq. 2 is incorrect.", "other_comments_or_suggestions": "No.", "questions_for_authors": "No.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces a multilevel informed prompt tuning (MIPT) framework to enhance pretrained GNNs for molecular property prediction. The approach incorporates multi-level prompt learning to capture task-specific knowledge and a noise penalty mechanism to enhance robustness. All reviewers support acceptance, highlighting the method's novelty, thorough experimentation, and practical relevance. While some reviewers point out limitations such as missing comparisons with additional prompt-based baselines or moderate gains on specific datasets, the rebuttal effectively addressed these concerns through extended results and further analysis.
Decision: Accept (poster)