Dynamic and Chemical Constraints to Enhance the Molecular Masked Graph Autoencoders
Authors: Jiahui Zhang
Wenjie Du
Yang Wang
Venue: NeurIPS 2025 poster ()
Keywords: molecule representation learning
graph auto-encoder
information bottleneck
Primary area: machine_learning_for_sciences
TL;DR: We boost masked graph autoencoders for molecules by introducing DyCC, a dynamic, chemistry-aware training framework with adaptive masking (GIBMS) and soft label reconstruction (SLG).

Abstract: Masked Graph Autoencoders (MGAEs) have gained significant attention recently. Their proxy tasks typically involve random corruption of input graphs followed by reconstruction. However, in the molecular domain, two main issues arise: the predetermined mask ratio and reconstruction objectives can lead to suboptimal performance or negative transfer due to overly simplified or complex tasks, and these tasks may deviate from chemical priors. To tackle these challenges, we propose Dynamic and Chemical Constraints (DyCC) for MGAEs. This includes a masking strategy called GIBMS, which preserves essential semantic information during graph masking while adaptively adjusting the mask ratio and content for each molecule. Additionally, we introduce a Soft Label Generator (SLG) that reconstructs masked tokens as learnable prototypes (soft labels) rather than hard labels. These components adhere to chemical constraints and allow dynamic variation of proxy tasks during training. We integrate the model-agnostic DyCC into various MGAEs and conduct comprehensive experiments, demonstrating significant performance improvements. Our code is available at \url{https://github.com/forever-ly/DyCC}.

Dynamic and Chemical Constraints to Enhance the Molecular Masked Graph Autoencoders
Jiahui Zhang!?, Wenjie Du Yang Wangli2/ University of Science and Technology of China, China
Suzhou Institute for Advanced Research, USTC, China kongping@mail uste. edu. en {duwenjie, angyan) edu.. cn
Abstract
Masked Graph Autoencoders (MGAEs) have gained significant attention recently. Their proxy tasks typically involve random corruption of input graphs followed by reconstruction. However, in the molecular domain, two main issues arise: the predetermined mask ratio and reconstruction objectives can lead to suboptimal performance negative transfer due overly simplified complex tasks, and these tasks may deviate from chemical priors. To tackle these challenges, we propose Dynamic and Chemical Constraints (DyCC) for MGAEs. This includes masking strategy called GIBMS, which preserves essential semantic information during graph masking while adaptively adjusting the mask ratio and content for each molecule. Additionally, we introduce a Soft Label Generator (SLG) that reconstructs masked tokens as learnable prototypes (soft labels) rather than hard labels. These components adhere to chemical constraints and allow dynamic variation of proxy tasks during training. We integrate the model-agnostic DyCC into various MGAEs and conduct comprehensive experiments, demonstrating significant performance improvements. Our code is available at atthtps: //ttps://githull ltt/fforeverrly/yCCC
Introduction
Molecular Representation Learning (MRL) plays a pivotal role in many related applications such as
drug discovery. material design, and reaction prediction 88] II 44 By representing molecules as graphs, where atoms are treated as nodes and bonds edges, Graph Neural Networks (GNNs) 43
20 14 have exhibited remarkable performance across wide range tasks. However, significant challenge is the scarcity of labeled data, which limits the effectiveness of supervised learning. Inspired by the remarkable progress in self-supervised pretraining in natural language processing [12). Masked Graph Autoencoders (MGAEs) [17
41 25 [6] have arisen promising approach to addressing these challenges. The pioneering work [17 on this topic introduced the pretraining of GNNs using a mask-then-ree recontrrccnt task called AttrMask. Specifically, they randomly mask some proportions of atoms and then pretrain the models to predict them. AttrMask has emerged as a fundamental subtask for pretraining. Despite their success, we have identified two limitations that still lack exploration.
pretraining task and many subsequent works [16] [5] adopt
The first limitation is that proxy tasks are predetermined and lack dynamic adaptability. The effectiveness of MGAFs largely governed by their proxy tasks, which are defined by the graph- masking strategy and the reconstruction objective. Prior approaches rely on fixed mask strategies and tokenizers, which result suboptimal performance. From the perspective of input corruption, the
corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Ammosk
MaleBet
GogaMAB
ROC-AtC (50)
ROC
Mask Ratio (19)
K-hop Tokonbetr
(8)
(b)
Figure 1: (a) The difficulty of proxy tasks for the AttrMask, MoleBert, and GraphMAE models was increased by raising the mask ratio. The average ROC-AUC scores were computed across the eight classification datasets in MoleculeNet [39]. (b) The average ROC-AUC scores of AttrMask with K K-hop subgraph token as the reconstruction target. The larger the value of K, the more challenging the proxy task becomes. (c) Reconstructing the masked atom as either bromine or chlorine can generate valid molecules.
masking strategy significantly impacts pretraining performance. As shown in Fig. Da). involving GraphMAE[16], AttrMask [17]. and MoleBert [41], we progressively increased the mask ratio to make the proxy task harder. Initially, the mask ratio increased, pretraining performance improved; however, beyond a certain threshold, performance deteriorated. From the perspective the reconstruction target, the choice of reconstruction tokens greatly affects pretraining performance. AttrMask treats atoms as tokens, but the simplicity of its reconstruction objective can cause suboptimal performance
negative transfer. Consequently, several studies have proposed more challenging tokenizers [54] 41 255. For example, SimSGT (55 uses K-hop subgraphs as reconstruction targets and employs a simple GNN-based tokenizer generate subgraph-level tokens. By adjusting the value of K, the complexity of the reconstruction task can be controlled larger the K, the more difficult the task. However, we observe that more challenging tokenizers do necessarily lead to better pretraining performance. As shown in Fig. [b), we used K K-hop subgraph tokens as the reconstruction target (with K corresponding to node-level tokens). While increasing K indicates a higher tokenizer complexity (i.c., a harder proxy task), the performance did not consistently improve. In summary, employing fixed masking and reconstruction strategies (i.e., predetermined tokenizer) necessitates cumbersome search for optimal hyperparameters and even then yields only suboptimal results. A more effective solution
introduce dynamic adaptability into proxy tasks.
The second limitation is that proxy tasks may not adhere to the constraints imposed by chemical priors, potentially leading nonsensical even harmful self-supervised learning signals. We outline three aspects that deviate from chemical constraints (CC), labeled as CCI, CC2, and CC3. CCI: Each molecule has specific mask ratio. Currently, most approaches adopt globally fixed mask ratio. However, different molecules exhibit varying degrees of structural redundancy. rendering uniform mask ratio suboptimal-too high for certain molccules. CC2: The importance of atoms within a molecule varies, as certain key atoms play pivotal roles in determining molecular functional properties, reactivity, and biological interactions. Consequently, prioritizing the masking of these important atoms can encourage the model to engage in more challenging contextual reasoning, thereby fostering deeper understanding of critical structural features. CC3: The reconstruction target should not be unique. Given the vast chemical compound space, many compounds exhibit significant structural similarity. As illustrated Fig. [c), the masked atom could be reconstructed as either Cl or Br. However, current methods often constrain the reconstruction by favoring specific atom types (or substructures), producing conflicting self-supervised signals.
To address these issues, we propose Dynamic and Chemical Constraints (DyCC) MGAEs, a model- agnostic approach dynamically adjusts the proxy task while adhering chemical constraints. Specifi- cally, we leverage the Graph Information Bottleneck (GIB) theory 52 51l to redefine graph masking as graph compression problem, introducing the GIB-based Mask Strategy (GIBMS). The core con- cept is to identify a compressed core substructure within the molecular graph that encapsulates its key properties, and to place greater emphasis on reconstructing these core substructures during the graph masking stage. This design explicitly addresses CCI and CC2 by encouraging the model to focus on essential structural information while learning robust molecular representations. However, traditional GIB relies supervisory signals, which are unavailable during pretraining when downstream task

---

labels are not accessible. Our contribution lies in extending GIB theory to the unsupervised setting. We adopt the common multi-view assumption [30] 42 enabling us to demonstrate that the mutual information between the molecular graph and self-supervised signals acts as a lower bound for the mutual information between the graph and downstream task labels. As result, we reformulate the problem as maximizing the mutual information between the molecular graph and self-supervised signals, which can be achieved through a contrastive learning paradigm.
Additionally, we introduce the Soft Label Generator (SLG) module, which transforms the recon- struction objective from a specific token (hard label) to a soft cluster assignment (soft label), thereby fulfilling [C3 4 2). We define potential clusters as prototypes represented by learnable vectors and subsequently map the hard labels to probability distributions of these prototypes using the Sinkhorn- Sinhhor-Knopp algorithm|[U [0]. Specifically, we randomly initialize set of prototypes (learnable vectors), and both the token labels and the reconstruction predictions are evaluated for similarity against these prototypes, yielding two probability assignment matrices (soft labels). Minimizing the discrepancy between these two matrices is equivalent to minimizing the difference between the labels and the reconstruction predictions. During the training process, as the prototypes are updated, the mapped soft labels dynamically change, thereby enabling the proxy task to be adjusted throughout the training phase (fulfilling dynamic). In extreme cases, when the assignment probability distributions converge to the one-hot distribution, the soft labels degenerate into hard labels, resulting in high inter-class distinctiveness and simplifying the reconstruction task. Conversely, the assignment prob- ability distributions approach a uniform distribution, inter-class separability diminishes, rendering the reconstruction task exceptionally challenging.
In summary, our core contributions are as follows: First, we identified the lack of dynamism and the failure to adhere to chemical constraints in the proxy tasks of MGAEs. To address these challenges, we extend the supervised GIB theory to the unsupervised setting and design the GIBMS module, generating the optimal mask ratio and mask content for each molecule. Additionally, we introduce the concept of soft assignment into the graph reconstruction stages, avoiding conflicting self-supervised signals and dynamically adjusting the tokenizer. Lastly, we integrate these two model-agnostic modules into multiple MGAEs. Extensive experimentation shows consistent improvements across the integrated models, validating the effectiivness and generality of our approach.
2 Related Work
Graph Contrastive Self-supervised Learning Contrastive self supe-vupe learning, follows the principle of mutual information maximization M.. which typically works to maximize the correspon- dence between the representations of an instance (e.g., node, subgraph, or graph) in its different augmentation views. GraphCI. [49] performs graph-level contrastive learning with combinations of four graph augmentations, namely node dropping, edge perturbation, subgraph cropping, and
feature masking. InfoGraph [32] conducts graph representation learning by maximizing the mutual information between graph-level representations and local substructures. GraphLOG (33] leverages
clustering to construct hierarchical prototypes of graph samples. They further contrast each local instance with its corresponding higher prototype for contrastive learning. JOAO
48 proposes a
framework to automatically search proper data augmentations for GCL. GraphMVP [24 uses a contrastive loss and a generative loss connect the 2-dimensional view and 3-dimensional view of the same molecule, in order to inject the 3-dimensional knowledge into the 2-dimensional graph encoder. RGCL 22 trains rationale generator to identify the causal subgraph in graph augmentation. Although the contrastive learning paradigm is very successful, it relies on data augmentation, which depends on domain knowledge.
Graph Generative self-supervised learning Generative self-supervised learning focuses on re- covering missing parts of input data. It can be further divided into two families: autoregressive and autoencoding models. Autoregressive models break down joint probability distributions into a product of conditionals. In supervised graph generation, earlier methods like GraphRNN [47 and GCPN [А]] have been proposed. More recently, GPT-GNN [88 represents an attempt to incorporate graph generation as a training objective. On the other hand, graph autoencoders are designed to reconstruct input data without enforcing a decoding order. Early work in this field includes GAE and VGAE 21), which use 2-layer GNN as encoders and dot-product decoding for link prediction.
AttrMask [17 adopts random masking strategy, where portion of the nodes are randomly masked,
3

---

(a)
GIBMS
Ow
(g: 9m
Pool
Enceder
MLP
Sample
(Y;9m)
Pool
(9) 918
Gm
(b)
SLG
HF
GIBMS
Encoder
Q
softmax
Duoth (8", ,a))
Tokemizess
Eaabodiing
HP
HH Q 8
softmaa
Tv
Figure 2: Overall architecture of DyCC. (a): The GIBMS module identifies the sampling probabil- ities of nodes in the molecular graph, based graph information bottleneck theory. (b): During reconstruction, both the hard labels of the tokens HD and the predictions are mapped into soft labels g and soft predictions s through the SLG module, forming probability distributions. The training objective Caot is to minimize the distance between these two distributions.
propose focus on feature reconstruction with both a masking strategy and scaled cosine error that benefit the robust training of GraphMAE.
and the goal is reconstruct them. GraphMAE [16
Mole-BERT 41 observes that mask atom prediction an overly easy pretraining task. Therefore, they employ a GNN tokenizer pretrained by VQ-VAI VAE to generate more complex reconstruction targets for masked atom modeling. SimSGT (25) utilizes simple GNN-based tokenizer, which removes the nonlinear update function in each GNN layer to derive subgraph level tokens. These works all adopt fixed proxy task setup and do not take into account the three chemical constraints we proposed.
Method
To meet the constraints of chemical priors and enable dynamic adaptability in proxy tasks, have designed the GIBMS and SLG,, as depicted in Fig. 2 Next, we will provide detailed explanations of these two modules.
3.1 GIBMS: Graph Information Bottleneck for Mask Strategy
aims to identify a condensed core substructure within a graph that maximizes its informativeness regarding the graph property while discarding redundant information. Inspired by this, we propose to dynamically generate core subgraph for each molecule (CC1) and place greater emphasis masking the important atoms within this core substructure (CC2). Subgraph recognition can be formulated by optimizing GIB (52) with a mutual information estimator. For graph G and its label information Y. the optimal IB IB-graph GIB which keeps minimal sufficient information:
Subgraph recognition 52
51 53
Gus arg min I (Y; GBB) 31 (g: Gis)
9m
where 3 serves as a Lagrangian multiplier to balance the two mutual information terms.
Obtaining node representations of molecular graph To begin, it is essential to convert the molecular graph G into a vector representation. GNNs have emerged as the predominant approach for handling molecular graphs due their effectiveness in capturing graph-structured data. Let & denote a GNN encoder, which used to generate the node-level representations of the graph. Formally, this process can be expressed as:
H (G))
(2)
where H represents the node representations of the graph.

---

Injecting Noise to Obtain the IB-Graph The discrete nature of graphs renders direct acquisition of the GIB impractical due to the exponential proliferation of candidates (2*) for given graph G with N nodes. Therefore, we propose a relaxation by assuming gilbert random graph [13], where node selection from the input graph g conditionally independent. This assumption enables us to factorize the probability of the GBB as:
P(Gmlg)
II P (V iEV
Here, V denotes the nodes of G, and P (V.l)) signifies the probability distribution for node V, A straightforward instantiation of P (Vilg) is the Bernoulli distribution V; ~ Bernoulli(p:) param- eterized by pi. We compute the probability p for each node based on its embedding H using Multi-Layer Perceptron network M, expressed as:
p MM(H)
(4)
Here, the Sigmoid function ensures the probabilities are normalized. The graph GIB then derived by performing Bernoulli sampling on all nodes:
9IB = (V: V;
1,,
(5)
Here GIB is sampled combination of nodes. Furthermore, we adopt noise injection following VGIB [51] to optimize the subgraph, as it has been proven to mitigate inefficiency and instability in GIB optimization caused by mutual information estimation. The key idea is to introduce more noise into less informative subgraphs while injecting less noise into more informative ones. Specifically, with the calculated probability we perturb the representation H by adding noise e:
H = AH + (1 A)
(6)
where À Bernoulli (p) and E ~ N (μH. oi)). Here, μии and o are the mean and variance of H. respectively, and Bernoulli represents the Bernoulli distribution. Thus, the information of G is compressed into Gi with the probability by replacing non-important nodes with noise. Moreover, to make the sampling process differentiable, we adopt gumbel-sigmoid 26 19 for discrete random variable X, i.e.,
A Sigmoid( log (1-
log
u
(1 (1-u) u)
where ~ Uniform (0, 1), and / is the temperature hyperparameter To ensure that the obtained GIB is meaningful, we need to solve Eq. (I). Next, we will discuss the optimization of the first prediction term -7 (Y; Gmn) and the second compression term I (9; 9m) separately.
Minimizing the Prediction Term The first term, -I (Y; 9IB), encourages GIB to be informative about the label Y. Since the goal of pre-training is to enhance the performance of downstream tasks, Y here refers to the labels of the downstream task dataset. To avoid confusion in notation moving forward, we will denote the labels of the downstream tasks as Ysup, Given the input Gm and the downstream task labels Yaup, our objective is to learn a vector representation of GIB denoted as Z (F)) which can effectively predict the labels Ysup, Here, Pool represents the pooling function.
sop 9m
arg max! (Zgm Y
(8)
Zom
However, since GIBMS is applied during the pretraining stage, obtaining downstream task labels Y is not feasible. Here we approach it from a self-supervised learning perspective. Considering the input GIB and self-supervised signals S (c.g., augmentations of GIB) two different views of the data, we aim to derive sufficient self-supervised representations Z that can maximize the preservation of shared information between the views.
arg max/ (Ze, S)
Zum
where Zen. is the representation of the graph, obtained by pooling the node representations Å. i.e. ZGn Pool (H), where Pool is the pooling function. By adopting the common multi-view
assumption 30
have (Appendix B.I::
1(9 Y (P) azap, Y > 1 (Z Y > [(GB: Y Cinfo:
Eino > 0 (10)
5

---

In this paper, we assume that with appropriate self-supervised signals S, Citto is negligible. Conse- quently, the above formula suggests that the self-supervised learned representations Gi can capture almost as much task-relevant information about Y as the supervised representations sup In this case, minimizing - (GIB; Y is approximately equivalent to maximizing I ( S). Since recently proposed contrastive learning methods [35 [5] [0], which aim to pull positive samples closer and push negative samples apart in the representation space, have been theoretically proven to maximize the mutual information between positive pairs, we leverage this approach. Given GIB, we can repeatedly sample from Eq. (5) to obtain its positive sample IB, i as the self-supervised signal S.
Then, maximize I Sup S) using contrastive learning loss, such as InfoNCE [37]
exp (sim Zes... Zoin ) )1T)
I TYYU;; GIB) Pped (Y GIB)
log
(11)
K
K jj exp (sim (Zom. Z /T)
Here, the representations z... and Zojs. of two IB-graphs GBB and 9iB; are considered as positive samples, while representations Zem.s of GIB.j from other graphs in the same batch are treated as negative samples. K and r indicate the number of paired graphs in a batch and the temperature hyperparameter, respectively.
Optimizing the Compression Term The second term minimizes the mutual information of & and Gus so that GιB only receives limited information from the input graph G. We can derive its variational upper bound (see Appendix E.2
1 (G: S Eg (-2 log A
log
1 2N
A
1 y
b²)
Ccomp (G, 91B)
(12)
2N
22)² and B = Σ> (H,
where N represents the number of nodes in g, A Ey (1-x))
6h
The Final Training Objective Based on the aforementioned analysis, the training objective of GIBMS as follows:
Ciaal = Cpred (Y" GiB) + Epedd (9IB, Y + 3 (9, G1s)
(13)
Dynamic Masking Strategy The trained GIBMS is able to generate importance score p; for each atom node v; in the molecular graph G according to Eq. where p; represents the probability of retaining the node under the information bottleneck framework. Subsequently, we adopt the Gumbel-Sigmoid approximation to sample a dynamic masking factor A; for each node, as shown in Eq. Here, X; is a continuous variable the range [0. 1]. which is further converted into a binary masking decision by applying a threshold where A > r indicates that the corresponding atom will be masked. This strategy allows the masking ratio be adaptively adjusted at the molecule level according to the specific structural characteristics of each graph, without the need to predefine fixed global masking ratio (CC1). Moreover, the mechanism tends to preferentially mask nodes with higher importance scores (CC2), thereby encouraging the pretraining model to focus capturing the key structures within molecular graphs and enhancing the discriminability and generalization of the learned representations.
3.2 Soft Label Generator
The output of the tokenizer can be either discrete values (c.g., atomic numbers) or continuous vectors (e.g., representations of subgraphs). For the sake of convenience in subsequent discussions, we map the categorical variable y of tokens through embedding into a vector, unifying the reconstruction label as d-dimensional vector H Rd
Regardless of the tokenizer used, existing methods reconstruct masked atoms into specific types. In the preceding discussion, we emphasized the importance of soft labels. The question now arises: how do we obtain soft label ? Since we cannot access all molecules in the world, it is not feasible to directly acquire the probability of reconstructing masked atom into different tokens. To address

---

this limitation, we introduce soft label assignment 7 4 2]. We assume the existence of n learnable latent prototypes Q R where each masked atom has probability of being reconstructed into all these prototypes. Firstly, we map hard labels H to soft labels 8h:
H
8 = softmax
(14)
Ty
where Ty E (0, 1) is a temperature. Similarly, for the node representations H2 outputted by the encoder Q', we generate their predictions s by measuring the cosine similarity to the same prototype matrix Q with temperature Tp E (0, 1).
H Q Tp
8 softmax
(15)
Note, we always choose Ty < Tp to encourage sharper target predictions, which implicitly guides the model produce confident low entropy anchor predictions. We penalize when the prediction 8 is different from the soft label 8². We enforce this criterion using a standard cross-entropy loss H (s",s²) s²). We also incorporate the mean entropy maximization (ME-MAX) regularizer B S to
encourage the model to utilize the full set of prototypes. Denote the average prediction of batch of M samples as Sp:
(16)
The ME-MAX regularizer simply seeks maximize the entropy of s denoted H HsP), or equivalently, minimize the negative entropy of sP. Thus, the overall objective is:
C S5, )0)
Σ (s), - -aH()))
(17)
where 0 controls the weight of the ME-MAX regularization.
4 Experiments
4.1 Experiments setup
Pretraining setup For the pretraining stage, utilized 2 million molecules sourced from the ZINC15 database [3I], following the precedent of prior studies [17]. The GIBMS module was trained using the loss function defined in Eq. (33), where the temperature factor 0.1 for the InfoNCE loss, and 3 = 0.01 controls the trade-off between prediction and compression. After training the GIBMS module, we utilized it to generate corresponding mask probabilities for each atom of the 2 million molecules in ZINC15 and sampled masked atoms based on these probabilities. In the reconstruction phase, we mapped the hard labels outputted by the tokenizer soft labels using the SLG module. By default, we set temptures тр = 0.25, Ty 0.1, and the number of prototypes n 128. After pretraining, we employed the widely-adopted 8 binary classification datasets within MoleculeNet (39 to evaluate performance on downstream molecular property prediction tasks (see Appendix B). These downstream datasets are divided into train/valid/test sets using scaffold split by 8:1:1 to facilitate an out-of cot-of-distribution evaluation setting. We report the mean performances (ROC-AUC) and standard deviations on the downstream datasets across ten random seeds.
Baselines We integrated our method into three MGAFs: Attraask [17], MoleBert 41, and SimSGT 225. All our settings remain consistent with the configurations of these models. It is noteworthy that, for a fairer evaluation of DyCC on the masked atom modeling proxy task, we have excluded irrelevant enhancements of MoleBert and SimSGT. Specifically, we removed the GraphTrans variant from SimSGT to ensure consistency in using the GIN architecture. Additionally, we eliminated the triplet masked contrastive learning (TMCL) from MoleBert as it is unrelated to MGAEs (see Appendix C.5). Furthermore, we selected several other self-superviser graph pretraining models for further comparison, including InfoGraph [32] GPT-GNN (18), EdgePred [7]
ContextPred [17] GraphL.OG (33], G-Contextual [29] G-Motif 29 AD-GCL
34 JOAO [8]]
SimGRACE [40] GraphCL. 49 GraphMAE 16 GraphMVP 24 and MGSSL.
54
The results
collect from MoleBert
41
7

---

Table 1: Transfer learning ROC-AUC (%) scores eight MoleculeNet datasets. The suffix "DyCC"
implies the introduction of both the GIBMS and SLG modules.
Dataset
Tox21
ToxCast
Sider
ClinTox
MUV
HIV
BBBP
Bace
Avg.(t)
No Pretrain 74.6204
61.7
58.2ata
58,4
70.7
75.5ma
65.7a33
72.4
67.0
InfoGraph
73.3
61.8at4
58.7
75.4
74,4
74.2a09
68.7
74.3
70.1
GPT-GNN
74.9.0.
62.5a
58. 1an3
58.3252
75.9
65.2221
64.5at4
77.9
68.5
EdgePred
76.0ss
64,
60).4.7
64.1
75.1
7
67.3aa4
77.3.15
70.1
ContextPred
73. 6
62.6
59.7+1
74.0.sa
72.5+1s
7
70.6
78.8+12
70.1
GraphLoG
75.0cm
63, 63,4
59. 6.19
75.7a24
75.5a16
76.
68.7at6
78.6
71.6
G-C G-Contextual
75.0
62.8.0.7
58.7.10
60.6i52
72.1
76.3
69.9I2
79.3u
69.3
G-Motif
73.6
62.3
61.0
77.7,27
73.0
73.8
66.94s1
73.0
70.2
AD-GCI.
74.9.04
63.4
61.5449
77.2.07
76.3+14
76.7412
70.7-03
76.6415
72.2
JOAO
74.8206
62.8
60.421s
66.623.1
76.6217
76,9
66,4atm
73.2st6
69.7
SinGRACE
74.4
62.6
60.22a9
75.5220
75.4
75.0aas
71.2at
74.922n
71.2
GraphCI.
75.1.00
63.0ma
59.8
77.5
76.4
75.
67.8.24
74.6.21
71.2
GraphMAE
75.2
63.6
60.5
76.5230
76.4220
76,8s0s
71.2sto
78.2s15
72.3
GVPphMVP
74.9208
63.1
60.2at.1
79.1
77.7zas
76.0am.
70.8
79.3a15
72.6
MGSSL
75.2
63,3w0.
61.621.
77.1
77.6204
75.8
68.8
78.8.0.
72.3
AttrMask
75.1
63.3
60.5mno
73.5
75.8
75.3a1
65,2a14
77.8
70.8
Attraak--yyCC
76.6.8.
64.6wa4
61.3am
79,8.s
76.7aa
77.611.
70.5
82.1
73.7
Mole-BERT
76.2a0s
63.9213
61.4al 6119
75.1.30
77.4221
77.5ato
66, 8a1.5
78.9
72.2
Mole-BERT-DyCC
76.3.es
64.4ws
61.4wu
78.9.24
78.6.19
77.7
70,8
82.2aa
73.8
SimSGT
75.
63.5
61.0am4
79.1
76.0aas
76.3an.
70.9am.
82.5m0
73.0
SimSGT-DyCC
76.0.a.
64.6
61.6.tt
80.5.2
77.7a09
77.3aaM
71.4
83.4
74.1
Paplextt
Vn/w Mhyt Ma
Pencentoge (%
9.
Porcentage (%6)
0:3
3:55-
0:3
3.5-
1960 Deraatins
200 3990
1890
0 01
100
156
Prototyps
(b)
(c)
Figure 3: (a): The variation of perplexity of prototypes during the training process. (b): The atoms ratios of various chemical elements in the ZINC datasets. (c): The distribution of 128 prototypes in the ZINC datasets after training.
4.2 Model performance
As depicted in T the incorporation of DyCC into AttMask, MoleBert, and SimSGT significantly boosts the performance of pretraining. Particularly noteworthy its integration into SimSGT, where surpasses the "No pretrain" model by 10.4%, achieving new state-of-the-ane result. Moreover, DyCC demonstrates effective mitigation of the impact of tokenizers on pretraining, Previously, substantial performance variations were observed among the original AttrMask, MoleBert, and SimSGT models due to differences in tokenizers. However, following the integration of DyCC, these performance gaps narrow considerably, indicating reduced reliance on tokenizers. This can be attributed DyCC's adaptive adjustment of the reconstruction targets, consequently diminishing dependency on tokenizers. In Appendix C.4 we verified the efficacy of DyCC across a broader spectrum of downstream tasks and datasets, including four molecular property prediction regression tasks and two Prug-Taarer Affinity (DTA) regression tasks 27
4.3 Detailed Analysis of GIBMS and SLG
Here, we primarily analyze the role of GIBMS and SLG, while experiments other hyperparameters can be found in Appendix C.6
The role of GIBMS module. further evaluate GIBMS, we employed the MUTAG dataset, which includes 4,337 molecular graphs, each classified into one of two categories based on its mutagenic effect. As noted in GNNExplainer [45), carbon rings with chemical groups N H2 or NO, are known to be mutagenic. We labeled the top 30% most important atoms identified by GIBMS, and if these

---

Note
GIIMS
SL4
74
73
721
AttMaa
MediBat
SixSOT
Figure 4: (Left) Core substructures (highlighted) extracted by GIBMS for four molecules. (Right)
Component ablation of GIBMS and SI.G.
atoms included N H2 or NO2, it was deemed success. The success rate of GIBMS was 74%, demonstrating the effectiveness of the GIBMS module. In addition, we conducted a qualitative analysis of substructures based on our prior chemical knowledge. As depicted in Fig. f we randomly selected four molecules and utilized GIBMS to extract core substructures. The results indicate that the model tends not to focus on aromatic rings but rather tends to discover the substructures around them. This finding aligns with chemical knowledge, aromatic rings, which contribute to the stability of molecules, are not directly related to chemical propertics, whereas substructures in the side chains more likely to contain chemical information.
The role of SLG module. SLG is proposed for dynamically adjusting task difficulty. To investigate its effectiveness, we utilize perplexity as an evaluation metric to assess the probability of different prototypes being utilized. A higher perplexity suggests implying increased difficulty the reconstruction task. As depicted in Fig. alg perplexity initially increases during training, then gradually decreases and converges stable value. This indicates that SLG enables our model to dynamically adjust the difficulty of the reconstruction task. Moreover, SLG effectively mitigates issues such as small vocabulary size and token imbalance. For instance, in the widely used ZINC15 dataset, which comprises 12 types of atoms, with 95% of the atoms distributed among the top three atom types (Fig. B()), SLG allows flexible specification of the number of prototypes (determined by n.), and yields more uniform distribution of tokens, as illustrated in Fig. c)).
uniform utilization of prototypes,
Are both GIBMS and SLG necessary. To investigate this, we separately added only one module, either GIBMS or SLG, into AttrMask, MoleBert, and SimSGT. As depicted in Fig. we observed that while introducing either module alone improves the effectiveness of pretraining across all MGAEs. combining both strategies leads to better results.
Conclusion
We identified two significant issues when applying existing MGAEs methods to the molecular domain. On one hand, the proxy tasks are predetermined and lack the capability for dynamic adjustment during training. On the other hand, there are designs that do not align with chemical priors. To address these challenges, we propose the DyCC framework, which consists of two modules: GIBMS and SL.G. The GIBMS module employs graph information bottleneck theory to identify nodes that preserve semantics during masking, enabling adaptive masking. The SL.G module utilizes set of learnable prototypes to map the hard labels of tokens soft labels, dynamically updating these soft labels throughout the training process. This allows the reconstruction objectives to adaptively adjust as well. We integrated DyCC into various existing MGAEs, significantly enhancing pre-training performance while reducing reliance on tokenizers.
Acknowledgement
This paper partially supported by the National Natural Science Foundation China (No. 12227901). The Al-driven experiments, simulations and model training were performed on the robotic Al-Scientist.

---

platform of Chinese Academy of Sciences., Anhui Science Foundation for Distinguished Young Scholars (No.190808572) Natural Science Foundation of China (No. (No.62502491..
References
[1] Alexander A. Alemi, lan Fischer, Joshua V. Dillon, and Kevin Murphy. Deep variational information bottleneck. CoRR, abs/1612.00410, 2016.
[2] Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Florian Bordes, Pascal Vincent, Armand Joulin, Mike Rabbat, and Nicolas Ballas. Masked siamese networks for label-efficient learning. In Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision ECCV 2022 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXI, volume 13691 of Lecture Notes in Computer Science, pages 456-473, Springer, 2022.
[3] Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Florian Bordes, Pascal Vincent, Armand Joulin, Mike Rabbat, and Nicolas Ballas. Masked siamese networks for label-efficient learning. In Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision ECCV 2022 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXI, volume 13691 of Lecture Notes in Computer Science, pages 456-473. Springer, 2022.
[4] Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Armand Joulin, Nicolas Bal- las, and Michael G. Rabbat. semi-supervised learning of visual features by non-parametrically predicting view assignments with support samples. In 2021 IEEE/CVF International Confer- ence on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 8423-8432. IEEE, 2021.
[5] Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Armand Joulin, Nicolas Bal- las, and Michael G. Rabbat. Semi- Semi-supervised learning of visual features by non-parametrically predicting view assignments with support samples. In 2021 IEEE/CVF International Confer- ence on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 1021, pages 8423 8432. IEEE, 2021.
[6] Simon Axelrod and Rafael Komez-Bombarelly GEOM: energy-annotated molecular conforma- tions for property prediction and molecular generation. CoRR, abs/2006.0555 05531, 2020.
[7] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florin, Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, viriual, 2020.
[8] K. Choudhary, B. DeCost, Chen, and al. Recent advances and applications of deep learning methods in materials science. npj Comput Mater, 8:59, 2022.
[9] Thomas M Cover and Joy A Thomas. Elements of information theory. John Wiley Sons, 2012.
[10] Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In Christopher J. C. Burges, Léon Bottou, Zoubin Ghahramani, and Kilian Q. Weinberger, editors, Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pages 2292-2300, 2013.
[11] Laurianne David, Amol Thakkar, Rocío Mercado, and Ola Engkvist. Molecular representations ai-driven drug discovery: a review and practical guide. J. Cheminformatics, 12(1):56, 2020.
[12] Jacob Devlin, Ming-Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics, 2019.
10

---

[13] Edgar N Gilbert. Random graphs. The Annals of Mathematical Statistics, 30(4): 144-114,, 1959.
[14] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio. Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Anmual Conference Neural Information Processing Systems 2017,
December 4-9, 2017, Long Beach, CA, USA. pages 1024-1034, 2017.
[15] R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In 7th International Conference on Learning Representations ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
[16] Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang. Chunjie Wang, and Jie Tang, Graphmae: Self-superviser masked graph autoencoders. In Aidong Zhang and Huzefa Rangwala, editors, KDD 22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14- 18, 2022, pages 604. ACM, 2022.
[17] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay S. Pande, and Jure Leskovee. Strategies for pre-training graph neural networks. In 8th International Con- ference Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
[18] Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, and Yizhou Sun. GPT-GNN: gen- erative pre-training of graph neural networks. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash, editors, KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 1857-1867.
ACM, 2020.
[19] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameteriization with gumbel-softmax. In Sth International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. Open 2017.
[20] Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations (ICLR), 2017.
[21] Thomas N. Kipf and Max Welling. Variational graph auto-encoders. CORR, abs/1611,00300 2016.
[22] Sihang Li, Xiang Wang, An Zhang, Yingxin Wu, Xiangnan He, and Tat-Seng Chua. Let invariant rationale discovery inspire graph contrastive learning. In Kamalika Chaudhuri. Stefanic Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning. ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 13052-13065. PMLR, 2022.
[23] Shengchao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. Pre- training molecular graph representation with 3d geometry. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
[24] Shengchao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. Pretraining molecular graph representation with 3d geometry. In International Conference on Learning Representations, 2022.
[25] Zhiyuan Liu, Yaorui Shi, An Zhang. Enzhi Zhang, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. Rethinking tokenizer and decoder in masked graph modeling for molecules. CoRR. 10/23300.14753. 2023.
[26] Chris Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: contin- uous relaxation of discrete random variables. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings OpenReview.net, 2017.

---

[27] Thin Nguyen, Hang Le, Thomas P. Quinn, Tri Nguyen. Thuc Duy Le, and Svetha Venkatesh. Graphdta: predicting drug-target binding affinity with graph neural networks. Bioinform. 17(8):144-11147, 2021.
[28] Tapio Pahikkala, Antti Airola, Sami Pietilä, Sushil Kumar Shakyawar, Agnieszka Szwajda, Jing Tang, and Tero Aittokallio. Toward more realistic drug-target interaction predictions. Briefings Bioinform., 16(2):325-333 2015.
[29] Yu Rong, Yatao Bian. Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. Self-supervised graph transformer on large-scale molecular data. Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS 20, Red Hook, NY, USA, 2020. Curran Associates Inc.
[30] Karthik Sridharan and Sham M. Kakade. An information theoretic framework for multi-view learning. In Rocco A. Servedio and Tong Zhang. editors, 2 Annual Conference on Learning Theory-- COLT 2008, Helsinki, Finland, July 9-12, 2008, pages 403-414. Omnipress, 2008.
[31] Teague Sterling and John J. Irwin. ZINC 15 ligand discovery for everyone. J. Chem. Inf. Model., 55(11 (((11)::2 ):2324 2337, 2015.
[32] Fan- Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia. April 26-30, 2020. OpenReview.net, 2020.
[33] Susheel Suresh, Pan Li, Cong Hao, and Jennifer Neville. Adversarial graph augmentation to improve graph contrastive learning. In Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference Neural Information Processing Systems 2021, NeurlPS 2021, , December 6-14, 2021, virtual, pages 15920-15933, 2021.
[34] Susheel Suresh, Pan Li, Cong Hao, and Jennifer Neville. Adversarial graph augmentation to improve graph contrastive learning. Conference on Neural Information Processing Systems (NeurIPS), pages 15920-15933, 2021.
[35] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. CoRR, abs/1906.05849, 2019.
[36] Yao-Hung Hubert Tsai, Yue Wu, Ruslan Salakhutdinov, and Louis-Philippe Morency. Self- supervised learning from multi-view perspective. In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[37] Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. CoRR, abs/1801 807.03748. 2018.
[38] Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. Deep graph infomax. CoRR, 0///1809.10344 2018
[39] Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S. Pappu, Karl Leswing, and Vijay S. Pande. Moleculenet: A benchmark for molecular machine 2017.
learning. CoRR,
[40] Jun Xia, Lirong Wu, Jintao Chen, Bozhen Hu, and Stan Z. Li. Simgrace: A simple framework for graph contrastive learning without data augmentation. In Proceedings The Web Conference 2022. Association for Computing Machinery, 2022.
[41] Jun Xia, Chengshuai Zhao, Bozhen Hu, Zhangyang Gao, Cheng Tan, Yue Liu, Siyuan Li, and Stan Z.. Li. Mole-BERT: Rethinking pre-training graph neural networks for molecules. The Eleventh International Conference Learning Representations, 2023.
[42] Chang Xu, Dacheng Tao, and Chao Xu. A survey on multi-view learning. CoRR, 2013.
12

---

[43] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
[44] Chaochao Yan, Qianggang Ding, Peilin Zhao, Shuangjia Zheng, Jinyu Yang. Yang Yu, and Junzhou Huang. Retroxpert: Decompose retrosynthesis prediction like A chemist. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
[45] Zhitao Ying. Dylan Bourgeois, Jiaxuan You, Marinka Zitmik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 2019.
[46] Jiaxuan You, Bowen Liu, Zhitao Ying. Vijay S. Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Desa-Bianchi, and Roman Garnett, edi- tors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 6412 6422, 2018
[47] Jiaxuan You, Rex Ying, Xiang Ren, William L. Hamilton, and Jure Leskovec. Graphrnn: Generating realistic graphs with deep auto-regressive models. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML. 2018, Stockholmsmüssan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 5694-5703. PMLR, 2018.
[48] Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. Graph contrastive learning automated. In Marina Meila and Tong Zhang, editors. Proceedings of the 38th International Conference on Machine Learning, ICML. 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 12121-12132. PMLR, 2021.
[49] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang. and Yang Shen. Graph contrastive learning with augmentations. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florins Balcan, and Esuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
[50] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang. and Yang Shen. Graph contrastive learning with augmentations. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Anmual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6- 2020, virtual, 2020.
[51] Junchi Yu, Jie Cao, and Ran He. Improving subgraph recognition with variational graph information bottleneck. CORR, abs/2112.09899, 2021.
[52] Junchi Yu, Tingyang Xu, Yu Rong, Yatao Bian, Junzhou Huang, and Ran He. Graph infor- mation bottleneck for subgraph recognition. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[53] Junchi Yu, Tingyang Xu, Yu Rong, Yatao Bian, Junzhou Huang, and Ran He. Recognizing predictive substructures with subgraph information bottleneck. IEEE Trans. Pattern Anal. Mach. 2024.
Intell., 4603)
[54] Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Chee-Kong Lee. Motif-based graph self-supervised learning for molecular property prediction. In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 202 I, NeurIPS 2021, December 6- 14, 2021, virtual, pages 15870-15882, 2021.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We have claimed the paper's contributions and scope in abstract and introduc-
tion
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Limitations are discussed in Appendix A
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
The proof could be found in Appendix E
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The code and data are provided.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification:
The code and data are open.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cppppi.ccppulli//uuuiu wwwiii/gguie///eeeeeeoecc CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have provided the data splits, hyperparameters, etc.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We have provided the statistical significance of the experiments result.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
16

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide provide sufficient information on the computer resources in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics btttps://neuripppt httppp//wwppppppcecccccctet.....
Answer: [Yes]
Justification:
The research is within the Code of Ethics
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
All of them are properly respected.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
20

---

A
Limitations
When training the GIBMS module, we adopt the common multi-view assumption 30 42 assuming that self-supervised proxy task is sufficiently effective to yield small Gifo- However, this assumption may not always hold. Despite this, we empirically observe that leveraging this assumption indeed benefits the training of GIBMS. Therefore, we proceed with this assumption.
B Details of molecular datasets
We provide detailed information of the datasets for molecular property prediction (classification and regression) and drug target affinity prediction in Table
Table Summary for the molecule datasets for downstream tasks.
Dataset
Task
# Tasks
# Molecules
# Proteins
# Molecule-Prottein
BBBP
Classification
2,039
Tox21
Classification
12
7,831
ToxCast
Classification
617
8,576
Sider
Classification
27
1,427
ClinTox
Classification
1,478
MUV
Classification
17
93,087
HIV
Classification
41,127
Bace
Classification
1,513
Delaney
Regression
1,128
Lipo
Regression
4,200
Malaria
Regression
9,999
CEP
Regression
29,978
Davis
Regression
68
379
30056
KIBA
Regression
2,068
229
118,254
Molecule representations. For simplicity, we use minimal set of node and bond features that unambiguously describe the two-dimensional structure of molecules following previous works. We use RDKit obtain these features, as show in Table and Table
Table 3: Atom features.
features
size
description
atom type
100
type of atom (e.g., C, N, O), by atomic number integer electronic charge assigned to atom number of bonds the is involved in number of bonded hydrogen atoms number of bonded hydrogen atoms
formal charge
5
number of bonds
chirality number H atomic mass
5
mass of the atom, divided by 100 whether this atom is part of aromatic system sp, sp2. sp3, sp3d, or sp3d2
aromaticity
hy hybridization
5
Table 4: Bond features.
features
size
description
bond type stereo in ring conjugated
4
single, double, triple, aromatic none, any, E/Z or cis/trans whether the bond is part of ring whether the bond conjugated
6
1
21

---

Dataset Splitting. We apply the scaffold splitting for all tasks on all datasets. It splits the molecules with distinct two-dimensional structural frameworks into different subsets. It is more challenging but practical setting since the test molecular can be structurally different from training set. Here we apply the scaffold splitting to construct the train/validation/test sets.
C Experimental Details
C.1 Computational resources
Our experiments are conducted using an NVIDIA DGX A100 server. Each experiment can be exccuted on a single GPU while staying within the limit of GB GPU memory consumption.
C.2 Implementation and pretraining Details
We used the official source code provided by AttrMask, MoleBert, and SimSGT, retaining the exact same settings. Building upon this foundation, we introduced the GIBMS and SLG modules. The three additional hyperparameters for GIBMS were f 1, 3 0.01, and 0.1, respectively. The four additional hyperparameters for SLG were set to Ty = 0.1, 0.25, a = and n = 128.
C.3 Baselines
We now describe the details our reported baseline methods:
InfoGraph 32
conducts graph representation learning by maximizing the mutual informa- tion between graph-level representations and local substructures of various scales.
GPT-GNN introduces a self-supervised attributed graph generation task to pre-train a GNN that it can capture the structural and semantic properties of the graph. They factorize the likelihood of the graph generation into two components: 1) Attribute Generation and 2) Edge Generation.
ContextPred
17 uses the embeddings of subgraphs predict their their
context graph structures.
leverages clustering construct hierarchical prototypes of graph sam- ples. They further contrast each local instance with its corresponding higher prototype for contrastive learning.
GraphLOG
33
Infomax 388 learns node representations by maximizing the mutual information between the local summaries of node patches and the patches' graph-level global summaries.
G-Contextual 29 views the prediction problem as a multi-class prediction task, where each class corresponds one contextual property.
G-Motif (29) formulates the prediction task as multi-label classification problem, where each motif corresponds to one label.
applies adversarial learning for adaptive graph augmentation to remove the redundant information in graph samples.
AD-GCL 34
JOAO 48 proposes a framework to automatically search proper data augmentations for
GCL.
SimGRACE 40) take original graph as input and GNN model with its perturbed version two encoders obtain two correlated views for contrast. SimGRACE inspired by the observation that graph data can preserve their semantics well during encoder perturba- tions while not requiring manual trial-and-on errors, cumbersome search or expensive domain knowledge for augmentations selection.
GraphCL [49 performs graph-level contrastive learning with combinations of four graph augmentations, namely node dropping, edge perturbation, subgraph cropping, and feature masking.
shows that linear classifier is insufficient for decoding node types. It applies GNN for decoding and proposes remask to decouple the functions the encoder and decoder in the autoencoder.
GraphMAE 16
22

---

Table 5: Transfer learning performance for molecular property prediction (regression) and drug target
affinity (regression). Bold indicates the best performance.
Molecular Property Prediction (RMSH 4
Drug-Target Affinity (MSF ()
ESOL
Lipo
Malaria
CEP
Avg.
Davis
KIBA
Avg.
No Pre-train
1.178.004
0.744.0.00
1.
1.254
1.076
0.286.nms
0.206i000
0.246
ContextPred
1.196-amm
0.702....om
1.101-ams
1.243
1.061
0.279.amm
0. vo
0.238
AttrMask
1.1 1.112-001s
0.730
1.1 19-0mm
1.256x0.00
1.054
0.29 11911am -0000
0.2033
0.248
JOAO
1.120
0.708
1. 1.54-ommo
4.293
1.066
0.28 1000
0. 19
0.239
GraphMVP
1.064
0.691 20.013
1. 1066mmm
1.228
1.022
0.2 0.274
0. 75
0.225
Mole-BER BERT
1.192aums
0.706
1.1 7enuss
1.078
1.024
0.277
0.2 10
0.243
SimSGT-G
1.039amm2
0.670
1 1.090gam3
1.060g001
0.965
0.263am00s
0.14 442000
0,204
SimSG1-G-DyCC
0.988
0.672a0006
1.082aam2
1.035
0.944
0.256
0.1 0.140at.01
0.198
Meatter
SendUT
75-
ROC-AUC (1)
ROC-AUC AUC-AUC (%6
ROC AUC-AUC (%)
72
0
n/a (8)
01
61
125
256
11
0.55 Yaul
0%
835
Figure 5: The impact of three hyperparameters 3, n, and Tp.
GraphMVP (24 uses a contrastive loss and generative loss to connect the 2-dimensional view and 3-dimensional view of the same molecule, in order to inject the 3 3-dimensional knowledge into the 2-dimensional graph encoder.
MGSSI, introdue a novel self-supervised motif generation framework for GNNs. First, for motif extraction from molecular graphs, they design molecule fragmentation method that leverages a retrosynthesis-basee based algorithm BRICS and additional rules for control- ling the size of motif vocabulary. Second, they design a general motif-based generative pretraining framework in which GNNs are asked to make topological and label predictions.
RGCL 22 trains a rationale generator to identify the causal subgraph in graph augmentation. Each graph's 's causal subgraph and its complement are leveraged in contrastive learning.
Mole-BERT 41 combines a contrastive learning objective and masked atom modeling objective for MRL. Specifically, they observe that mask atom prediction is an overly easy pretraining task. Therefore, they employ GNN tokenizer pretrained by rQ-VAEto generate more complex reconstruction targets for masked atom modeling.
C.4 Broader Range of Downstream Tasks
We verified the efficacy of DyCC across a broader spectrum of downstream tasks and datasets, including four molecular property prediction regression tasks and two Drug-Targe Target Affinity (DTA) regression tasks (27 28
DTA aims predict the affinity scores between molecular drugs and target proteins. Following prior work 23 we pretrain SimSGT-DyCC on 50 thousand molecule samples from the GEOM dataset [6] and report the mean performances and standard deviations across three random seeds. We report the RMSE for the molecular property prediction datasets with scaffold splitting and report the MSE for the DDT datasets with random splitting. The results are summarized in Ta)le It is evident that SimSGT-DYCCC surpasses the original version of SimSGT, achieving significant improvement over other baseline models. This suggests that DyCC can effectively enhance performance across a wider spectrum of downstream tasks.
C.5 Additional experimental results
Regression tasks We verified the efficacy of DyCC across a broader spectrum of downstream tasks and datasets, including four molecular property prediction regression tasks and two Drug-Targel
23

---

Table 6: Transfer learning ROC-AUC (%) scores eight MoleculeNet datasets. The suffix "DyCC" implies the introduction of both the GIBMS and SLG modules.
Datuset
Tox21
ToxCast
Sider
ClinTox
MUV
HIV
BBBP
Bace
Avg.
No Pretrain
74.6.0.
61,7a05
58.2 58.4164 70.7 ai 75.5.08
65.7a1a
72.413x
67.0
Mole-BERT
76.2
63.9203
61,4 75.1a50
77.4 77.5a10
66.8a1.
78.9209
72.2
Mole-BERT DyCC
76.3-0,5
64.4405
61,4-09 78.9.24
78.6-19 77.7.08
70.8.0s
82.2-a9
73.8
Mole-BERT DyCC TMCL
76.6.0.4
64.8a05
61.8a0.s 78.6x22
78.8 21.
77.9
71.3
82.8aa9
74.1
SimSGT
75. aans
63.5s0.4
61 .004
19.1
76.0au
76.3
82,5a09
70.9
73.0
SimSGT DyCC
76.0.0.5
64,6x0.4
80.5x22
77.7xn9
77.3
71.4077
83.4a1n
74.1
SimSGT DyCC GraphTrans
76.6 20.6 66.320s
62.0at.2 83.6 22.1 80.3g22
77.8 84.9 218 72.2a08
75.5
Table 7: Mean Average Error (MAE) performanceon the QM datasets.
QM7
QM8
QM9
#Tasks
12
GraphCL
80.4
0.0200.00004
5.7620.37
GraphMAE
78.422.
0.01 90
5.84s016
Mole-BERT
79.822.6
0.019020000
5,752016
Mole-BERR DyCC
78.7
0,0188
5.70
SimSGT
78.8
0.0189 0000
5.73
SimSGT DyCC
77.6218
0.0180=0.000
5.6020.11
DTA aims predict the affinity scores between molecular drugs and target proteins. Following prior work [23] we pretrain SimSGT-DyCC on 50 thousand molecule samples from the GEOM dataset 6 and report the mean performances and standard deviations across three random seeds. We report the RMSE for the molecular property prediction datasets with scaffold splitting and report the MSE for the DTA datasets with random splitting. The results are summarized in Table 5 It is evident that SimSGT-DyCC surpasses the original version of SimSGT, achieving significant improvement over other baseline models. This suggests that DyCC can effectively enhance performance across a wider spectrum of downstream tasks.
Affinity (DTA) regression tasks 27 28
Additional Modules of SimSGT and MoleBert the main paper, for fair comparison, we excluded the GraphTrans variant of SimSGT and the TMCL proxy task of MoleBert. We provide the complete version in Table and the results indicate that restoring these strategies further improves the model's performance. This suggests that DyCC can work in conjunction with other enhancements to MGAFs.
Quantum chemistry property prediction. We report performances of predicting the quantum chemistry properties of molecules. We divide the downstream datasets by scaffold split. Specifically, we attach a two-layer MLP after the pretrained molecule encoders and fine-tune the models for property prediction. We report average performances and standard deviations across 10 random seeds. The performances are reported in Table 7 performance by DyCC.
We observe a consistent enhancement of pre- pre-trained trained model
C.6 Hyperparameter experiments
Here, we explore several crucial hyperparameters of DyCC. The first one is B, which controls the trade-off between prediction and compression in our final objectives. As shown in Fig 5]a). there exists an optimal point at 3 = 0.01 in terms of model performance, indicating the trade-off between prediction and information compression. Setting a larger 3 1 encourages aggressive information compression, leading to difficulties in capturing the core subgraph related to the target task. Conversely, decreasing & encourages the model to retain the original information of the given graph structure. In an extreme case, i.e., when 8 = O, the model focuses solely on the prediction term, potentially leading to a lack of generalization ability. The second parameter is the number of prototypes We find that as increases from 16 to 128, the pretraining performance gradually improves, while using more prototypes has little effect. Therefore, n 128 is a suitable choice. Lastly, Ty and Tp control the soft label sharpening level. We always choose Ty Tp to encourage
24

---

sharper target predictions, implicitly guiding the model to produce confident low-entropy predictions. We fix Ty 0.1 and set Tp {0.15, 0.2, 0.25, 0.3). Experimental results show that different models have different optimal values of Tp. which may be due to the different tokenizer types.
D Algorithm for GIBMS and SLT
Algorithm T and Algorithm SLG process, respectively.
2 provide a comprehensive description of the GIBMS algorithm and the
Algorithm The training process of GIBMS
I: Input: Unlabeled molecular pre-trained graph dataset {91, 92. GNN encoder Φ, and node importance evaluation MI.P M..
2: Initialize parameters of & and M.
3: for each graph 9 in Dj do
4: Encode 9 into node representations: H (G))
Generate sampling probability for each node: p MMH)).
6:
Apply the Gumbel-Sigmoid function to sample À from based on Eq.
Inject noise into obtain H based Eq. (6).
Compute the unsupervised prediction loss based on Eq.
9:
Compute the loss for the compression term based on Eq. 12
10:
Calculate the total loss for the first stage based on Eq. 13
11: Perform backpropagation optimize the training objective.
end
13: Return: The well trained d and M jointly constitute the GIBMS module M()(
Algorithm 2 Soft Label Generator
1: Input: Unlabeled molecular pre-trained graph dataset 2) (G,, G2, the pre-trained GIBMS model (G)())) GNN encoder d' for MGAEs, and learnable prototypes matrix Q.
Initialize the parameters Ø' and Q.
3: for each graph G in D2
4: Compute the sampling probability for cach node as = Sigmoid(.M(H).
of important nodes Viuss {V; V/ ~ Bernuuli(1 p.), ., N). 6: Replace the nodes Vinsk within graph with MASK token obtain Geosk
5: Sample
Obtain the node representations
(Gmms))
8:
Compute the soft label assignments s" for all nodes by applying Eq. Compute the soft label assignments s, for all nodes by applying Eq.
15
to H and Q.
9:
(4 to the node labels and Q.
10: Minimize the distance between 8, and s" according to Eq. (17).
Perform backpropagation to optimize training objective.
12:
end
13: Return: The pre-trained GNN encoder Th for various downstream tasks.
E
Proof
E.1 Proof of Eq. 10
By adopting the common multi-view assumption
30 42
we have:
I(9i;; Yuu) = / (Zep sup)
2 1 (Zel suu
> (9m; Yuu)
finfoi
Cinfo
>>
0
The proofs contain two parts (36] The first one is showing the results for the supervised learned representations and the second one is for the self- self-supervised learned representations.
Lemma (Determinism) If (Z GIB) Dirae, then the following conditional independence holds: suu 1 ZZg GIB and S 1 Zgn GIB. inducing Markov chain S ++ Yaup GIB
25

---

Zem- When Zgm is a deterministic function of GIB, for any A in the sigma-algebra induced by Zgn
we have E 1 iooee EA] implies Y Zeu GIB and $ Zer GIB-
GIn, { {Ysup, [S] - E 1 IZore EA]
9IB, S
= E 1o EA]
Gin which
Supervised Learned Representations Adopting Data Processing Inequality [9] in the Markov chain S Y <> 9IB - Om (ZGm Ysuu) is maximized at T99:; Y Since the supervised
learned representations 2 maximize/ (Ze Y we conclude I (Z Y I91s: Y
Self-supervised Learned Representations First, we have
1 (Zem S)
I (Zggs Y
- (Zgm² Y () + I (Zem: S |
I (Zgi Y
S) + I (Zeis ; Y
and
I(9I: S) I(I:: Y
- 199B; suu S) + [(9 S Y
[(IB: Y ) (9B: S Y
By DPI in the Markov chain S ++ Y + GIB - Zam we know
(Zers S) maximized at ((
I (Zggs i S; YsuP) is maximized at / (91B; S; Yaup)
7 (Zois: S Youp) is maximized at T(G:;; S Ysup)
Since the self-supervised learned representations Z maximize (Zeis S), we have I (Z :S) I(Gm; Hence (Z
Ysup) [( S Ysup). Using the result / (Zeel S; Ysup)
T(Gin;
S; YouP), we get
Ysup) 1(91B; Y
Gie
- I(GIB; Ysup S)
I (Z Y S)
Now, we are ready to present the inequalities:
I[GI:; Y 2 I (Zgl) Y due to I(GiB; sup
S) 2 I (Z Y
|S) by DPI and
[Zgs] Y T(II:; Ysup)
Gifaa due to
I(GI;; Youp) - I(GIB: Youp S)+ ZZal Vin
Ysup S)
> ((IB; Yuu)
1(GB: Y
Einfo
(18)
where 1(GB;; Yuu () S Cinfo by the redundancy assumption.
E.2
Proof of Eq.
We derive the upper bound of I (G; GiB) by introducing the variation approximation (91)) of distribution p (
Ps (Gm (9)
(9; G1B) Eg.gm
log
p(g)
log Ps (GIn ())
Eg..m
q(G1B)
Bgn. [K (p (G)) llq (Gm))
According to the non non-negativity of KL divergence, we have:
(GBB 9) Eg [KL (ps ((G 99) ||q (GB))
26

---

We assume that (1B) ) is obtained by aggregating the node representations in a fully perturbed graph. The noise 6 ~ N (1 ( sampled from a Gaussian distribution where μи and of are mean and variance of H. Choosing sum pooling as the aggregation function, since the summation of Gaussian distributions is a Gaussian, we have the following form:
(GB) = N (NPH. NoH
Then for Ps (9IB 9), we have the following equation:
N NHH,
N
N/
NOH + Σ
AHH
(1-
11
j=1 ==1
j=1
Finally, we have following inequality:
1
1
7 19mB 9) S Rg
log A
B²
+C
2N
2N
ΣN As
where A= E= (1 - 22)² and B
6H
27


Reviews:
[{"review_id": "QcwDZkmYF2", "reviewer": "Reviewer_oeWH", "summary": "This paper introduces a new method of dynamic masking strategy to enhance molecular masked graph autoencoder. The authors first show that a static mask ratio and random graph corruption are suboptimal in molecular graph pretraining with masking-reconstruction loss. To address this issue, this paper utilize Graph Information Bottleneck to calculate an importance socre of each atom, based on which a masking probability is determined, so that a adaptive masking strategy can be applied to each molecule. Experiments are conducted on a dataset sourced from the 252 ZINC15 database and the proposed strategy are integrated into three different Masked Graph Autoencoders for molecular representation learning. Experimental results show that the proposed method enhance the performance of the three Masked Graph Autoencoders and the integrated models outperform existing a series of molecular pretraining methods.", "questions": "1. Please respond to the weakness points.\n2. Does the proposed method bring high time cost to pretraining?", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength\n1.It is valuable to show that dynamic masking based on the importance score of each atom is superior than random masking with a fixed masking ratio in masked graph autoencoder pretraining. \n2.The paper is generally well written and easy to follow and the motivation is reasonable. \n3.The strategy of using information bottleneck to calculate an importance score of each atom is feasible. The contribution of using information bottleneck in an unsupervised scenario is valuable. \n\nWeakness\n1.The major weakness is that the baseline methods are weak and outdated. The proposed method is compared to quite a lot self-supervised graph pretraining models, but the results are from MoleBert [41], which is published in 2023. After the publication of [41], self-supervised pretraining for molecular representation learning has advanced rapidly and quite a lot new methods have been proposed. A search of papers citing [41] can find some of them. Here I list several papers that the authors may consider.\n\n[1] Hongxin Xiang, et al. An Image-enhanced Molecular Graph Representation Learning Framework. IJCAI 2024\n[2] Liang Wang, et al. MOLSPECTRA: PRE-TRAINING 3D MOLECULAR REPRESENTATION  WITH MULTI-MODAL ENERGY SPECTRA. ICLR 2025. \n[3] Yue Wan, et al. Multi-channel learning for integrating structural hierarchies into contextdependent  molecular representation. Nature Communications 2025. \n\n2.Relevant to the first point, this paper lacks a review of self-supervised molecular pre-training techniques. A comprehensive review of this field can place the current research in the correct context.\n\n3.The proposed method is only evaluated on one dataset. \n\n4.The is not comparison to conventional masking strategies with proper hyper-parameter selection.", "was_revised": true, "final_justification": "My major concern was the weak baselines in the experiments. In the response, more recent methods are added in comparison and with these new baselines, the performance gains looks even less significant.  Therefore, I keep my original score.", "rebuttal": "Thank you very much for your valuable feedback and suggestions. Below, we address each point in turn.\n\n---\n\n> **Q1.** _Weakness 1. The major weakness is that the baseline methods are weak and outdated. The proposed method is compared to quite a lot self-supervised graph pretraining models, but the results are from MoleBert [41], which is published in 2023. After the publication of [41], self-supervised pretraining for molecular representation learning has advanced rapidly and quite a lot new methods have been proposed. A search of papers citing [41] can find some of them. Here I list several papers that the authors may consider._  \n> [1] Hongxin Xiang, et al. IJCAI 2024  \n> [2] Liang Wang, et al. ICLR 2025  \n> [3] Yue Wan, et al. Nature Communications 2025\n\nThank you for pointing this out and for listing several recent works. We agree that the baseline set can be further strengthened and have updated the experiments accordingly.\n\n1. **Expanded baseline set.** In addition to the MGAE‑style methods originally reported (AttrMask [2020], GraphMVP [2021], Mole‑BERT [2023], and SimSGT [2024]), we have added the recent models you suggested (IEM‑GraphMVP, IEM‑Mole‑BERT, StructMAE‑P/L). The updated table below reports their numbers on the same scaffold‑split evaluation protocol as used in our paper:\n\n|Method|Tox21|ToxCast|Sider|ClinTox|MUV|HIV|BBBP|BACE|**Avg.**|\n|---|---|---|---|---|---|---|---|---|---|\n|IEM‑GraphMVP|75.6|64.8|62.0|79.2|77.0|78.2|71.4|81.9|**73.8**|\n|IEM‑Mole‑BERT|77.8|65.6|65.3|79.7|72.2|78.8|68.1|83.0|**73.8**|\n|StructMAE‑P|75.8|64.5|62.0|86.0|77.7|77.4|84.3|72.6|**75.0**|\n|StructMAE‑L|75.3|64.0|61.3|87.9|78.0|78.3|72.5|83.2|**75.1**|\n|**SimSGT‑DyCC‑GraphTrans (Ours)**|76.6|66.3|62.0|83.6|80.3|77.8|72.2|84.9|**75.5**|\n\nThe strongest average result among the new baselines is 75.1 (StructMAE‑L), which is still lower than our **SimSGT‑DyCC‑GraphTrans** (75.5) (see Table 6).\n\n2. **Scope clarification.** Our primary goal is not only to chase SOTA, but to diagnose limitations of existing molecular masked pre‑training paradigms (e.g., fixed masking ratio, rigid reconstruction targets) and propose a general‑purpose solution (**DyCC**) that can be plugged into a variety of MGAEs.\n    \n3. **References and discussion.** We will add the recent works you suggested (IEM‑GraphMVP, IEM‑Mole‑BERT, StructMAE) and others published after 2023 into the related‑work section and discuss how DyCC differs conceptually.\n    \n\nIn summary, the updated results show that our approach remains competitive or superior to recent methods, while being a **model‑agnostic plug‑in** rather than a full‑stack redesign.\n\n---\n\n> **Q2.** _This paper lacks a review of self-supervised molecular pre-training techniques. A comprehensive review of this field can place the current research in the correct context._\n\nThank you for this constructive suggestion. We agree that a more comprehensive overview will improve the clarity and positioning of our contribution.\n\nIn **Section 2**, we currently review two main paradigms of self‑supervised molecular graph pretraining:\n\n- **Graph contrastive learning**, e.g., GraphCL, JOAO, GraphLOG, RGCL, SimGRACE\n    \n- **Graph generative (masked) learning**, e.g., AttrMask, GraphMAE, Mole‑BERT, SimSGT\n    \n\nThese representative methods are directly relevant to our proposed DyCC framework. In addition, **Appendix C.3** provides detailed descriptions of all baselines used in Table 1.\n\nThat said, we acknowledge that the field has been evolving rapidly, and several recent works were not explicitly covered. In the revised manuscript, we will expand **Section 2** into a broader taxonomy of self‑supervised molecular pretraining, including **multi‑view/multi‑modal methods**, **contrastive learning**, **3D‑aware and SE(3)‑equivariant models**, and **generative objectives**.\n\n\n---\n\n> **Q3.** _The proposed method is only evaluated on one dataset._\n\nWe respectfully clarify that our method is evaluated on **multiple datasets and task types**, not just a single dataset:\n\n- In **Section 4 (Table 1)**, we report results on **eight classification tasks** from MoleculeNet.\n    \n- In **Appendix Table 5**, we further evaluate on **four molecular property regression tasks** and **two drug–target affinity (DTA) regression tasks**.\n    \n- In **Appendix Table 7**, we also report results on **quantum chemistry property prediction** tasks.\n    \n\nThese evaluations span different domains (classification, regression, and quantum property prediction), demonstrating the **robustness and generality** of the proposed method.\n\n---\n\n> **Q4.** _There is no comparison to conventional masking strategies with proper hyper-parameter selection._\n\nWe would like to clarify that for **AttrMask**, **Mole‑BERT**, and **SimSGT**, the masking strategy is primarily controlled by the **masking ratio**. The authors of these methods have already conducted extensive hyper‑parameter studies in their original papers, and we follow their **default settings** to ensure fair and consistent comparison.\n\nFor example, Mole‑BERT (MAM) reports results across multiple masking ratios in Table 7 of the original paper (reproduced below), showing that 0.20 is near optimal:\n\n|Masking ratio|0.10|0.15|0.20|0.25|0.30|\n|---|---|---|---|---|---|\n|MAM|71.64|72.16|**72.21**|71.51|71.36|\n\nOur **AttrMask‑DyCC**, **Mole‑BERT‑DyCC**, and **SimSGT‑DyCC** experiments keep the same experimental setup and hyper‑parameter configurations as the corresponding baselines. We did not conduct additional hyper‑parameter searches to avoid introducing bias and to ensure a fair comparison.\n\n---\n\n> **Q5.** _Does the proposed method bring high time cost to pretraining?_\n\nWe appreciate the reviewer’s concern.\n\n**Theoretical analysis:** Our method introduces two lightweight modules—**GIBMS** and **SLG**.\n\n- GIBMS is trained **once and reused** across all runs, so its cost is amortized.\n    \n- SLG only adds two matrix multiplications (Eq. 14 and Eq. 15: $\\mathbf{H}^y \\cdot \\mathbf{Q}$ and $\\mathbf{H}^p \\cdot \\mathbf{Q}$), which introduce **negligible computational overhead**.\n    \n\n**Empirical results:** All experiments were conducted on an NVIDIA DGX A100 server.\n\n- GIBMS training takes **683 minutes** (one‑time).\n    \n- Adding SLG to AttrMask increases pretraining time from **512 min to 561 min**, a modest 8.7% increase.\n    \n\nThus, the proposed method achieves performance gains with only a **small and manageable time cost**.\n\n___\n\nWe hope these explanations and corrections satisfactorily address your concerns. Thank you again for your thoughtful review."}, {"review_id": "Eaww5ZOOtb", "reviewer": "Reviewer_MkJF", "summary": "The manuscripts presents two techniques to improve self-supervised learning on graphs. First, a masking strategy that learns an individual masking probability for each atom in a molecule based on its importance. Second, a soft label technique that replaces the learned atom embeddings by learned prototype embeddings and allows the model to effectively lower the loss for predicting atoms or atom environments that are similar to each other.", "questions": "- I remember a similar paper by Liu et al. called \"Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders\". They also developed a node importance strategy and I hoped to compare the benchmark results between both papers. I found that the numbers differ, though. For example, in Liu et al. the numbers for AttrMask are better than in this manuscript. The same is for the original manuscript of Hu et al. The manuscript mentions that the numbers of the baseline methods were taken from prior studies, is this also the case for AttrMask and, if so, why are the numbers different?\n\n- can you evaluate if the importance scores of the atoms are stable across different pretraining runs or if a different set of atoms is chosen to be \"important\" every time?\n\n- The soft label approach somewhat reminds me on vector quantization (VQ), which is also often used to bridge regression methods and discrete tokens. Can you elaborate why your softlabel approach works better and what makes it different?", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 2, "strengths_and_weaknesses": "The manuscripts provide two different techniques for improving self-supervised learning on graphs and demonstrate on benchmarks that \n1.) these techniques can be applied on different models\n2.) consistently improve the pretraining results for these models\n\nThe manuscript is well written. I am not able to verify the proofs in manuscript but the method sounds reasonable. Given that both techniques, node importance and soft labels, can be easily visualized and manually verified, I wished the manuscript would put more emphasize on discuss if the node importance or soft labels do make sense. For example, Figure 1a says that Br and Cl are interchangeable in this molecule, but there is no discussion or evaluation in the manuscript if the soft labels indeed cluster together atom environments that are easily exchangeable or have similar chemical properties. The same is true for the node importance prediction. In 4.3 it is mentioned that in 74% of the cases in a mutagenic dataset, 30% of the most important atoms were NH2 or NO2, which is chemical reasonable. However, molecules usually consist of a carbon backbone and 30% of the most important atoms in a molecule should already cover most of the non-carbon atoms. It is not that surprising that the method founds out that carbon atoms are not important. \nI also wonder if the atom importance is \"stable\" over multiple training runs, or if the method always selects a different set of atoms to be important.", "was_revised": true, "final_justification": "From my own reading of the manuscript and the other reviews, the main concern about this paper are the benchmarks. Even though the improvements in benchmarks are relatively small, I agree with the authors that they are consistent across a wide variety of tasks and models. The method presented here is relatively method-agnostic and might be very helpful in the field of graph neural networks for molecules.", "rebuttal": "Thank you very much for your valuable feedback and suggestions. Below, we address each point in turn.\n\n---\n\n\n> **Q1.** _I remember a similar paper by Liu et al. called \"Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders\". ..._\n\n\nWe sincerely thank the reviewer for this careful observation. We clarify the issue and its impact in detail below:\n1. **Why the differences exist.** The baseline results reported in our Table 1 (e.g., for AttrMask) were taken from the reproduction experiments of Mole‑BERT for consistency. As you correctly observed, these numbers differ slightly from those reported in StructMAE (e.g., 70.8 vs. 71.1 for AttrMask). Similar discrepancies can also be seen for other baselines such as GraphMAE. Unfortunately, neither Mole‑BERT nor StructMAE provides full baseline reproduction code, which makes it difficult to align every implementation detail (e.g., random seeds, preprocessing, training hyperparameters). In future updates, we will re‑evaluate all baselines using the official codes of these works (where available) to further minimize such inconsistencies.\n\n2. **Impact on our conclusions.** We would like to emphasize that these differences do not affect the validity of our conclusions:\n+ Regardless of the baseline number (AttrMask = 70.8 in Mole‑BERT vs. 71.1 in StructMAE), our DyCC‑enhanced model AttrMask‑DyCC achieves 73.7, consistently outperforming the baseline.\n+ Across all baselines cited in these works, the best performance reported by StructMAE is 74.0 (Mole‑BERT). This is still lower than the best result achieved by our framework: 75.5 using SimSGT‑DyCC‑GraphTrans (Appendix Table 6).\n+ The core focus of this paper is to demonstrate that DyCC is a plug‑and‑play framework that improves a range of existing MGAE models. We integrated DyCC into AttrMask, Mole‑BERT, and SimSGT, and in all cases it significantly improved the underlying baseline model.\n\n3. **Clarifying Mole‑BERT and SimSGT settings in Table 1.** We apologize for any confusion caused by differences in the reported numbers and the original papers. In Table 1, we intentionally excluded auxiliary components unrelated to masked atom modeling in order to evaluate DyCC more fairly:\n\n+ Mole‑BERT includes two modules: MAM (Masked Atom Modeling with VQ‑VAE) and TMCL (Triplet Masked Contrastive Learning). Since TMCL is contrastive learning–based and outside the scope of our study, we only report the MAM results in the main text. Full results including TMCL are provided in Appendix Table 6.\n\n+ SimSGT uses a Graph Transformer backbone by default, while other baselines (e.g., AttrMask, Mole‑BERT) use GNN backbones. To avoid backbone effects, we report the GNN‑backbone variant of SimSGT (also reported in the original SimSGT paper) in the main text. Results with the original Graph Transformer backbone are reported in Appendix Table 5.\n\nIn the revised manuscript, we will update the naming of these variants to make this clearer.\n\n4. **Comparison with StructMAE** \nStructMAE also focuses on node importance but differs fundamentally from our GIBMS module in its motivation and design:\n\nImportance scoring: StructMAE computes a structural importance score for each node either using:\n\n+ Pre‑defined heuristics such as PageRank (StructMAE‑P), or\n+ A lightweight learnable scoring network trained jointly with the model (StructMAE‑L).\n\n**Principle**: In contrast, GIBMS is derived from the information bottleneck principle, focusing explicitly on preserving semantic completeness in molecular graphs.\n\n**Training vs. reuse** : StructMAE jointly learns masking scores during training for each model, whereas GIBMS is trained once and can be reused across different backbones, making it more modular.\n\n**Masking curriculum** : StructMAE adopts an “easy‑to‑hard” curriculum (gradually masking more important nodes), which we do not use.\n\n**Additional modules**: We further introduce the SLG module to mitigate label imbalance, which StructMAE does not consider.\n\n**Focuses**: Our method focuses specifically on the molecular domain, emphasizes cross‑model generality, and can be integrated into multiple MGAE models, while StructMAE is designed as a stand‑alone general‑graph pretraining model.\n\n**Performance comparison**: When integrated into SimSGT, DyCC achieves a mean ROC‑AUC of 75.5 on MoleculeNet, surpassing StructMAE’s best result (75.1).\n\n\nTo summarize:\n\n+ The baseline discrepancies arise from differences in sources (Mole‑BERT vs. StructMAE) and unavoidable implementation details.\n\n+ These differences do not alter our conclusions, as DyCC consistently outperforms the baselines regardless of the exact baseline number.\n\n+ Table 1 uses simplified settings (MAM‑only for Mole‑BERT, GNN‑backbone for SimSGT) to enable a fairer comparison.\n\n+ Compared to StructMAE, GIBMS has a stronger theoretical foundation (information bottleneck), is reusable, and is tailored for molecular graphs.\n\n\n\n> **Q2.** _Can you evaluate whether the atom importance scores are stable across different pretraining runs, or does each run select a different set of “important” atoms?_\n\n\nWe thank the reviewer for raising this point.\n\n1. **Stochastic sampling design.**  \n    The GIBMS module learns an importance score for each atom, but the masking itself is sampled stochastically using a Gumbel–Sigmoid reparameterization:\n    \n\n$$ \\begin{aligned} &\\lambda=\\operatorname{Sigmoid}\\left(\\frac{1}{t} \\log \\left[\\frac{p}{(1-p)}\\right]+\\log \\left[\\frac{u}{(1-u)}\\right]\\right)\\\\ \\end{aligned} $$\n\nwhere $t$ is the temperature hyper‑parameter. Thus, the learned scores are deterministic once the model is trained, but the actual mask selections during training are stochastic by design.\n\n2. **Stability evaluation.**  \n    To directly assess whether the learned importance scores are stable across pretraining runs, we retrained GIBMS three times with different seeds and repeated the Section 4.3 MUTAG functional‑group identification test, using detection accuracy of the mutagenic groups (NH2_2/NO2_2) as the metric. The accuracies across runs were **0.74, 0.72, and 0.75**, demonstrating a high degree of consistency.\n    \n3. **Conclusion.**  \n    This consistent performance suggests that GIBMS produces **stable importance scores across different pretraining runs**, enabling reliable identification of key functional groups, even though the masking process itself is stochastic.\n   \n\n\n>**Q3** The soft label approach somewhat reminds me on vector quantization (VQ), which is also often used to bridge regression methods and discrete tokens. Can you elaborate why your softlabel approach works better and what makes it different?\n\n\nWe sincerely thank the reviewer for this insightful question regarding the relation between our soft label approach and vector quantization (VQ). Indeed, Mole‑BERT adopts a VQ‑style method. While both approaches share the idea of using prototype‑based representations, there are fundamental differences in their motivation, mechanism, and application, as detailed below:\n\n1. **Purpose and Conceptual Distinction**  \n    VQ is primarily designed to _quantize_ continuous representations into discrete codes through a hard nearest‑neighbor assignment to a fixed codebook, often with commitment losses to stabilize training. This makes VQ well‑suited for bridging continuous and discrete representations (e.g., in VQ‑VAE). In contrast, our method does **not** quantize representations. Instead of selecting a single prototype, we **retain the entire soft distribution** over prototypes as the target. This preserves uncertainty and provides richer supervision via _soft targets_, which is particularly beneficial in masked atom modeling, where multiple reconstructions can be semantically plausible.\n    \n2. **Soft Labels vs. Hard Assignments**  \n    VQ’s hard assignments inherently discard uncertainty and can introduce gradient mismatch. Our approach deliberately avoids hard assignments by using a **temperature‑controlled softmax** over cosine similarities to prototypes. \n    \n3. **Task‑Specific Motivation**  \n    Masked atom modeling often involves chemically similar atoms as potential reconstructions. Hard targets (as in VQ or standard classification) ignore this nuance. Our soft label design explicitly leverages **temperature‑scaled similarity** to learnable prototypes, generating probabilistic targets that better capture chemical variability.\n    \n4. **Prototype Utilization and Regularization**  \n    To further enhance learning, we introduce **Mean Entropy Maximization (ME‑MAX)** regularization, which encourages balanced prototype utilization and prevents collapse. This regularization is not typically considered in VQ approaches.\n    \n5. **Empirical Effectiveness**  \n    As shown in our experiments, the proposed soft label method consistently outperforms hard label (e.g., AttrMask) and VQ‑style (e.g., Mole-Bert) baselines. We attribute this to its ability to encode uncertainty and chemical similarity through soft supervision.\n    \n\n**Summary:**  \nWhile both methods involve prototypes, our soft label approach fundamentally differs from VQ by **eschewing quantization**, embracing **soft distributions** as supervision, and explicitly modeling uncertainty. These differences yield richer training signals, more robust prototype usage, and ultimately better generalization, particularly in the context of molecular masked modeling.\n\n\n---\n\n\n\n\n> **Q4.** _I think the $\\alpha H\\left(s^{\\bar{y}} p\\right)$   in equation 17 should be $\\alpha H\\left(\\bar{s}^{\\bar{p}}\\right)$ but I am not sure. The formula seems to have a typo, though._\n\n\nWe sincerely thank the reviewer for carefully checking our equations. You are correct that there is a typo in Equation 17. The correct formulation should be:\n\n$$\n\\alpha H\\left(\\bar{s}^{\\bar{p}}\\right)\n$$\n\ninstead of\n\n$$\n\\alpha H\\left(s^{\\bar{y}} p\\right)\n$$\n\n___\n\nWe hope these explanations and corrections satisfactorily address your concerns. Thank you again for your thoughtful review."}, {"review_id": "S4kfU2MqqY", "reviewer": "Reviewer_Uxih", "summary": "The authors identify that pre-training masked graph autoencoders (MGAEs), commonly used to model moecules, lack the flexibility necessary to represent chemical information. This is because: 1) the pre-training masks a fixed number of tokens, 2) the tokens are chosen arbitrarily when some components of molecules clearly matter more than others, and 3) the tokens are decoded into specific discrete labels. To mitigate these issues, the authors introduce Dynamic Chemical Constraints (DyCC), a model-agnostic pre-training algorithm composed of two modules: a) graph-information bottleneck mask strategy (GIBMS), which identifies the most relevant tokens to mask using graph information theory, and b) Soft Label Generator, which replaces the discrete labels per token with a distribution over learnable prorotypes for each token. DyCC is used to pretrain a number of MGAEs, showing an improvement over regular pretraining in most cases.", "questions": "- Can you report the training time as well in Appendix C.1?\n- In figure 3, you mention that SGL yields a more uniform distribution of tokens on the ZINC dataset, when the data has 95% of atoms with the 3 most common labels. Is a uniform distribution desired here?", "limitations": "You could add additional limitations regarding the societal impact of your work.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths\n- The method is theoretically grounded. The GIBMS loss (equation 13) is derived from graph information bottleneck following detailed and well-justified steps. \n- Adapting GIB to the self-supervised setting is an interesting use case extending the applicability of the framework to various applications.\n- The method was thoroughly tested on a number of applications, and the results are reported transparently with detailed experimental procedures and error intervals.\n\n## Weaknesses\nMy main concerns with this work are with its **analysis of empirical results** and its **presentation**. I elaborate below on these two points and ask for related clarifications in **Questions**.\n* **Unsubstantiated claims in the empirical analysis (Section 4.2)**: I am unable to find evidence for the following claims in the paper:\n    - \"SimSGT-DyCC surpasses the \"no pretrain\" model by 10.4%.\" the average difference between the two models is actually 7.1% according to Table 1. If the authors are referring to the results for a specific dataset, this should be made clearer (I could not find any dataset with the exact difference reported). It would also be misleading to refer to a specific dataset instead of the average performance unless justified. \n    - \"\"substantial performance variations were observed among the original AttrMask, MoleBert, and SimSGT models due to differences in tokenizers.\": why is a difference of approx 2% (according to Table 1) considered substantial? \n    - \"The success rate of GIBMS was 74%, demonstrating the effectiveness of the GIBMS module.\": this is a promising result, but can you compare it to other baselines (in identifying relevant components of a graph) to get a sense of its importance?\n    - Figure 4 (left): can you report the results of a similar experiment on a larger scale? for example, you can compute the accuracy of identifying non-ring components as the most relevant elements in ring molecules. \n\n- **Presentation**: I found the way you motivated your approach confusing, especially in the introduction. If I understand your text correctly, your main concerns are that regular masking does not leverage relevant chemical information, both in the choice of tokens to mask (relevant tokens differ per molecule) and in the labels chosen when unmasking (many possible labels are equally good). The introduction  breaks down these concerns into irrelevant details (e.g. the 3 chemical constraints) especially since most are addressed by the same solution. I suggest either justifying your taxonomy through additional examples and by linking it to the part of the module addressing it, or simplifying the motivation in the introduction. \n\n- **Pointing out errors in the text**: \n    - paragraph \"Are both GIBMS and SLG necessary\" is redundant. This point was already made in the paragraph right before. I suggest moving the paragraph \"limitations\" in the appendix to the main text instead.\n     - equation 13 has an additional L_cl term, I think this is supposed to be the same as L_pred and therefore only one of the two should be mentioned in the equation?\n     - you are missing a legend in Figure 1. Also the colors of the plot lines changed between subfigures a and b in the same figure. \n     - line 36: change make => make\n     - line 176: typo near  \"To ensure\"\n     - I suggest introducing $Y^{sup}$ notation from the beginning, instead of replacing $Y$ later in the text.\n     - table 6 in the appendix missing bolding.", "was_revised": true, "final_justification": "The authors have addressed my concerns in the rebuttal. I am still unsure though of the impact of their work on the field of molecular generation at large: while experiments demonstrate consistent improvement (of approx. 1%) compared to baselines, the improvement remains modest, especially for a method that supposedly addresses key shortcomings in previous models.\n\nUPDATE: the authors provided good arguments to explain the limitations of their empirical performance. They have thus addressed all my concerns and I adjusted my score accordingly. I am not thoroughly familiar with the related literature however, so I am happy to discuss with other reviewers if they contest any of the authors' claims.", "rebuttal": "Thank you very much for your valuable feedback and suggestions. We carefully address each point below.\n\n---\n\n> **Q1.** _\"SimSGT-DyCC surpasses the 'no pretrain' model by 10.4%.\" The average difference between the two models is actually 7.1% according to Table 1...\n\nThank you for pointing this out. The reported **10.4%** refers to the **relative improvement** in the average ROC-AUC between SimSGT-DyCC and the \"no pretrain\" model:\n\n$$ \\frac{74.1-67.0}{67.0} \\approx 10.4 \\% $$\n\nHere, 74.1 (\"SimSGT-DyCC\") and 67.0 (\"no pretrain\") are the average ROC-AUC scores across the eight MoleculeNet classification datasets (Table 1, L257-L263).  \n**Note:** dataset-level scores are rounded to two decimal places in the table, but the relative difference was computed using unrounded values.\n\n\n---\n\n> **Q2.** _\"Substantial performance variations were observed among the original AttrMask, MoleBert, and SimSGT models due to differences in tokenizers.\": why is a difference of approx 2% (according to Table 1) considered substantial?_\n\nThank you for raising this important question. While a ~2% absolute performance gap may seem modest at first glance, we consider it **substantial** in the context of molecular representation learning for the following reasons:\n1. **Small gains are meaningful on saturated benchmarks.**  \n   The MoleculeNet benchmarks we evaluate on (e.g., Tox21, HIV, BBBP) are well-established and highly competitive. Recent state-of-the-art (SOTA) works often highlight consistent improvements of 1–2% ROC-AUC as meaningful. For example, SOTA models evolved from ~71% (AttrMask, GraphCL, 2020) to ~74% (MoleBert, 2023) and 75% (SimSGT/StructureMAE, 2024). Our best model, **SimSGT-DyCC (GraphTrans)**, further reaches 75.5% (Appendix Table 6).  \n\n2. **Broad improvements across diverse tasks.**  \n   Beyond the eight MoleculeNet classification datasets (Table 1), DyCC consistently improves performance on **4 regression tasks**, **2 drug–target affinity prediction tasks**, and **3 quantum-mechanical property tasks** (Appendix Tables 5 & 7, L532–L544). These consistent gains demonstrate DyCC’s generality rather than isolated improvements.  \n\n3. **Statistical significance and consistency.**  \n   We conducted five independent runs for each model configuration. Even though the absolute improvements range from ~1.5% to ~4.3%, all differences are statistically significant and consistent across datasets and tasks.  \n\n   **Five-run results:**  \n\n   | Model           | Run1 | Run2 | Run3 | Run4 | Run5 |\n   |-----------------|------|------|------|------|------|\n   | AttrMask        | 70.8 | 70.1 | 70.9 | 70.9 | 70.7 |\n   | AttrMask-DyCC   | 73.7 | 73.7 | 73.8 | 73.6 | 73.8 |\n   | Mole-BERT       | 72.2 | 72.1 | 72.1 | 72.4 | 72.3 |\n   | Mole-BERT-DyCC  | 73.8 | 73.8 | 73.9 | 73.7 | 73.8 |\n   | SimSGT          | 73.0 | 73.2 | 72.9 | 73.0 | 73.1 |\n   | SimSGT-DyCC     | 74.1 | 74.3 | 74.2 | 74.2 | 74.0 |\n\n   **Statistical analysis:**  \n\n   | Comparison                    | p-value   | Mean Baseline | Mean DyCC |\n   |-------------------------------|-----------|---------------|-----------|\n   | AttrMask vs AttrMask-DyCC     | 0.000038  | 70.68         | 73.72     |\n   | Mole-BERT vs Mole-BERT-DyCC   | 0.000052  | 72.22         | 73.80     |\n   | SimSGT vs SimSGT-DyCC         | 0.000072  | 73.04         | 74.16     |\n\n4. **Real-world impact in virtual screening.**  \n   In practical applications such as virtual screening, even a 1–2% ROC-AUC improvement can translate into **significantly higher hit rates** and substantial reductions in experimental cost.  \n\n---\n\n**Summary:**  \nAlthough a ~2% absolute difference may seem small in isolation, in the context of mature molecular property prediction benchmarks, such improvements are **systematic, statistically significant, and practically important**. This fully justifies our use of the term “substantial” and further motivates the tokenizer-agnostic DyCC framework.\n\n---\n\n\n> **Q3** _The success rate of GIBMS was 74%, demonstrating the effectiveness of the GIBMS module.\": this is a promising result, but can you compare it to other baselines (in identifying relevant components of a graph) to get a sense of its importance?_\n\n\nThank you for this suggestion. Most prior subgraph identification methods are **supervised** and thus cannot be directly compared to our **unsupervised** setting. The method most similar to ours is **RGCL** [1], which uses Rationale-aware Graph Contrastive Learning to identify important nodes in a pretraining dataset, also in an unsupervised manner.\n\nFor reference, we also report the performance of **PGExplainer**, a post-hoc explanation model for supervised GNNs, on the same task.\n\n| Method           | Principle                                 | Type         | Accuracy |\n| ---------------- | ----------------------------------------- | ------------ | -------- |\n| **RGCL**         | Contrastive learning + invariant learning | Unsupervised | 69.5     |\n| **GIBMS (ours)** | Graph information bottleneck              | Unsupervised | 74.0     |\n| **PGExplainer**  | Mutual information                        | Supervised   | 87.3     |\n\nThese results indicate that, under the unsupervised setting, our proposed **GIBMS** more effectively identifies important nodes compared to RGCL. Its performance is lower than the supervised PGExplainer, which is expected because GIBMS is applied during pretraining on task-agnostic datasets rather than task-specific fine-tuning.\n\n[1] Li, Sihang, et al. \"Let invariant rationale discovery inspire graph contrastive learning.\" _International conference on machine learning_. PMLR, 2022.\n\n> **Q4**: _Presentation: I found the way you motivated your approach confusing, especially in the introduction. If I understand your text correctly..._\n\n\n\nThank you very much for your insightful feedback. We acknowledge that the motivation in the introduction could have been presented more clearly. In the original version, we decomposed the limitations into three chemically motivated constraints (CC1–CC3) to emphasize specific aspects often violated by existing approaches. However, as you correctly pointed out, these ultimately boil down to **two key challenges** in masked graph modeling for molecules:\n\n1. **Lack of adaptivity in masking strategies across molecules** (corresponding to CC1 & CC2);\n    \n2. **Ambiguity in reconstruction targets due to inherent chemical variability** (corresponding to CC3).\n    \n\nOur intention for presenting CC1–CC3 was to analyze the limitations from a **chemistry-centric perspective**, which might resonate better with chemical researchers. From a **model design perspective**, however, summarizing them as the two principles above is indeed clearer and easier to follow.\n\nWe appreciate your suggestion and will revise the **Introduction** to explicitly map each chemical constraint to the corresponding design component:\n\n- **CC1 & CC2 → Lack of adaptivity → GIBMS**\n    \n- **CC3 → Ambiguity in reconstruction targets → SLG**\n    \n\nThis restructuring will make the intuition–solution mapping clearer and ensure that readers from different research backgrounds can quickly grasp the motivation and contributions of our work.\n\n\n\n\n\n\n> **Q5** Pointing out errors in the text\n\nThank you very much for your careful reading of our paper. We appreciate you pointing out the typos and providing helpful suggestions for corrections. These improvements are valuable and will certainly enhance the overall quality of the manuscript.\n\n> **Q6** Can you report the training time as well in Appendix C.1?\n\nOur experiments were conducted on an NVIDIA DGX A100 server. The **GIBMS** module, which is a reusable component, requires **683 minutes** for training. For **AttrMask**, integrating the SLG module increases the training time from **512 minutes** (baseline) to **561 minutes**. We have included these details in Appendix C.1 in the revised version.\n\n\n > **Q7** In figure 3, you mention that SGL yields a more uniform distribution of tokens on the ZINC dataset, when the data has 95% of atoms with the 3 most common labels. Is a uniform distribution desired here?\n \nThank you for this thoughtful question. As noted in the paper, the ZINC dataset exhibits a highly skewed atom distribution, with **95% of atoms belonging to the three most frequent types** (e.g., C, N, O). This extreme imbalance can make the reconstruction task overly simple and limit the model’s ability to learn to distinguish rare atom types.\n\nOur goal in introducing the **Soft Label Generator (SLG)** is **not to enforce a perfectly uniform distribution**, but rather to **mitigate class imbalance**, thereby improving the model’s generalization and robustness. Specifically, SLG:\n\n- Maps each reconstruction target from a single hard label to a **soft-label distribution** over learnable prototypes, increasing the complexity of the reconstruction task and improving context modeling;\n    \n- Reduces the tendency to overfit dominant classes and encourages the model to better recognize long-tail (rare) classes;\n    \n- Provides a **more diverse expression space** at the prototype level (as illustrated in Figure 3(c)), which compensates for the limited expressiveness of a small token vocabulary.\n    \n\nThis diversity manifests as high prototype perplexity in the early training stages (i.e., more uniform prototype usage), which gradually converges as the model learns effective representations.\n\nTherefore, the “more uniform token usage” brought by SLG should be understood as a **mechanism to dynamically adjust reconstruction difficulty and increase representational diversity**, rather than an attempt to impose a strictly balanced distribution. We have further clarified this point in the revised manuscript.\n\n\n___\n\nWe hope these explanations and corrections satisfactorily address your concerns. Thank you again for your thoughtful review."}, {"review_id": "yHoGUzYJGx", "reviewer": "Reviewer_MrsU", "summary": "This paper proposes improvements to Masked AutoEncoder-based (MAE) models applied to molecular graph tasks. While MAE is trained through masked reconstruction, its performance is highly sensitive to how the mask is selected—both in terms of region and ratio. Moreover, challenges remain regarding the informativeness of the masked regions and the uniqueness of reconstruction targets.\n\nTo address these issues, the authors propose a masking strategy based on the Information Bottleneck (IB) principle. Since IB identifies salient regions per sample based on compression objectives, it offers a principled approach to mitigate the above concerns. The paper formulates a self-supervised learning framework that incorporates Graph IB and further introduces an objective function that evaluates token labels using soft-labels rather than hard-labels.\n\nThe main experiments are conducted on MoleculeNet for property prediction tasks, demonstrating consistent performance improvements across three variants of Molecular Graph AutoEncoders (MGAEs) when trained with the proposed method.", "questions": "- In Appendix Table 5 (DTA regression experiments), it appears that models such as MolBERT and AttrMask were not evaluated using the proposed method. Was there any practical difficulty in applying the method to these models? If so, this could constitute a limitation of the proposed approach and should be mentioned explicitly.\n- One of the key advantages of the proposed method is its reduced reliance on arbitrary mask selection. Conversely, has there been any experimental verification of how much performance fluctuates in existing methods when masks are chosen arbitrarily? If such analysis exists, it would strengthen the motivation for your approach and help quantify its benefits more precisely.", "limitations": "Yes (As far as I understand)", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Pros\n- The proposed learning strategy is broadly applicable, regardless of the specific MGAE variant, indicating good generalizability.\n- Using IB to extract sample-specific salient regions and reflect them in the mask strategy is a reasonable and theoretically grounded approach.\n- According to the Appendix, the performance gains appear robust across different hyperparameter settings, which is a valuable practical property.\n\nCons\n- I have some concerns about the performance improvements reported in Table 1. Given the magnitude of the variance, can it be confidently stated that the proposed method yields a statistically significant \"boost\" in performance?\n\nIn summary, I like the proposed methodology, but am not confident in the strength of experimental supports.", "was_revised": true, "final_justification": "Feedbacks effectively resolve my concerns. So I raised the score.", "rebuttal": "Thank you very much for your valuable feedback and suggestions. We carefully address each point below.\n\n---\n\n> **Q1.** _I have some concerns about the performance improvements reported in Table 1. Given the magnitude of the variance, can it be confidently stated that the proposed method yields a statistically significant \"boost\" in performance?_\n\nThank you for this question. We have repeated all experiments **five times** and report the results below:\n\n||Run1|Run2|Run3|Run4|Run5|\n|---|---|---|---|---|---|\n|AttrMask|70.8|70.1|70.9|70.9|70.7|\n|AttrMask‑DyCC|73.7|73.7|73.8|73.6|73.8|\n|Mole‑BERT|72.2|72.1|72.1|72.4|72.3|\n|Mole‑BERT‑DyCC|73.8|73.8|73.9|73.7|73.8|\n|SimSGT|73.0|73.2|72.9|73.0|73.1|\n|SimSGT‑DyCC|74.1|74.3|74.2|74.2|74.0|\n\nWe also conducted a **paired t‑test** between baseline and DyCC‑integrated models:\n\n|Comparison|p‑value|Mean Baseline|Mean DyCC|\n|---|---|---|---|\n|AttrMask vs AttrMask‑DyCC|0.000038|70.68|73.72|\n|Mole‑BERT vs Mole‑BERT‑DyCC|0.000052|72.22|73.80|\n|SimSGT vs SimSGT‑DyCC|0.000072|73.04|74.16|\n\nAs shown above, the performance improvements brought by DyCC are **statistically significant (p < 0.001)**, even though the absolute improvements range from ~1.5% to ~4.3%. This confirms that the observed gains are consistent and unlikely to be caused by random fluctuations.\n\n---\n\n> **Q2** _In Appendix Table 5 (DTA regression experiments), it appears that models such as Mole‑BERT and AttrMask were not evaluated using the proposed method. Was there any practical difficulty in applying the method to these models? If so, this could constitute a limitation of the proposed approach and should be mentioned explicitly._\n\nThank you for pointing this out. We have now evaluated **Mole‑BERT‑DyCC** and **AttrMask‑DyCC** on the DTA regression tasks and present the results below:\n\n||ESOL|Lipo|Malaria|CEP|Davis|KIBA|\n|---|---|---|---|---|---|---|\n|Mole‑BERT|1.192|0.706|1.117|1.078|0.277|0.210|\n|Mole‑BERT‑DyCC|1.094|0.692|1.108|1.003|0.279|0.198|\n|AttrMask|1.112|0.730|1.119|1.256|0.291|0.203|\n|AttrMask‑DyCC|1.104|0.701|1.104|1.006|0.275|0.192|\n\nThe results demonstrate that, after incorporating DyCC, both Mole‑BERT‑DyCC and AttrMask‑DyCC consistently achieve better performance. This further validates the **generality** of our approach and shows that DyCC can be applied to existing masked graph autoencoder (MGAE) methods without practical difficulty.\n\n---\n\n> **Q3** _One of the key advantages of the proposed method is its reduced reliance on arbitrary mask selection. Conversely, has there been any experimental verification of how much performance fluctuates in existing methods when masks are chosen arbitrarily? If such analysis exists, it would strengthen the motivation for your approach and help quantify its benefits more precisely._\n\nWe appreciate this insightful suggestion. Quantifying the sensitivity of existing MGAE methods to arbitrary mask choices indeed strengthens the motivation for DyCC.\n\nAs shown in **Fig. 1(a)** of the paper, we varied the _mask ratio_ for standard MGAEs (AttrMask, Mole‑BERT, GraphMAE) and observed a non‑monotonic trend: performance first improves and then degrades as the ratio increases. This confirms that fixed, arbitrarily chosen mask settings can **significantly affect result**\n\nFor instance, Mole‑BERT (MAM) reports the performance under different masking ratios, as shown below. The results clearly indicate that the choice of masking ratio has a non‑trivial impact on performance, with **0.20 yielding the best result**:\n\n|Masking ratio|0.10|0.15|0.20|0.25|0.30|\n|---|---|---|---|---|---|\n|MAM|71.64|72.16|**72.21**|71.51|71.36|\n\n\n\n\nSimilarly, **Fig. 1(b)** varies the tokenizer difficulty (K‑hop). The results indicate that “harder” targets do not uniformly help, implying that performance fluctuations stem from the **proxy‑task specification** rather than the model capacity itself.\n\nThese findings highlight the issue our **dynamic masking mechanism (GIBMS)** aims to resolve: it adaptively adjusts mask selection and target generation during training, eliminating instability and suboptimality caused by arbitrary fixed settings.\n\n\n---\n\nWe hope these explanations and corrections satisfactorily address your concerns. Thank you again for your thoughtful review."}]

Author rebuttal:

Meta-review: The paper proposes Dynamic and Chemical Constraints (DyCC) to enhance masked graph autoencoders for molecular representation learning, introducing a graph information bottleneck-based masking strategy (GIBMS) and a soft label generator (SLG) to address limitations in fixed masking ratios and reconstruction objectives. Reviewers praised the method's theoretical grounding, model-agnostic design, and consistent performance improvements across multiple baselines and tasks, though one noted modest gains on saturated benchmarks and initially leaned toward rejection before discussions. Strengths include the extension of information bottleneck theory to unsupervised settings, thorough empirical validation on MoleculeNet datasets, and interpretability of atom importance scores, with rebuttals effectively addressing concerns about baselines, computational overhead, and statistical significance. Weaknesses, such as reliance on older baselines and minor presentation issues, were mitigated through additional experiments and clarifications in the rebuttal phase. Overall, the contributions are solid and impactful for molecular pretraining, warranting acceptance at NeurIPS 2025.
Decision: Accept (poster)