FragFormer: A Fragment-based Representation Learning Framework for Molecular Property Prediction
Authors: Jiaxi Wang
Yaosen Min
Miao Li
Ji Wu
Venue: Accepted by TMLR ()

Abstract: Molecular representation learning is central to molecular property prediction, which is a vital component in drug discovery. Existing methods, which mainly focus on the atom-level molecular graphs, often find it challenging to directly model the relation between fragment (substructure) and function of molecules, largely due to insufficient fragment priors. In this work, we propose a molecular self-supervised learning framework \textbf{FragFormer},  which aims to learn the representation of fragments and their contextual relationships. Given the prior that an atom can be part of multiple functional groups, we develop $k$-\textbf{D}egree \textbf{Ove}rlapping fragmentation (\textbf{DOVE}), which generates overlapping fragment graph by employing the iterative line graph. Besides, DOVE can preserve the connection information during the fragmentation phase compared to non-overlapping fragmentation. In the pre-training stage, we design a \textit{nested masked fragment prediction} objective, to capture the hierarchical nature of fragments, namely that larger fragments can encompass multiple smaller ones. Based on FragFormer, we introduce a simple yet efficient \textit{fragment-level} interpretation method \textbf{FragCAM} for the molecular property prediction results with greater accuracy. Moreover, thanks to the fragment modeling, our model is more capable of processing large molecule, such as peptides, and capturing the long-range interactions inside molecules. Our approach achieves state-of-the-art (SOTA) performance on eight out of eleven molecular property prediction datasets on PharmaBench. On long-range biological benchmark with peptide data, FragFormer can beat strong baselines by a clear margin, which shows the model's potential to generalize to larger molecules. Finally, we demonstrate that our model can effectively identify decisive fragments for prediction results on a real-world dataset\footnote{Our code is available at \url{https://github.com/wjxts/FragFormer/}}.

Published in Transactions on Machine Learning Research (04/2025)
Framerrmer: A Fragment-based Representation Learning Framework for Molecular Property Prediction
Jiaxi Wang Deportment of Electronic Engineering, Tsinghua University
wjr300@mall..siignuuaeee...
Yaosen Min Microsoft Research Asia
yaosenminn@mmsccroosff..cmmm
Miao Li" Deportment of Electronic Engineering, Tsinghua University
Banooki@ttsingguu.ed...o
Ji Wu*
wuji ee we@tiinguuaaaaa. edu.cn
Department of Electronic Enginecring, Tsinghua University College of AI, Tsinghua University Beijing National Research Center for Information Science and Technology
Reviewed on OpenReview: https: // openreview. net/ forum? id= 9aiuB3kIjd
Abstract
Molecular representation learning is central to molecular property prediction, which is vital component in drug discovery. Existing methods, which mainly focus on the atom-level molecular graphs, often find it challenging to directly model the relation between fragment (substructure) and function of molecules, largely due to insufficient fragment, priors. In this work, we propose molecular self-supervised learning framework FragFormer, which aims to learn the representation of fragments and their contextual relationships. Given the prior that an atom can be part of multiple functional groups, we develop k- k-Degree Overlapping fragmentation (DOVE), which generates overlapping fragment graph by employing the iterative line graph. Besides, DOVE can preserve the connection information during the fragmentation phase compared to non-overlapping fragmentation. In the pre-training stage, we design a nested masked fragment prediction objective, to capture the hierarchical nature of fragments, namely that larger fragments can encompass multiple smaller ones. Based FragFormer, we introduce simple yet efficient fragment-leuel interpretation method FragCAM for the molecular property prediction results with greater accuracy. Moreover, thanks to the fragment modeling, our model is more capable of processing large molecule, such as peptides, and capturing the long-range interactions inside molecules. Our approach achieves state-of-the-art (SOTA) performance on eight out eleven molecular property prediction datasets on PharmaBench. On long-range biological benchmark with peptide data, FragFormer can beat strong baselines by a clear margin, which shows the model's potential to generalize to larger molecules. Finally, we demonstrate that our model can effectively identify decisive fragments for prediction results real-world datasct
1 Introduction
Finding molecules with the desired properties one of the biggest challenges in drug discovery (Dickson & Gagnon 2004). Using traditional wet-lab experiments to assess molecular properties is time-consuming, labor-
intensive, and costly (Mullard 2014 Simoens Huys 2021 Wouters et al, 2020 Machine learning models
*Corresponding authors.
!Our code available https://githbb. http://shhhccceeeeete//////rrrre

---

Published in Transactions on Machine Learning Research (04/2025)
(Wllters & Barzilay 2020 Wieder et al. 2020). However, the amount of data for labeled molecular properties is often
have been developed to predict molecular properties, which can significantly reduce the cost and time
limited (Guo et al.) 2021) compared with the large space of plarmaccooogiall--reeeaat molecules (Virshup
ct al. 2013), making the generalization performance of machine learning models unsatisfactory. Fortunately, a
wealth I of unlabeled molecular data is available (Gaulton et al. 2017: Kim et al. 2022 Sterling & Trwin 2015
Therefore, many studies investigate self-supervieed learning (SSL. methods for molecular representation
learning and achieve better results than supervised learning methods (Rong et al., 2020 You et al.) 2020
Zhou et al. 2023 Li et al. 2023). These SSL methods primarily focus on the atom-level molecular graphs. In pharmaceutical chemistry, fragments serve as functional groups within molecules, fundamentally influencing their properties (Guvench 2016). Methods based on aton-level molecular graphs often suffer from over- smoothing issue (Rusch el al. 2023) and lack fragment priors (Jiang et al.) 2023), making it difficult to caplure the relation between fragments and the properties of molecules. Several works develop fragment biased GNNs (Zhu et al.) 2023: Bouritsas et al.) 2023 Wollschläger et al., 2024 from a thoorctical perspoctive. Nevertheless, these models primarily aim passing certain Weisfeiler & Leman (WL) Lest (Zhang al. 2023) and are not designed for molecular property prediction. There are also studies (Zhang et al. 2021b Luong & Singh 2023 Jiang et al. 2023) that explore the use of fragment-level molecular graphs to develop deep learning models for predicting molecular properties and achieve encouraging results. However, their molecular fragmentation methods lack overlap between fragments, overlooking the fact that a single atom can belong to different functional groups (Merlot et al. 2003). Moreover, non-overlapping fragmentation hinders their fragment graphs from distinguishing the connections between adjacent fragments. These limitations impede the model's expressivity and generalization ability. Beyond prediction accuracy, it also important to understand the model's decision-making process. Existing interpretation methods for molecular property prediction are either atom-level (Yuan et al. 2023 or slow to achieve fragment-level interpretation (Yuan ct al. 2021). In this work, we propose TragFormen fragment-based based molecular representation learning framework to tackle the above challenges. Our contributions can be summarized as follows:
Given the prior that an atom can be part of multiple functional groups, we propose novel k- k-Degree Overlapping fragmentation (DOVE) method. DOVE can generate overlapping fragment graph by utilizing the ilerative line graph transformation of the molecular graph and exist non-overlapping fragmentation methods, Besides, DOVE can retain the connection information in the fragmentation step compared to non-overlapping fragmentation.
Taking the hicrarchical nature of fragment into considcration, we design a nested masked fragment prediction self-supervised objective to model this prior.
Our FragFormer consistently surpasses previous fragment-based based models and achieves SOTA perfor- mance on eight out of eleven molecular property prediction datasets on CharmaBenchh
Based on the fragment modeling, our model is more capable of processing large molecules, outperforms strong baselines on long-range biological benchmark with peptide data.
Building on fragment modeling, we introduce a simple fragment-level interpretation technique, called FragCAM, for the prediction outcomes. Our method achieves greater accuracy and faster speed on a real-world molecular property prediction dataset with labeled decisive fragments.
2
Related Works
2.1
Molecular Representation Learning
Obtaining good molccular representation is crucial step for accurately predicting molecular proper-
ties
TTktchenko 2020h. Conventional quantitative structure activily relationship (QSAR) methods (Hansch) 1962 Tropsha 2010) formulate a variety of handcrafted molecular descriptors to serve as fixed features
et al.
for molccules. These features can be used as input for various machine learning models, such as SVM (Cortes)
& Vapnik 1995). XGBoost (Chen & Guestrin 2016), etc. Such fixed features can reflect simple substructures and physicochemical properties of molecules, but may not be able to capture the necessary and sophisticated
2

---

Published in Transactions on Machine Learning Research (04/2025)
structure feature of molecules for a specific task. To adaptively learn the molecular features, supervised deep learning methods are proposed to learn the molecular representat ion from the molecular graph or SMILES [Weininger] 1988) string directly with labeled data (Chen et al. 2018). These methods can effectively capture the relevant features of molocular properties and outperform traditional methods when provided with sulficient. labeled data (Masters el al. 2023 However, the performance of these methods is constrained by the amount of labeled data, which is usually limited in practice (Guo et al. 2021). Owwng to the success of self-supervised learning (SSL) in the field of computer vision (CV) (He et al. 2022; Chen al. 2020b), natural language processing (NLP) (Devlin et al. 2019) and audio analysis (Baevski et al., 2020: Hsu et al. 2021 SSL methods have been proposed for learning molecular representation by harnessing the abundant unlabeled molecular data. These methods primarily fall into two categories: those based on masked autoencoders, and those that employ contrastive learning. MolBERT (Fabian et al. 2020) and MolGPT (Bagal el al. 2022) use the string representation of molecules and predict the masked tokens in the SMILES string in the pre- tre-trainin stage. GraphLoG (Xu et al. 2021). GROVER (Rong et al. 2020) and GEM (Fang et al. (021) treat the molecule as 2D graph, and predict the masked nodes in the graph with graph neural networks. KPGT (Li et al. 2023) further improves the performance by modeling the global information of the molecular graph with transformer style architecture. Contrastive beaniig-bassdd methods depend on constructing different views of the same molecule. GraphCL (You et al. 2020 2021) achieves this by randomly dropping, perturbing, and masking parts of the molecular graph. MoleculeSTM (Tiu et al. 2023 utilizes text descriptions of the molecule as an alternative view, while GraphFP (Luong & Singh 2023) and Holi-Mol (Kim et al. 2024) employ the fragment graph of the molecule for this purpose. Although these methods have advanced the performance of molecular property prediction, they mainly focus on the atom-level molecular graphs, and often struggle to directly model the relation between fragments and molecular properties due to the lack of fragment prior, restricting their interpretability and generalization. In this work, we propose novel overlapping fragmentation method, along with a nested masked fragment prediction pre-training task, to effectively learn the representation of fragments and their contextual relationships.
2.2 Fragment-based Molecular Learning
Fragment-basse drug discovery (FBDD) is an emerging method in drug discovery (Murray & Rees 2009
Unlike high-throughput screening (Macarron et al. 2011), FBDD focuses on the interactions between small fragments and target proteins, and extends the fragments to larger molecules with higher binding affinity. FBDD can explore the chemical space more efficiently and reduce the cost of drug discovery with higher success rate (Murray & Roes 2009). Motivated by the success of FBDD, rescarchers have proposed tragment-based methods in molecular learning lasks, mostly on molecule generation. Compared with atom-based molecule generation, fragment- based methods can generate molecules more efficiently and with more validity. Most fragment-based methods in moleculo generation employ a variational autoencoder (VAE) framework (Voloboev)
2024 Kingma& & Welling 2014 Jin el al. 2018 2020; Kong el al., 2022: Yu Yu 2024 J. JT VAE decomposes the molecule into a tree structure (Rarey & Dixon 1998) and generates the molecule by a tree expansion. Hier constructs fragment with multiple properties and subsequently expand this fragment. PS VAE employs data-driven method to build fragment vocabulary and generate molecules one fragment time, which emulates the subword vocabulary construction and lokenization process in NI.P (Sennrich) et al. 2016). Recently, there are some studies that explore fragment- based methods for molecular property prediction (Zhang et al.) 2021a b Luong & Singh 2023 Jiang et al., 2023 Kim et al. 2024). Fragment-based
methods take advantage of the prior that fragments are the functional groups within molecules (Guvench
2016), which are basic building blocks pharmaceutical chemistry. MGSSL (Zhang et al.) 2021a) utilizes
predefined motifs as supervision in its self-supervised learning task. FragGAT (Zhang et al.] 2021b) segments the molecular graph in different ways, processes each segment with an Attentive FP network (Xiong et al.
2020), and aggregates the segment embedding to predict the molecular properties. GraphFP Luong & Singh 2023) employs the fragment graph of the molccule as an alternative view in contrastive learning to enhance alom-level molecular modeling. Holi-Mol (iim et al.) 2024) further utilizes multiple views of fragmentation to enrich the representation. PharmHGT (Jiang et al. 2023) integrates both fragment and atom graphs to model the molecular propertics. Although existing fragment- based methods have demonstrated encouraging results in molecular property prediction and heightened the interpretability of the model, they have yet to surpass atom-level molecular graph based methods (Chen et al. 2024 We postulate that the reason is
3

---

Published in Transactions on Machine Learning Research (04/2025)
their fragmentation methods primarily focus on the non-overlapping fragmentation, which cannot model the fact that a single atom can contribute to different functional groups (Merlot et al. 2003) Besides, the non-overlapping fragmentation results in connection information loss between fragments. In the fragment graph with non-overlapping fragmentation, an edge cannot specify how two fragments are connected, as any two atoms within each fragment may serve as the connection points. In this work, we propose a novel overlapping fragmentation method with tunable overlapping degree and learnable fragment vocabulary. Our method allows single atom to participate in multiple fragments and the overlapped atoms maintain the connection information between fragments.
2.3 Interpretation Methods for Molecular Property Prediction
Identifying the critical fragments for prediction results is essential for applying machine learning models in real-world scenarios for molecular property prediction (Proietti et al., 2024). Existing interpretation methods can be divided into two categories: atom-level and tragment-level. Atom-level interpretation methods aim to identify the important atoms for prediction results. Gradient-based altribution methods from CV, such as class activation maps (CAM) (Zhou et al.) 2016), GradCAM (Selvaraju et al. 20200,
GradInput (Shrikumar et al. 2017) and Integrated Gradients (IG) (Sundararajan et al. 2017). can be directly applied to score the importance of atoms in molecular property prediction tasks, Rao et al. (2021) builds a benchmark DrugXAT for interpreting molecular property prediction models and shows these methods can achieve good performance on synthetic datasets, but poor performance on real-world datasets. They also find that GradInput and IG perform better than other gradient-based methods. Many interpretation methods
2020 Yuan et al. have been specifically developed for graph neural networks (Ying al., 2019 Luo al.
2021). GNNExplainer (Ying ct al. 2019) learns masks for node features and edges by maximizing the mutual information between the masked subgraph and the prediction. PGExplainer (Luo et al. 2020) utilizes a
deep network to model the explanation generation process. While atom-level interpretation methods can highlight key atoms relevant to the predictions, they do not necessarily guarantee fragment-level explanations. SubgraphX (Yuan et al., 2021) aims to identify the most important connected subgraph for prediction results using Monte Carlo Tree Search (MCTS), offering better fragment-level interpretation compared to CNNExplainer and PGExplainer. However, it involves greater computational overhead. Existing methods either fail to guarantee fragment explanations or are slow in producing them. In this work, we propose a simple but efficient fragment-level interprelation method based on fragment modeling. Our method achieves better performance with faster speed on a real-world mutagenicity dataset.
3 Methods
In this section, we introduce our self-supervised Icarning framework, FragFormer, designed for molecular properly prediction based on fragment modeling. The overall pipeline is depicted in Figure [ In the following subsections, we first propose a novel overlapping fragmentation method and derive the fragment graph. Next, we introduce our model architecture, which takes fragment graph as input and produces the represcntation for both fragments and the whole molecule. Finally, we describe the nested masked fragment prediction pre-training, which aims to capture the hierarchical nature of fragments and their contextual relationships.
3.1 k: Degree Overlapping Fragmentation (DOVE)
Existing fragmentation methods mainly focus on non-overlapping fragmentation, which partition the molecule into disjoint atom sets (Lewell al. 1998 Degen et al., 2008 Kong et al.) 2022). The non-overlapping fragmentation methods have two drawbacks. First, one atom can only belong to one fragment, violating the fact that a single atom can be part of multiple functional groups (Merlot et al. 2003). Second, the induced fragment graph loses certain topological information from the original molccular graph. Such fragment graph cannot capture how two fragments are connected (see Appendix the above issues, we propose a novel method of overlapping fragmentation with learnable fragment library. It permits a single atom to participate multiple fragments. The overlapping atoms can retain pivotal connection details between fragments, allowing for effective assembly and reducing connection information loss during the transformation from molecular graph to fragment graph. Our method is based the iterative
B for an illustration). To solve
4

---

Published in Transactions on Machine Learning Research (04/2025)
Sample
DOVE
A Subset
Unlabeled Molecule Dataset
Build Fragment Library
Vocabulary
Fragmentation
Pre-training
xL
Attention Layer
FFN
Nested MFP
Layer
Fragment Graph
Finetuning
Pre-trained Model
Pooling Prediction
Downstream Task
Fragment Graph
Interpretation (FragCAM)
Figure 1: Pipeline of the FragFormer. "FFN": feed forward network. 'MFP': masked fragment prediction. represents our core design in FragFormer. We propose k-degree overlapping fragmentation (DOVE) to generate overlapping fragment graph, and nested MFP self-supervised learning to capture the hierarchical nature of fragments. FragCAM can effectively provide decisive fragments for prediction results.
k-order Line Graph L*())
0-overlap Fragment Graph in LYG))
l-overlap Fragment Graph in
Graph
Line Graph Transform
Non-overlap Fragmentation
O-O
Induced Fragmentation
l=1
Fragment Graph
Figure 2: Pipeline the k-degree overlapping fragmentation (DOVE) with 1.
line graph transformation of the molecular graph and can take advantage of the existing non-overlapping fragmentation methods. The pipcline is shown in Figure 2 The main idea is that the non-overlapping fragmentation of the k-order line graph be transformed to the k-degree overlapping fragmentation of the original molecular gruph. We first formally define k-degree overlapping fragmentation and k-order line graph. Then, we show that the k-degree overlapping fragmentation of the molecular graph can be obtained by the non-overlapping fragmentation of the k-order line graph. We assume all the graphs G are undirected in the following description.
Definition (Fragmentation of graph). Given graph G (V, P= (Vi) A m we define F == (P, fragmentation (Iragment graph) of G if

---

Published in Transactions on Machine Learning Research (04/2025)
1. P is vertex cover of G, Lc, U - V. P also represents the nodes in fragment graph.
2. vi E f1,2,...,,, m). G [V] is a connected subgraph. G [Vi] represents the subgraph restricted to vertex set V₁.
Here, A symbolizes the adjacency matrix for the fragment graph, indicating whether two fragments are neighbors. This adjacency is flexible and can be adjusted based on specific requirements.
Definition (k-degree overlapping fragmentation). Given graph G (V, ; P (Vi A Rmxm we call fragmentation F (P, A) a k-degree overlapping fragmentation if:
Vi.j (1,2, (1.2,.....
i i j, if Ag i.., Vi and Vy are neighbors in G, then IV k .m) ..,
Next, we define k-order line graph and show how it can be used to generate the k-degree overlapping fragmentation of the original molecular graph.
Definition 3 (Line graph transformation). Given a graph G = (V, the line graph L(G) is an undirected graph whose vertices correspond to the edges of G and two vertices are connected by an edge if the corresponding edges in G share a common vertex.
We assume is connected and leave the discussion for disconnected graph in Appendix Definition 4 (k-order line graph). Let L"(G) G. For integer k 2 1, the k-order line graph L*(G) is defined recursively (+) = L(L
Each vertex in I (G) corresponds to a (k: + -ssiee connected subgraph in G. We denote its node set by C(v).
Definition 5 (Standard 0-degree overlapping fragmentation). Given a graph G = (V, E), P= (V))
A E Rmxm we call a fragmentation F (P.A) as a standard O-degree overlapping fragmentation if:
V i.j E f1,2, m) ( j, we have Vú and V, are disjoint, i., VV; Mynv, = 0.
2. Aij 1 if there exists node os E V, and by V,, such that es and vy are neighbors in G, otherwisc Aij = 0.
Most commonly used fragmentation methods are standard O-degree overlapping fragmentation, such as principal subgraph mining (Kong et al. 2022), BRICS (Degen el al. 2008) and Recap (Lewell et. al. 1998).
Definition 6 (Induced fragmentation). Given a graph G = EV and a fragmentation F = =(P.A (P.A) of T (G), where = ( A E Rmxxm the induced fragmentation of is defined as F' (Q, A), where Q = (V) V42 C(v).
Net,
Theorem 1. Given a graph -V,VE), the induced fragmentation of standard O-degree overlapping fragmentation of (G) a k-degree overlapping frogmentation of G.
The proof can be found in Appendix Al By Theorem ! we first transform the molecular graph to the k-order line graph, then use existing graph fragmentation method to get the standard O-degree overlapping fragmentation of (G),,, and finally get the k-degree overlapping fragmentation of G by the induced fragmentation. The overlapping degree can be controlled by adjusting the order k of the line graph. We apply principal subgraph mining (Kong et al. 2022 as our standard O-degree overlapping fragmentation method, which can generate a succinct and fruitful fragment vocabulary.
3.2 Model Architecture of FragFormer
Assume the fragmentation of the molecular graph is F= (P, A), where P (Vi) is the fragment set and A is the fragment adjacent matrix. The architecture of FragFormer designed to model the contextual relationships among fragments and learn the representation of those fragments. The model architecture of FragFormer consists of three main components: subgraph encoder, fragment-level graph transformer and knowledge-aware fusion. The overall architecture is shown in Figure

---

Published in Transactions on Machine Learning Research (04/2025)
X
Am
W
+
W
K
Add & Norm
Knowledge Fusion Layer
FFN
Wy
KNz
GRU
Add Norm
11
GRU
Fragment-level Graph Transformer
hg
Pooling & Prediction Head
Path Attention A
fsub
Ita
Subgraph Encoder
h4 Fragment Graph
Distance Attention A
Figure 3: Model archilecture of FragFormer.
Subgraph Encoder
Given a fragment V; in the overlapping fragmentation, we first obtain the embedding of fragment V, by using a subgraph encoder faub:
h, = frao
(h:)
m serve as the node features in the fragment graph.
Fragment-level Graph Transformer
Using fragment embeddings [h:) and A, apply transformer-based architecture (Vaswani et al. 2017 that operates on the fragment graph to model contextual dependency between fragments and learn a graph-level representation for molecular property prediction.
Fragment-level graph transformer composed L graph transformer layers, each which consists of a self-attention mechanism and feed-forward module. We follow the design self-attention mechanism et al. (0023); Ying et al. (2021) Shi al. (2022), which includes path attention and distance attention as reflection of graph topology.
Li]
Specifically, given the fragment graph and fragment features H' E Rdxm at the 1-th layer, where d is the dimension of the feature and m is the number of fragments, the self attention is calculated as:
Q = W K = -' == wVH,
KTQ
softmax
+ + + AP) Vd
H +++ LayerNorm (VA + H')
Wº, WK and WV pdxd are projection weight matrices for query (Q), key (K) and value matrix V. A and AP are distance attention and path attention. Assume the distance between fragment i and fragment in fragment graph is Np, and the shortest path between fragment. i and fragment, j is Pij (io.... .......,))
then Ad and AP are computed as:
A
CN2:
N
A2 =
W
N
k=0
7

---

Published in Transactions on Machine Learning Research (04/2025)
where ON, is learnable scalar, W Rdx is learnable weight matrix. The feed forward module is defined 8s:
H = LayerNorm (FEN(H+*)) + H++)
where FFN is two-layer MLP with ReLU activation Agarap 2018). The initial feature H is the fragment embedding [h:) The output feature H is the final fragment representation. At last, we apply a global pooling operation on H to obtain the graph-level representation hg.
Knowledge Fusion Layer
Incorporating fixed substructure-rrlatee features, such as extended connectivity fingerprints (ECFP) RRogers & Hahn 2010), are beneficial for molecular representation learning (Li et al. 2023 Rong et al. 2020 We refer those fixed features as knowledge vectors. Here, we propose a knowledge fusion layer incorporate the knowledge vectors into the fragment representation learning. We integrate the knowledge vector k with the fragment representation h; through GRU unit (Xiong et al. 2020 right after the subgraph encoder:
h; f- GRU h,).
If there are multiple knowledge vectors (k.) we sequentially integrate them with the fragment representa- tion, as illustrated in the bottom left corner of Figure 3
3.3 Nested Masked Fragment Prediction Pre-training
To distill knowledge from massive unlabeled molecular data, is essential to pre-train the model with a self-supervised objective. Here, we propose a novel fragment-based pre-training task, which is designed to model the hierarchical nature of fragments. Several atoms can assemble into small fragments, and combination of these small fragments
constitute larger ones. Our design follows the philosophy of mask autocncoder (Devlin et al. 2019 He et al. 2022), where we mask the fragment nodes and predict the masked nodes based on the context. Instead of barely predicting the class label of the masked nodes, we predict the existence of all substructures in the vocabulary V for the masked nodes. We use a 0-1 vector y E RIM represent the existence of the substructure the vocabulary. //1 = the i-th substructure exists in the masked node, otherwise y O. We predict the existence of the substructure by using a linear classifier built the fragment representation h4 and binary cross entropy loss:
L
Σ ys log (sigmoid
+ (1 ys) log (1 sigmoid (w/h²))
3.4 Fragment-level Interpretation: FragCAM
Inspired by the class activation mapping (Zhou al. 2016 in CV, we combine with our fragment modeling and propose Fragment Class Activation Mapping (FragCAM) for explaining the prediction results with
fragments. Assune the final fragment representation is {he 1 Ji=1 and predict the molecular property with
a linear classifier w E Rd.
w SumPooling (b/))))
Then, the allribution of the i-th fragment to the prediction result is calculated as:
Attribution; = w wh Min
Max - Min
where Max and Min are the maximum and minimum value of {w bm respectively.
4
Experiments
4.1 Pre-training
Dataset We use two millions unlabeled molocular SMILES (Weininger 1988 from CHEMBL29 (Gaulton
et al. 2017) pre-training data. We use RDKit (Landrum 2016 generate the molecular graph from

---

Published in Transactions on Machine Learning Research (04/2025)
Table I: Results on PearmaBench. The best results are in bold. "t' means the higher the better, while "4' means the lower the better. FragFormer can achieve the SOTA preformance on eight out of cleven tasks. ":" indicates traditional QSAR method. "*" means the methods include a pre-training stage. Baselines labeled with "f' are fragment-based methods. The best performance of previous bascline methods (annotated with underlines) on each dataset mostly comes from methods based on atomic graphs.
Classfication (AUROC f)
Regression (RMSE L)
Model/Datasel /Dataset
AMES
BВB
CYP2C9
CYP2D6
CYP3A4
HLMC
MLMC
RLMC
LogD
PPB
Sol
RF$
Breiman 2003
0.761
0.731
18.471
18.041
16,540
0.813
0.987
0.958
1.249
0.201
0.918
XGBoost
Chen & Guestrin
2016
0.768
0.750
17.582
17.819
16.123
0.647
0.844
0.819
1.071
0.186
0.832
CMPN'V
Song aL
0.808
0.857
18:377
19.156
16.701
0.921
1.130
0.930
0.807
0.236
0.858
FPGNN
4122
0.858
0.923
16.933
17.611
15.606
0.604
0.774
0.716
0.838
0.179
0.747
DHINN
Song ct m.
2025
0.841
0.909
17.449
17.890
16.156
0.729
01922
0.915
0.912
0.235
0.828
KANO"
Fang el mJ
2123
0.965
0.915
17.350
17. 17.622
15.307
0.564
0.767
0.762
0.706
0.185
0.772
MPG'
21
0.569
0.923
17.417
17.527
14.376
0.541
0.723
0.685
0.758
0.170
0.758
Unimall
Zhon et al
20125
0.878
0.920
17.774
18.071
15,896
0.613
0.824
0.651
0.745
0.179
0.707
Traus-M
Luo et a]
20255
0.509
0.935
18,080
17.677
13.867
0.567
0.711
0.677
0.737
0.172
0.831
KPGT"
1125
0.880
0.935
17.036
16,8600
16.379
0.564
0.726
01881
0.728
0.172
1.221
GraphFpet
L Luong & Singh
2023
0.830
0.893
17.367
21.183
17:219
0,764
0.878
0.771
0.836
0.208
1.935
FraGAT'
Zhang ot al.
20/1
0.778
0.684
17.788
22.503
20.313
0.775
0.840
1.050
0.945
0.220
1.352
PharmHGT !
(Jiang et al
0.963
0.913
17.490
15.020
16.077
0.544
0.820
0.677
0.676
0.172
0.954
FragFormer-001
(ours)
0.868
0:920
18,091
17.096
17,036
0.624
0.875
0.619
0.774
0.190
0.968
FragFormer-1"
(ours)
0.889
0.928
16,855
14.425
15.894
0.514
0.702
0.596
0.667
0.157
0.895
FragFormer-2"1
(ours)
0.870
0.921
17.017
14.505
15,708
0.525
0.688
0.5-1
0.670
0.177
0.901
FragForman-3" (ours)
0.870
0.918
17.635
16.780
16.977
0.545
0.732
0.571
0.714
0.193
0.961
SMILES and extract the atom features with 137 dimension. The detailed description of atom featurization can be found in Appendix D.. We randomly sample 100k molecules to generate the fragment vocabulary by principal subgraph mining (Kong et al., 2022). We set the vocabulary size as 500 for the vocabulary constructon. We separately generate the library for different degree of overlapping fragmentation. The higher the degree of overlapping, the slower the vocabulary construction and molecule fragmentation. We select the degree of overlapping from {0, 1,2,33 } in our experiments. FragFormer with k-degree overlapping fragmentation is denoted as FragFormer-k. For the knowledge fusion layer, we extract ECFP (Rogers & Hahn 2010) with radius- and 1024 bils, MACCS (Durant et al. 20022), TorsionFP (Schulz-Gasch et al.j 2012 and physic-chemical descriptors (Yang et al. 2019 Xue & Bajorath 2000) as the knowledge vectors. We linearly transform the raw knowledge vectors to the same dimension as the fragment representation with a learnable projection for each kind of knowledge before the fusion.
Training Details We use L= 6 graph translormer layers in FragFormer with model dimension d d= 512. We use a two-layer Graph IsoMorphism Network (GIN) (Xu et al. 2019) with mean pooling and hidden dimension as the subgraph encoder Saub. We use batch size bs = 4096, learning rate ir = 2e - 4, and Adam optimizer (Kingma& & Baj 2015) with (B1, B2) = (0.9, 0.999) in the stochastic training. We apply linear learning rate decay scheduler with 5% warm-up steps. We totally train the model for 25k steps on NVIDIA 4090 GPUs The masked fragment ratio is set as 0.3. The attention dropout rate and feed forward dropout rate set as 0.1 in all fragment-level graph transformer layer. We adopt the multi-hcad attention tochnique with 8 heads in each layer (Vaswani et al. 2017). In the fine-tuning stage, we add a sum pooling on top of the pre- pre-trained model and apply a linear predictor at last. More detailed description can be found in Appendix D
4.2 Downstream Tasks
We test our method on molecular ADMET (Absorption, Distribution, Metabolism, Excretion, and Toxicity) property prediction benchmark PharmaBench (Niu et al. 2024), long-range biological benchmark (Dwivedi
et al. 2022b,, and interpretability benchmark DrugXAI for molecular property prediction [Rao et al. 2021).
PharmaBench PearhaBBnch (Niu et al. 2024) is a comprehensive benchmark for predieting molecular ADMET properties, featuring eleven distinct molecular property prediction tasks curated with the assistance of a multi-agent large language model system. The benchmark encompasses a total of 52,482 entries drawn from 14,401 bioassays, inchuding two classification tasks and nine regression tasks. The detailed profile of the tasks can be found in the Appendix D. We follow the same scaffold splitting setting (Yang et al. 2019
9

---

Published in Transactions on Machine Learning Research (04/2025)
in PharmaBench which divides the dataset into training and test sets with a ratio of 4:1. We compare our method with traditional QSAR methods, atom-based methods and fragment-based methods. For traditional QSAR methods, we use ECFP descriptors with radius 2 and 1024 bits. For classification and regression tasks, we use AUROC and root mean square error (RMSE) as evaluation metrics, respectively. We report the mean performance of three runs with different random seeds.
Long-range Biological Benchmark To assess the model's generalization preformance on large molecules and its capacity to capture long-range interactions, we evaluate it using the peptides-func datatset from the Long Range Graph Benchmark (Dwivedi et al., 2022b 2 which predicts 10 biological functions of peptide data, c.g., Antibacterial, Antiviral, cell-cell communication, and others. This benchmark is designed to test the model's ability to model long range interactions between atoms. We compare our method with both graph convolutional based methods (Kipf & Welling, 2017 Chen et al. |202aa] Hu et al.) 2020 Bresson & Laurent 2017), transformer- baasod methods (Kreuzer et al. 2021 and fragment- based methods (Luong & Singh 2023). Due to the large GPU memory cost for large molecule, reduce the model dimension d to 32 in this benchmark. We report the average precision (AP) for the benchmark with three different runs.
DrugXAI Apart from the performance on molecular property prediction, explainability is also crucial for the model to be deployed in real-world applications. We evaluate our interpretation method FragCAM on DrugXAI (Rao et al.) 2021). We choose two synthetic datasets and one real-world dataset from DrugXAI. Detailed descriptions of the datasets can be found in the Appendix C Two synthetic datasets predict the existence of 3MR and Benzene substructures in the molccule, respectively. The ground truth attribution fragment is the corresponding ructure The real-world dataset aims at predicting the ames mutagenicity. We use the 46 substructure alerts given by SSuskko et al. (2012) as ground truth fragments. Following DrugXAI, we use AUROC metric proposed by McCloskey et al. (2019) to mcasure the model preformance, which computes the macro average AUROC between the predicted attribution and the ground truth attribution across test molecules. To evaluate FragCAM, we convert the fragment attribution into atom attribution by assigning the attribution of the fragment to the atoms it contains. If one atom is present in multiple fragments, we assign it the maximum attribution from those fragments. We compare our method with both alom-bbaed and subgraph-based attribution approaches. For the atom-based methods, we adhere to GradCAM (Selvaraju 2017) to CMPNN (?), Velickovic et al. 2018
the baseline settings established in DrugXAL. We apply CAM (Zhou et al. 2016).
et al. 2020), GradIuput (Shrikumar et al. 2017), and IG (Sundararajan et al.,
GraphSAGE (Hamilton et al.) (017a)), GraphNet (Battaglia et al.J [2088), and GAT
We report the best results among the four atom-based attribution methods for each model. For subgraph-basee method, we use SubgraphX (Yuan et al., 202Tp d. In SubgraphX, we set the larget number of fragment nodes Nm from (3, 7, 10, 15), use 20 rollouts in MCTS, and apply 20 steps in shapley value estimation. We also report the fidelity and sparsity score (Pope) ct al. 2019 Yuan et al. (023)) for fragment attribution. The fidclity metric evaluates how accurately the explanations reflect the model's decision-making process. It removes the crucial substructures from the input graphs and analyze the variation in predictions. The sparsity metric quantifies the proportion of structures ignored by the explanation methods. Higher fidelity under similar or lower sparsity are preferred.
two-layer GCN (Kipf & Welling) 2017) with dimension
4.3 Results and Discussions
PharmaBench The model performance on PharmaBench is presented in Table 1 Firstly, it is noteworthy that FragFormer achieves the SOTA performance in eight out of eleven tasks. Secondly, it consistently outperforms fragment-based baselines (indicated with f) with clear margins across all tasks. Thirdly, FragFormer employing overlapping fragmentation (A: = 1 or A = 2) exhibits substantially superior performance in contrast to the non-overlapping fragmentation (k- = 0). Excessive overlapping (k 3) lead to performance degradation. We recommend using = 1 or & = for optimal results. These outcomes substantiate the efficiency of the FragFormer framework, particularly in context to its overlapping fragmentation feature. However, our method still falls short of atom-based pre-training methods on three tasks, which we aim to address in future work. We also find that for each task, the best method always involves pre-training stage, which highlights the importance of pre-training in molecular property prediction.
2There is another peptide dataset called peptides-struct in the benchmark. However, peptides-struct pays attention to predicting the geometric quantities of peptides, which is beyond the scope of our work.
10

---

Published in Transactions on Machine Learning Research (04/2025)
Table 2: Results on long-range biological benchmark. "w.o KF"; without knowledge fusion layer.
Model
# Params.
Test AP (1)
GCN
Kipf & Welling 2017
508k
0.5930=1.0023
GCNII
Chen et al.
2020a.
505k
0.5543=0.000
GINE (uu et al.
2020
476k
0.5498=0.0079
GatedGCN Bresson & Laurent
2017
509k
0.5864=0.0077
GatedGCN Bresson & Laurent
2017
+RWSE Dwivedi et al.
2022a
506k
0.6069+10.0038
GraphFP Luong & Singh
2023
2.5M
0.6267=0.0073
Transformer (Vaswani ct al.
2017
+LapPE Dwivedi
2020
488k
0.6326=0.0126
SAN Kreuzer et al. 2021
LapPE
Dwivedi al. 2020
493k
0.6384=0.0121
SAN Kreuzer et al.
2021
RWSE
Dwivedi et al.
2022a
500k
0.6439+0.0075
Former (w.o KF)
500k
0.6571+000104
FragFormer
2M
0.6693:1.0154
Table Performance (AUROC) results on DrugXAI. The best results are in bold. For CMPNN, GraphSAGE, GraphNet, and GAT, report the best results among CAM, GradCAM, GradInput, and TG.
Dataset/ Model
CMPNN
GraphSAGE
GraphNet
GAT
FragCAM
3MR
0.966
0.932
0.967
0.905
0.981
Benzene
0.906
0.859
0.934
0.867
0.935
Mutagenicity 0.742
0.742
0.759
0.683
0.793
Long-range Biological Benchmark We show the results the long-range biological benchmark in Table g FragFormer without knowledge fusion layer has the same amount of paramcters as the basclines and outperforms the best baseline by more than 1% in terms of test AP. Compared to another fragment-basee method GraphFP, FragFormer contains less parameters, but achieves a absolute improvement of 4.26% in test AP. These results demonstrate the effectiveness of FragFormer in processing large molecule and modeling long-range interactions in biological data.
Interpretability Benchmark DrugXAI The results DrugXAI displayed in Table 3 and Table 4 In Table 3 FragCAM can significantly outperform the based methods for baseline models 3MR and mutagenicity dataset. For Benzene, FragCAM can also achieve competitive performance among all models. Compared to SubgraphX, FragCAM demonstrates superior AUROC and fidelity while achieving significantly laster speeds on the mutagenicily dataset. We also visualize the FragCAM results on mutagenicity datasets in Figure 4 FragCAM can accurately locate the nitro group, diazene and epoxyethane group as alerts for ames mutagenicity. These results demonstrate the offectiveness of FragCAM explaining the prediction results with fragments.
4.4 Ablation Study
A Epoxyethane Group
In this section, use the PharmaBench dataset to analyze the impact of different module designs the model performance. We set & = in k-degree overlapping fragmentation for fair comparison. We define the normalized performance as the RMSE di- vided by the standard deviation of the labels for regression tasks. For classification tasks, is cal- cuated by 1 AUROC. A lower normalized value indicates better performance.
Nitro Group
Diazene
Figure 4: Interpretation results FragCAM on mu- lagenicity datasets. The green nodes are the ground truth alerts. The blue fragments are the fragments with the highest attribution score in FragCAM.
11

---

Published in Transactions on Machine Learning Research (04/2025)
Table 4: Performance comparison between FragCAM and SubgraphX on DrugXAL. The best results are bold. Nmi represents the target number of fragment nodes in SubgraphX.
SubgraphX
FragCAM
Nmi AUROC Fidelity(pparsit)
3
7
10
15
0.655
0.683
0.663
0.592
0.
0. 0.24(0.81) 52
(.29(0.58)
0.30(0.41)
0.28(0.21) 11
0.35 (0.72)
Time per sample (s)
28
17
0.024
1.0
1.0
Normalizer lized Performance
Normalized Performance
0.6
0.6
0.4
0.4
0.2
AMES
BBB
CYP2C9
CYP206
CYP3A4
HLMC
MLML
RLMC
Logb
PPB
Sol
AMES
BB8
CYP2C9
CYPIDG
CYP3A4
HLMC
MLML
RLMC
Logb
PPB
Sol
Datasets
Datasets
Nested MFP
Nested MFP
KF
KF
Figure 5: Left: Comparison between nested masked fragment prediction (w.) and cross entropy loss (w.o).
'MFP': masked fragment prediction. Right: Comparison between models w./ w.o knowledge fusion (KF).
Table 6: Comparison between different mask rates in pre-training.
Classdication (AUROC
Regrussion
(RMSE
Average Rank
Mask Rate
AMES
BBB
CYP2C9
CYP2D6
CYР3А1
HLMC
MLMC
RLMC
LogD
PPB
Sol
0.879
17.886
16.676
16.298
0.523
0.733
0.592
0.698
0.171
0.922
2.2
0.3
0,889
0.927
16.855
14.425
15.894
0.514
0.702
0.396
0.667
0.157
0.895
1.1
0.5
0.868
0.919
17.027
16.627
16.379
0.627
0.738
0.621
0.689
0.180
0.924
2.7
Effectiveness of Nested Masked Fragment Predictions To evaluate the effect of the nested masked fragment prediction criterion, we pre-train the model with cross entropy loss which only predicts the identity of the masked fragment. The finetuning procedure remains unchanged. The results are illustrated in Figure (left). We that the nested masked fragment prediction consistently outperforms the cross-entropy loss, often by substantial margin, particularly for CYP2C9, CYP2D6, MLMC, RLMC, and PPB datascts.
Impact of Knowledge Fusion Layer We compare models with and without knowledge fusion layer. The results, displayed in Figure 5 (right),, reveal that the model utilizing knowledge fusion layer outperforms its counterpart on most datasets except for HLMC, demonstrating the efficacy of our knowledge fusion module.
How Vocabulary Size Affects Performance? To evaluate the influcnce of vocabulary size, we vary the fragment vocabulary size to 200, 500, and 1000. The results are presented Table 5 Notably, each vocabulary size achieves the best performance on at least one dataset. On average, a vocabulary size of 500 yields the best overall performance w.r.t average rank. If the vocabulary size is too small, it may not be able to capture the diversity of fragments, whereas an overly large vocabulary introduce redundaut fragments.
Effect of Mask Rate in Pre-training We investigate the effect of the mask rate pro-training by adjusting it 0.1, 0.3, and 0.5. The results, shown in Table D indicate that mask rate of 0.3 provides the best performance across nearly all datasets. A mask rate that is too high can make the pre-training task too
12

---

Published in Transactions on Machine Learning Research (04/2025)
Table 5: Comparison between different fragment vocabulary sizes.
Classfication (AUROC 1)
Regression (RMSE L)
Average Rank
Vorab Size
AMES
BHB
CYP2C9
CYP206
CYP3A4
HLMCC
MLMC
RLMC
Log!)
PPH
8ol
200
0.855
0.923
17.177
15.033
16,960
0.508
0.705
0.610
0.670
0.171
0.910
2.7
500
0.889
0.927
16,855
14.125
15.891
0.514
0.702
0.596
0.667
0.157
0.895
1.3
1000
0.878
0.922
16.678
15.234
15.697
0.535
0.675
0.549
0.679
0.170
0.897
2.0
hard for the model to learn, while one that is too low may make the pre-training process excessively easy, offering little benefit. for developing useful fragment representations.
5 Conclusion
In this paper, we propose a self-supervised learning framework, FragFormer, designed for molecular prop- erty prediction that enhances both performance and model interpretability. FragFormer features a novel molecule fragmentation method DOVE which generates overlapping fragments through iterative line graph transformation. Additionally, we implement nested masked fragment prediction task to help the model learn the hierarchical fragment representations, along with a knowledge fusion layer to integrate fixed molecular features. To facilitate interpretability, we propose a simple yet effective method called FragCAM,, which identifies the critical fragments contributing to predictions. FragFormer achieves state-of-the-art performance on eight out of eleven molecular property prediction tasks on PharmaBench and surpasses strong baseline models on long-range biological benchmark. Furthermore, FragFormer demonstrates superior interpretability on both synthetic and real-world molecular datasets, effectively identifying decisive fragments.
Acknowledgments
This work was supported by Beijing Natural Science Foundation NO. 4252046.
References
Abien Fred Agarap. Deep learning using rectified linear units (relu). CoRR, abs/1803.08875, 2018. URL
http://arxiv.ore wwtrxivv.orgaaan abs///03..... 08375
Bruce N. Ames, Frank Lee, and William Durston. An improved bacterial test system for the detection and classification of mutagens and carcinogens. Proceedings of the National Academy of Sciences of the Uniled States of America, 70 3:782 6, 1973. URL https://api. semantccscholarr wwwpCorppsII::::....... :29499p61 29499607
Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advonces in Neural Information Pro- cessing Systems 33: Annual Conference on Newral Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL htttps:/pproceedingg. wetpeiiiinnnnnuuiis...e.. hctps/cccppppp/c/////ppapp hhash/
922teeeeeecdddf11112272777666f Abstsrrr html
Viraj Bagal, Rishal Aggarwal, P. K. Vinod, and U. Deva Priyakumar. Molgpt: Molecular generation using a transformer-decooder model. J. Chem. Inf. Model, 62(9))2064 2076, 2022. doi: 10.00///ASS.JIII.1000000
URL https::/doi..rgg/ 1021/aca jcim. jcim.1c00600 1c00600
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Ganzalzz Vinícius Florcs Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adamn Santoro, Ryan Faulkner, Çaglar Gülçelure, H. Francis Song, Andrew J. Ballard, Justin Gilmer, George E. Dahl, Ashish Vaswani, Kelsey R. Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Hcoss, Daan Wierstra, Pushmoet Kohli, Matthew M. Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. Relational inductive biases, deep learning, and graph networks. CoRR, abs/ hts/18066.12611 2018. htll:htt:://ae http:://axxiii..... org. has//bb////00 01261
13

---

Published in Transactions on Machine Learning Research (04/2025)
Giorgos Bouritsas, Fabrizio Frasca, Stefanos Zafeiriou, and Michael M. Bronstein. Improving graph neural
network expressivity via subgraph isomorphism counting. TEFE Trans. Pattern Anal. Mach. Intell., 45 (1):657 668, 2023. doi: 10.1009/TPAAL..2022..2 3154319
URL
2022
Leo Breiman. Random forests. Mach. Learn., 45(1):5 32, 2001. doi: 10.1023/A410000933444441 URL
http://00...rr/110.00/////////
Xavier Bresson and Thomas Laurent. Residual gated graph convnets. CoRR, abs/1711.07553, 2017. URL
sttp://axxa..org/aab///ii http://orrrgabb//11.075555555
Hanxuan Cai, Huimin Zhang, Duancheng Zhao, Jingxing Wu, and Ling Wang. FP-GNN: a versatile doep
learning architecture for enhanced molecular property prediction. Briefings Bioinform., 23(6), 2022. doi: 10.1093/BIB BBBB///IIBBBACCC /BBAC088. URL httpp::///oll..rg1110000//////b/
Baiyu Chen, Ziqi Pan, Minjie Mou, Yuan Zhou, and Wei Fu. Is fragment-based graph a better graph- basod molccular representation for drug design? a comparison study of graph-based models. Comput. Biol. Med., C6), April 2024. ISSN 0010-4825, doi: 00.1016/..cmmpbonee..0000 202.pp.0ooome2..2020000....1100. 107811. URL https: ord.i.oro//1..100 1016/ compbi 2023. 107811
Hongming Chen, Ola Engkvist, Yinhai Wang, Marcus Olivecrona, and Thomas Blaschke. The rise of deep learn- ing in drug discovery. Drug discovery today, 23 6:1241 1112--12500, 2018. URL https::/appl semanticscholar org/ CCorpusID:467924777
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13- 18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 1725 1735. PMLR, 2020a. URL http:///proceedings: mlr, press/v119/chetiiiir chtn/ov.nnvvvmmmm html
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Balaji Krishnapuram, Mohak Shah, Alexander J. Smola, Charu Aggarwal, Dou Shen, and Rajeev Rastogi (eds.), KDD, pp. 785 794. ACM, 2016. ISBN 118-1-45033-232-44 URL http:://blp. httt:://bblp.uuittrier uni-trerr dw/dd//dddeeeeeeee.ee. conf /tt///////k/kk// htm1#ChenG16
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive lcarning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp- 1597-1600 PMLR, 2020b. URL htttp://proceeedings. mlr ppress/v19//a/
.html
Corinna Cortes and Vladimir Vapnik. Support-vector networks. Mach. Learn., 20(3):273 297, 1995. doi: 10.1007/BF00094001 URL
Jörg Degen, Christof Wegscheid-Gerlach, Andrea Zaliani, and Matthias Rarey. On the art of compiling and using 'drug-like' chemical fragment spaces. ChemMedCCemm 3(10):1503 1507, 2008. ISSN 1860-7187. doi: 10.1002/cmdec 200cc.00000178.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-ILT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Shorl. Papers), pp. 4171 4186. Association for Computational Linguisties, 2019. doi: 10.18653/V1/N15 1423. URL https::/t/oooii htw.atiiioogoo... org/10..01111.. 11t...11111111//11111111
Michael Dickson and Jean Paul Gagnon. Key factors in the rising cost of new drug discovery and development. Nature Reviews Drug Discovery, 3:417 429, 2004. URL https://app..emmaticcoot www.arrrrrrrraeeeeeeeee CorpusID 28009388
Joseph L. Durant, Burton A. Leland, Douglas R. Henry, and James G. Nourse. Rooptimization of mdl keys for use in drug discovery. Journal of chemical information and computer sciences, 42 6:1273-26 -00 2002. URL pestps://app.. semanticscholar. htt.:/olww....oooooouuupa///2222
14

---

Published in Transactions on Machine Learning Research (04/2025)
Vijay Prakash Dwivedi, Chaitanya K. Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Bench- marking graph neural networks. CoRR, 2bs//2003000882, 2020. http:htpps///rxi...oor/aaaa////p
Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Graph neural networks with learnable structural and positional representations. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. DpenReview.net, 2022a. URL https://openrewte http:///ppppoooooeeeeeoo/o/orrrr not///ooorrrt
Vijay Prakash Dwivedi, Ladislav Rampásek, Michael Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, and Dominique Beaini. Long range graph benchmark. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Arnual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022b. URL http://paperr: pesapers.nips..... wwwpappp_filaaeeeeeeeeeeeeeeeeae filffiee///pee ltsrsfffeseapppe//ee paper/202// has1/866660000055777666666f
Datassts- and Benchnarks..hheee hhttlmmkk...m.l..........
Benedck Fabian, Thomas Edlich, Héléna Gaspar, Marwin H. S. Segler, Joshua Meyers, Marco Fiscato, and Mohamed Ahmed. Molecular representation learning with language models and domain-relevant auxiliary tasks. CoRR, 0ttp/2001.1333311 2020. URL attpss//aarie http///axivv.o orgg/bs//////aaa 13230
Xiaomin Fang, Lihang Liu, Jieqiong Lei, Donglong He, Shanzhuo Zhang, Jingbo Zhou, Fan Wang, Hua Wu, and Haifeng Wang. Geometry-enhancee molecular representation learning for property prediction. Nature Machine Intelligence, 4:127 - 134, 2021. URL http://pp...mmmnncccchooooll....
wwwpcchhhoarrrro/oooooooroooo Argg/Corpusil 2334172265
Yin Fang, Qiang Zhang, Ningyu Zhang, Zhuo Chen, Xiang Zhuang, Xin Shao, Xiaohui Fan, and Huajun Chen. Knowledge graph-enhanced molecular contrastive learning with functional prompt. Not. Mac. Intell., 5(5): 542 553, 2023. doi:
URL https: http:///dooiooo////// otg/10. 000.1000//42255-02-0006----
Anna Gaulton, Anne Hersey, Michal Nowotka, A. Patrícia Bento, Jon Chambers, David Mendez, Prudence Mutowo, Francis Atkinson, Louisa J. Bellis, Elena Cibrián- Cibrián-Uhalte, Mark Davies, Nathan Dedman, Anneli Karlsson, María Paula Magariños, John P. Overington, George Papadatos, Ines Smit, and Andrew R. Leach. The chembl database in 2017. Nucleic Acids Research, pp. D945 D945-D954, Jan 2017. doi: (003///r/gww/li URL http://dx http://xxddi.org/////// 10t.00000//aa///1110
Zhichun Guo, Chuxu Zhang, Wenhao Yu, John Herr, Olaf Wiest, Meng Jiang, and Nitesh V. Chawla. Few-shot graph learning for molecular property prediction. In Jure Leskovec, Marko Grobelnik, Mare Najork, Jie Tang, and Leila Zia (eds.), WWW '21: The Web Conference 2021, Virtual Event / Ljubljana, Slovemia, April 19-23, 2021, pp. 2559 2567. ACM/ / IW3C2, 2021. doi: 0111444442281..23 sttps://doi.com org/ /0/ 1145/344433/111111 14414433112111 3450112.
URT
Olgun Guvench. Computational functional group mapping for drug discovery. Drug discovery lodag, 21 12: 1928 1931, 2016. CRL https://api.. httpp:://pp..eeaatiiiiiae semantccscoolarrr oot.olaarr.rg/oooouu ID: 399377188
William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Sys- lems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. Abbaaaeeartaebbbeeeaaabattt.... html
2017a. URL
neurips............o..m....... hccpppper/2017/ /hash/
William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning large graphs. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnell (eds.), Advances in Newral Information Processing Sys- tems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 1024 1034, 2017b. URL Ga59150034406666acc7777aaaa Abstract............... html
wtw.eeeeedinnnnnnuiiieeeee wwtppppe//111/////cec /hash/
Corwin Hansch, Peyton P. Maloncy, Toshio Fujita, and Robert M. Muir. Correlation of biological activity of phenoxyacetic acids with hammett substituent constants and partition coefficients. Nature, 194:178-180, 1962. URL. https::/ http::::/app.ssmaatiicccee www.steiscooolltaae.eeeeee ott.iooo......rroooooppaa/a11111
15

---

Published in Transactions on Machine Learning Research (04/2025)
Kaiming He, Xinlci Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. Masked autoencoders are scalable vision learners. In TEEE/CVE /VV Conference on Compuler Vision and Patlern Recognalion, CVPR 2022, New Orleans, LA, USA, June 18-24, 24, 2022, pp. 15979 15988. IEEE, 2022. doi: 10.1109/ CVPR52688.200 202.88.50222.115.... URL bttpp::/ddoi.ore 10. 1109/CVPR52688. 2022. 01553
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. Graphmae: self-supervised masked graph autoencoders. In Aidong Zhang and Huzefa Rangwala (eds.), KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14- 18, 2022, pp. 594 604. ACM, 2022. doi: 10. 00.1145/3334688.5553333 1584678.35399PP
URL https://doo.orr//1 1145/
Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE ACM Trans. Audio Speech Long. Process., 29:3451-3460, 2021. doi: 10.1109/TASIA,
2021.3122291. URL https: http:////.o.../......//. 1101000S11 2021 3122221
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay S. Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In 8th International Conference on Learning Rep- resentations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL
Yinghui Jiang, Shuting Jin, Xurui Jin, Xianglu Xiao, Wenfan Wu, Xiangrong Liu, Qian Zhang, Xiangxiang ZZng, Guang Yang, and Zhangming Niu. Plarmaccohorii-consstiiiie heterogeneous graph transformer model for molecular property prediction. Communications Chemistry, 6, 2023. URL https://app
semanticscholar. org/ CorpusID SDerpusID:25993005 257903087
Wengong Jin, Regina Barzilay, and Tommi S. Jaakkola. Junction tree variational autoencoder for molecular graph generation. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 2328 2337. PMLR, 2018. URL http: \/proceedings.o mlr. www...pressvv//// http:///vojjinn..nmea jinjja.nnna..m html
Wengong Jin, Regina Barzilay, and Tommi S. Jaakkola. Hicrarchical generation of molecular graphs using structural motifs. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 4839 4849. PMLR, 2020. URL http:///proceedingss mlr wtt.pppeee/.11//jraa.... htmml
Seojin Kim, Jaehyun Nam, Junsu Kim, Hankook Lee, Sungsoo Ahn, and Jinwoo Shin. Holistic molecular representation learning via multi- fragmentation. Trans. Mach. Learn. Res., 2024, 2024. URL https////oppneeeeeeeeeee/tttttee net:/oorur/idduuhh5551111
Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A. Shocmaker, Paul A. Thicssen, Bo Yu, Leonid Y. Zaslavsky, Jian Zhang, and Evan E. Bolton. Pubchem 2023 update. Nucleic acids research, 2022. URL https://api. www.sseitaantool.....oe.eoo htttp::/app..emmatticcchoh
otg.emariiiicccoo...ooooooo/pppp wrw/Corpustaooooooooooooooee 253182955
Diederík P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, Son Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http://arxiv. org/ abs/ /etes/112..66 6980
Diederik P. Kingma and Max Welling, Auto-encoding variational bayes. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Leurning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http::/arxiv. otgp abs/ http//131...11 6114
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.nn net, 2017. URL htttps://ppenreviiet http:///opppooroooooooooooooa1. net/forummid SJU4ayYg1
16

---

Published in Transactions on Machine Learning Research (04/2025)
Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz-Richardson. Captum: A unified and generic model interpretability library for pytorch. CoRR, abs/2009.07896, 2020. URL
https://arxie wwxiv..rg/abaa.... hagg/bbs/20 2009 0789
Xiangzhe Kong, Wenbing Huang, Zhixing Tan, and Yang Liu. Molecule generation by principal sub- graph mining and assembling. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Confer- ence on Neural Information Processing Systems 2022, NewrIPS 2022, New Orleans, LA, USA, Novem- ber 28- December 9, 2022, 2022. URL http::/papenn nips cttpaperrcpppp...ppp hcc/ppper_flle//ppppeeeeeee pttpppppppppppp//pppppppppppp 2022/hash/ 1167722abb.deebbbfffffffffefeee -abrrectbbttaatt--b... CanirrcctCoooffeee htmll
Devin Kreuzer, Dominique Beaini, William L. Hamilton, Vincent Létourneau, and Prudencio Tossou. Rethinking graph transformers with spectral attention. In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jemnifer Wortman Vaughan (eds.), Advances in Newral Information Processing Systems 34: Annual Conference on Newral Information Processing Systems 2021, NewrIPS 2021, December 6-14, 2021, virtual, pp. 21618 21629, 2021. URL https::/proceeddin htwproceeeiigs..err....c.cpppppp html
Greg Landrum. Rdkit: Open-source cheminformatics software. 2016. URL https::/github..cmm
rcthtt/releessee//g///llleee _0____2212_____ __0__
Xiao Qing Lewell, Duncan B. Judd, Stephen P. Watson, and Michael M. Hann. Recapretroosythetii combinatorial analysis procedure: A powerful new technique for identifying privileged molecular fragments with useful applications in combinatorial chemistry. Journal of Chemical Information and Compuler Sciences, 38(3):511-522 May 1998. doi: 10.1021/ci970422
Han Li, Ruotian Zhang, Min Yaosen, Dacheng Ma, Dan Zhao, and Jianyang Zeng. A knowledge-guicee pro-training framework for improving molecular representation learning. Nature Communications, 14, 11 2023. doi:
Pengyong Li, Jun Wang, Yixuan Qiao, Hao Chen, Yihuan Yu, Xiaojun Yao, Peng Gao, Guotong Xie, and Sen Song. An effective self- supervisd framework for learning expressive molecular global representations drug discovery. Briefings Bioinform.. 22()), 2021. doi: 10.1093/BII, /B////BB//BBBOO URL hhttps://dol 1tgg/10.1993/ibb/bbaeee
Shengchao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. Pre-training molecular graph representation with 3d geometry. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https
//openreview
Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Anima Anandkumar. Multi-modal molecule structure-text model for text-based retrieval and editing. Nature Machine Intelligence, (112)):4447145:5 Dee 2023. ISSN 2522-5839. doi: 10.1038/s42256-022 00222226202200775556 URL https://aoi. org/10. 1001
Dongshong Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tier Lin (eds.), Advances in Neural Information Pro- cessing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL https://prooceedingg 33770Gc00153300066666666888888 Abstract html
Shengjie Luo, Tianlang Chen, Yixian Xu, Shuxin Zheng, Tie- Yan Liu, Liwei Wang, and Di He. One transformer can understand both 2d & 3d molecular data. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.nett 2023. URL
17

---

Published in Transactions on Machine Learning Research (04/2025)
Kha-Dinh Luong and Ambuj K. Singh. Fragment-basee pretraining and finetuning molecular graphs. In Al- ice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2028, NewrIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. URL/http://pppers: http::///ppprr..p.......p cc/ html
paper http:///ee../eeeeeceeee///eeaaea
Ricardo Macarron, Martyn N. Banks, Dejan Bojanic, David J. Burns, Dragan A. Cirovic, Tina Garyantes, Darren V. S. Green, Robert P. Hertzberg, William P. Janzen, Jeff W. Paslay, Ulrich Schopfer, and G. Sittampalam. Impact of high-throughput screening in biomedical research. Noture Reviews Drug Discovery, 10:188-195, 2011. URL http:://api.sonntiiccco wwwanticschollllttt. wwwp:oopps..o.........o.m. /tts///orpuII::257777799
Dominic Masters, Josef Dean, Kerstin Kläser, Zhiyi Li, Samuel Maddrell-Mander, Adam Sanders, Hatem Helal, Deniz Beker, Andrew W. Fitzgibbon, Shenyang Huang, Ladislav Rampásek, and Dominique Beaini. GPS++: reviving the art of message passing for molecular property prediction. Trans. Mach. Learn. Res., URL https:://peenrevier
2023, 2023.
Kevin McCloskey, Ankur Taly, Federico Monti, Michael P. Brenner, and Lucy J. Colwell. Using attribution to decode binding mechanism in neural network models for chemistry. Proceedings of the National Academy of Sciences, 116(24)111624 11629, Jun 2019. doi: 10.1073 1..0078/pnaa.118055511666 URL http: //dx dog.....//..oo.o. 1073/pnas. 1820657116
Cédric Merlot, Daniel Domine, Christophe Cleva, and Dennis J Church. Chemical substructures in drug discovery. Drug discovery today, 8 13:594-600 -020 2003. httpss//::///pp.. httapp/s/ppp.smmmatiicccooooooo
swwanticscholaaat.... org/ Stsp/CrrpusID
35874487
Asher Mullard. New drugs cost us$ billion to develop. Nature Reviews Drug Discovery, 13:877 877, 2014. URL https://aphl semanticscoolar org/CorrpsII::22100 12198576
Christopher W. Murray and David C Rees. The rise of fragment-based drug discovery. Nature chemistry, 1 3:
187-92, 2009. URT. https::/appp http:/ppp..emmnnccccchol.l.r
Zhangming Niu, Xianglu Xiao, Wenfan Wu, Qiwei Cai, Yinghui Jiang, Wangzhen Jin, Minhao Wang, Guojian Yang, Lingkang Kong, Xurui Jin, Guan Yang, and Hongming Chen. Pharmabench: Enhancing admet benchmarks with large language models. Scientific Data, 11, 2024. URL http:pss//app..emattcccccoaaaa semanticcchoinrr
272553326
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high bighhperformance deep learning li- brary. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' A'Ach Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Confer- ence Neural Information Processing Systems 2019, NewrIPS 2019, December 8-14, 2019, Vancou- ver, BC, Canada, pp. 8024 8035, 2019. URL httpp::/ppoceeednng: www.eeediinngn.euiii..ee. wwwpurpss.ccppppee//eeee.e Abstract. html
Phillip E. Pope, Sohcil Kolouri, Mohammad Rostami, Charles E. Martin, and Heiko Hoff- mann. Explainability methods for graph convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16- 20, 2019, pp. 10772-10781. Computer Vision Foundation IEEE, 2019. doi: 10.1109/CVPR. 2019.01103. URL http:://ppeacceesst thecce thecvf con/content, CVPR_ h0019//tl//Pope_pppaeee Methods_ for CrapphConvvluutiona Neural Networks_ CVPR 2019 2019_paperr_____ _aaperr...... html
Michela Proietti, Alessio Ragno, Biagio La Rosa, Rino Ragno, and Roberto Capobianco. Explainable AI in drug discovery: self-interpretable graph neural network for molecular property prediction using concept whitening. Mach. Learn., 113(4):2013 2044, 2024. doi: 00.1007/S109994-23-00832 URL
18

---

Published in Transactions on Machine Learning Research (04/2025)
Jiahua Rao, Shuangjia Zheng, and Yuedong Yang. Quantitative evaluation of explainable graph neural networks for molecular property prediction. CoRR, abs/2107.04119, 2021. URL https://arxiv http /bs/
2210. 04119
Matthias Rarey and J. Scott Dixon. Feature trees: new molecular similarity measure basod on tree matching. J. Comput. Aided Mol. Des., 12(5):471 490, 1998. doi: 10. 10.1023/ A:1008068904628. URL
https://doi.org 10. 1023/022/4222 10080689046288
David Rogers and Mathew Hahn. Extendded-commectivvyy fingrrrrints J. Chem. Inf. Model., 50(5):7422 754, 2010. doi: 10.1021/
10.1021/CT1000001 CT100050T. URL htttps://do. 1tt1://.112111111111001100100110
Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. Self-supervised graph transformer on large-scale molecular data. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NewrIPS 2020, December 6- 2020, virtual, 2020. URL httpp://prooceedings http://nngsneuur..ccceeeeeeeeeee cc/paper/2000 hash/ 04ee3344eea33300eeeefffffffffff
Jerret Ross, Brian M. Belgodere, Vijil Chenthamarakshan, Inkit Padhi, Youssef Mrouch, and Payel Das. Large-scale chemical language representations capture molecular structure and properties. Nature Machine Intelligence,
2021. URL httpp:://pp.eeenattccccooo..... org/Corpus ID: 254636625
semanticscholas
T. Konstantin Rusch, Michael M. Bronstein, and Siddhartha Mishra. A survey on oversmoothing in graph neural networks. CoRR, abs/ abs/2303.1nm 10993, 2023. doi: 10.48550/ARXIT 20..23333.1009 10993. URL https:
2303. 10993
Tanja Schulz-Gasch Christin Schärfer, Wolfgang Guba, and Matthias Rarey. TFD: torsion fingerprints as a new measure compare small molecule conformations. J. Chem. Inf. Model. 52(6))1499-115) 1111199999111,-1 2012. doi: 1011/110023111
10.1021/CT2003188 URL
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vis., 128(2):336-335) 2020. doi: 10.1007/S112633 12111119-111---222--2- URL. 0rtp://////...r//1..00000000
611263-019-01228-7
Rico Semnrich, Barry Haddow, and Alexandra Birch. Neural machine translation rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Linguistics, 2016. doi: 10.18653/ (18663/V11P16-1162 URL https:///oi.orrg wttp//001165///1/111116661
Yu Shi, Shuxin Zheng, Guolin Kc, Yifci Shen, Jiacheng You, Jiyan Hc, Shengjie Luo, Chang Liu, Di Hc, and Tie- Yan Liu. Benchmarking graphormer on large-scale molecular modeling datasets. CoRR, abs/2202 22200.040,,, 2022. doi: 10.48550/ ARXIV STT.V.2203.000 .200.04880. URL https: //do: org/ /00g//1..../ 48550/arXiv. arxiv... www.xii............
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In Doina Precup and Yee Whye Teh (ods.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp- 3145-3151 3153. PMLR, 2017. URL http:://rooceednng.. mlr
Steven R A Simoens and Isabelle Huys. R&dd costs of new medicines: A landscape analysis. Frontiers in Medicine, 8, 2021. URL https://api.
https:://ap..emmaticchhhht www.eeiiitanolt.e.ee.......eeeee www.llaarrrg/ooooooeee org/CorpusID: 239770727
Ying Song, Shuangjia Zheng, Zhangming Niu, Zhang- Hua Fu, Yutong Lu, and Yuedong Yang. Communicative representation learning on attributed molecular graphs. In Christian Bessiere (cd.), Proceedings of the Twenty-Niie Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pp. 2831-2838. ijeai.org, 2020. doi: 22.4463/TCCAA.22200 URL http::::://aaii.rt///oo/22ee// 202033332222222222222
19

---

Published in Transactions on Machine Learning Research (04/2025)
Yuanbing Song, Jinghua Chen, Wenju Wang, Gang Chen, and Zhichong Ma. Double-head transformer neural network for molecular property prediction. J. Cheminformmalics, 15(1):27, 2023. doi: 10.1186/
013321-023-00700-- URL https://doi. http://6.....0r1010.111111111111 111111///11332100200000000000
Hannes Stärk, Dominique Beaini, Gabriele Corso, Prudencio Tossou, Christian Dallago, Stephan Günnemann, and Pietro Lió. 3d infomax improves gnns for molecular property prediction. In Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (ods.), International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 20479 20502. PMLR, 2022. URL htttps://proceedingg mlr press//
v162/stark22a.11 html
Teague Sterling and John Irwin. ZINC 15 ligand discovery for everyone. J. Chem. Inf. Model., 51(: 2324-2337, 2015. doi: 00.002/ACS..CIIM50005555 URL https://doi. wtt.///i..rr/................. 1tt.///......mmmm......o......o. 5b00559 5
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Doina Precup and Yee Whye Tch (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Reseorch, pp. 3319 3328. PMLR, 2017. URT http://prooceedings mlr
lurii Sushko, Elena Salmina, Vladimir Potemkin, Gennadiy Poda, and Igor V. Tetko. Toxalerts: A web server of structural alerts for toxic chemicals and compounds with potential adverse reactions. J. Chem. Inf. Model., (288)22310-20 2216, 2012. doi: (0.1021/C13002455 URL http:://ooi.org//10.10/1////1/
Alexandre Tkatchenko. Machine learning for chemical discovery, Nature Communications, 11, 2020. URL https://api. wwwppp/ssssaaaccccca.c...o /
ottgs///ocooorarooor/rrrrooooooa 2211337182
Alexander Tropsha. Best practices for qsar model development, validation, and exploitation. Molecular Informatics, 29, 2010. URL atttp:://apl.sematticeeee htt.:/aar....//ooousss2222225550
Ashish Vaswani, Noam Shazoer, Niki Parmar, Jakob Uszkorcit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Newral Information Processing Systems 30: Annual Conference Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 6008, 2017. URL https://prroceoding: nouripp cc/paper/2017 hc/pappe//2111naaa//5eee22222 http:///ww..teeeeeeeeeeaaaaaaaa. wtwrre/ccaaaaaaaaaaaatttt...
neurips
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pictro Liò, and Yoshua Bengio. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL / wwweeeerrriieeeeeeefeeoeeeeeeeee wwttormmmiiirJoooo JXMpikCZ
Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. Deep graph infomax. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.ndd nee, 2019. URL
Aaron Virshup, Julia Contreras-Gardh García, Peter Wipf, Weitao Yang, and David N. Beratan. Stochastic voyages into uncharted chemical space produce a representative library all possible drug- compounds. Journal of the American Chemical Society, 135 19:7296 303, 2013. URL https://api.... www.:p/ssssnnnnnnnna....oo CorpusID:10540378
Sergei Voloboev. A review on fragment- based de novo 2d molecule generation, 2024. URL https: //arxiv org/abs/2405 05293
W. Patrick Walters and Regina Barzilay. Applications of deep Iearning molccule generation and molccular property prodiction. Accounts of chemical research, 2020. URL htttps://aapp httpss///pp..mmmnnccccloooo.. www.sticscholar.ncoooooo org/ CorpusID:229711900
20

---

Published in Transactions on Machine Learning Research (04/2025)
Minjie Wang, Lingfan Yu, Da Zheng, Quan Gan, Yu Gai, Zihao Ye, Mufei Li, Jinjing Zhou, Qi Huang, Chao Ma, Ziyue Huang, Qipeng Guo, Hao Zhang, Haibin Lin, Junbo Zhao, Jinyang Li, Alexander J. Smola, and Zheng Zhang. Deep graph library: Towards efficient and scalable deep learning on graphs. CoRR, abs/1909.01315, 2019. URL http:://axxxv.orrabb///////s
David Weininger. Smiles, chemical language and information system. 1. introduction to methodology and encoding rules. J. Chem. Inf. Comput. Sci., 28:31-36, 1988. URL [
Io 5445756
Oliver Wieder, Stefan M. Kohlbacher, Mélaine A. Kuenemann, Arthur Garon, Pierre Ducrot, Thomas Seidel, and Thierry Langer. A compact review of molecular property prediction with graph neural networks. Drug discovery today. Technologies, 37:1-12, 2020. URL CtR.hhtts::/// httpss//app. semanticscholar org/ Tag/CCrppuID:
230528669
Tom Wollschläger, Niklas Kemper, Leon Hetzel, Johanna Sommer, and Stephan Günnemann. Expressivity and generalization: Fragment-biases for molecular gnns. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. Open net, 2024. URL https //openreview wwt..e/fooouuiiidnnnsebbbbb.
Olivier J. Wouters, Martin Mckee, and Jeroen Luyten. Estimated research and development investment needed to bring new medicine to market, 2009-2018. JAMA, 323 9:844 853, 2020. URL https //api. saapiisemantccscholl.. wttp:/scchhaaa.ooo//oooooo/eeeee wwt.:oorrposDD:21118877777
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. Moleculenct: A benchmark for molecular machine learning. Chemical Science, doi: 10.1039/ CC7SC2262266.
9, 10 2017.
Jun Xia, Chengshuai Zhao, Bozhen Hu, Zhangyang Gao, Cheng Tan, Yue Liu, Siyuan Li, and Stan Z. Li. Mole-bert: Rethinking pre-training graph neural networks for molecules. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview net, 2023. URL
Zhaoping Xiong, Dingyan Wang, Xiaohong Liu, Feisheng Zhong, Xiaozhe Wan, Xutong Li, Zhaojun Li, Xiaomin Luo, Kaixian Chen, Hualiang Jiang, and Mingyue Zheng. Pushing the boundaries of molecular representation for drug discovery with graph attention mechanism. Journal of medicinal chemistry, 2020. URL httpp:://app httpppppmmatiiccccolarrrraa
Darg/CorpusID:: II: 199572628
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanic Jegelka. How powerful are graph neural networks? In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May G-9, 2019. OpenReview.net net, 2019. URL http::///pppueeeeeeeeeeeerrrrrrr
Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, and Jian Tang. Self-supervised graph-level representation learning with local and global structure. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Evenl, volume 139 of Proceedings of Machine Learning Research, pp. 11548 11558. PMLR, 2021. URL http://proceedinng mlr. E.r.press/v13)) http////19///x..g./
Ling Xue and Jürgen Bajorath. Molecular descriptors in chemoinformatics, computational combinatorial chemistry, and virtual screening. Combinatorial chemistry & high throughput. screening, 3 5:363-77 72, 2000.
URL https://api. sewanticscholaar
httppp////pp..essaiicccccloll Bacilaarorr//orro org/CorpusID :29 29690850
Kevin Yang, Kyle Swanson, Wengong Jin, Connor W. Coley, Philipp Eiden, Hua Gao, Angel Guzman-Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, Andrew Palmer, Volker Settels, Tommi S. Jaakkola, Klavs F. Jensen, and Regina Barzilay. Analyzing learned molccular representations for property prodiction. J. Chem. Inf. Model, 58)8:33370-3383 2019. doi: 10.121//ACS.JIMM..BBO URL hattps://di..og: 10. n0.1021/aca.. jcim. 9b00237
21

---

Published in Transactions on Machine Learning Research (04/2025)
Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tic- Yan Liu. Do transformers really perform badly for graph representation? In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jemnifer Wortman Vaughan (eds.), Advences in Neural Information Processing Systems 94: Annual Conference on Neural Information Processing Systems 2021, NewrIPS 2021, Decembbe 6-14, 2021, virtual, pp. 28877 28888, 2021. URL httpp::/pproceedingg satpoueedinggn.euriipss neurips cc/paper/ cc//apper/221/haah/EEeeeeee htt0////aaa.aatee56000000000a0a. Abstractt.rttt... .tmml
Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovee. Gnnexplainer: Generating explanations for graph neural networks. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' d'Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Newral Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NewrIPS 2019, December 8-14, 2019, Vancouwer, BC, Canada, pp- 9240 9251, 2019. URL htttps://proceediinggg wtwpoccediinnn..eniis....o...
2019
Aww.rf-ttbstttttt..m html
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Had- sell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL httpps:/pproceedinggs wwwpooeeiingg..nuuiis.....o wwtpeipp.ccpppppe//0000000.0.... 1t620088893122113277466683383882
Abstract. html
Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. Graph contrastive learning automated. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning
Research, pp. 12121-12131 PMLR, 2021. URL URL http::/pproceedingg... mlr. wttp....e.s//r//3/yoo........
Hongyang Yu and Hongjiang C. Yu. Tensorvae: a simple and efficient generative model for conditional molecular conformation generation. Trans. Mach. Learn. Res., 2024, 2024. URL htttp://opperrviai net/forumfiddrDggtdYyoo
Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 12241-12255 PMLR, 2021. URL http://prrooeeeiig mmw.:/..eeeiinnmmm.......mm mlr. /.rrs/
v139/yuan21c.2.
Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: taxonomic survey. IEEE Trans. Pallern Anal. Mach. Intell., 45(5):5782 5799, 2023. doi: PTA.://TTPA...222.2222.2.. 1109/TPAMI 2022. 32042366
URL https://doi.org:
Xiangxiang Zeug, Hongxin Xiang, Linhui Yu, Jianmin Wang, KenLi Li, Ruth Nussinov, and Feixiong Cheng. Accurate prediction of molecular properties and drug targets using a self-supervised image representation learning framework. Nature Machine Intelligence, 4:1004 1016, 2022. URL https://api
semanticscholar. www.c/oooooorroooo.ooooooooooolt www.oogg//oooo.. Chrp/orrpss:::: : 253631433
Bingxu Zhang, Changjun Fan, Shixuan Liu, Kuihua Huang, Xiang Zhao, Jincai Huang, and Zhong Liu. The expressive power of graph neural networks: A survey. CoRR, abs/2308..888 08235, 2023. doi: 00.48550/ARXTV 2308.08235. URL httph::/t//ooo . org/10. wtg//0.48550////// www.i..w0........... 03308008233
Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Chee-Kong Lee. Motif-based graph self-supervised learning for molecular property prediction. In Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Newral Information Processing Systems 2021, NewrIPS 2021, December 6-
14, 2021, virtual, pp. 15870-15882 2021a. URL htttps:// htwp:ooceeddinggnn.nii..ee.. neurips cttpcc/p////..cccccccccccccccc/ pptppppp///22//aah 682077899566671f00000eoffffedee2
html
Ziqiao Zhang, Jihong Guan, and Shuigeng Zhou. Fragat: a fragmentt-oriented multi-scale graph attention model for molccular property prediction. (110))00000 2021b. doi: FO.DBB//IOOTFFFRMAATCC//
BTAB195. URI. http:pps//ooooorr///////////aoaa
22

---

Published in Transactions on Machine Learning Research (04/2025)
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In 2016 IEFE Conference Compuler Vision and Paltern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp. 2921 2929. IEEE Computer Society, 2016. doi: (AL109//VVP.2000
URL https: /topps://ooi. http::://dooooo/////// 10. 1109/CVPRP 2016. .19
Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular represcntation learning framework. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.nel, 2023. URL
Jinhua Zhu, Kehan Wu, Bohan Wang, Yingee Xia, Shufang Xie, Qi Meng, Lijun Wu, Tao Qin, Wengang Zhou, Houqiang Li, and Tie- Liu. O-gnn: incorporating ring priors into molecular modeling, In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL /ttp////e//////////////////////n http://wwww...tt/oooooooot/e//on
A Proof of Theorem
Recall that (G) is the k-order line graph of G. Each vertex in (G) corresponds to (k + 1)-size connected subgraph in G and we denote its node set by C(v). We assume G is connected with more than k vertices.
Lemma 1. Given graph G EV, for every two adjacent nodes u, u in L (G), we have (C(u) nC(()) k.
Proof. Since u and u are adjacent in [²), they share a vertex in IC=)((), which means they share k-size connected subgraph in G. Thus, [C(u) n CCv) k.
Theorem. Given a graph = EV, , the induced fragmentation of standard 0-overlapping fragmentation of (G) is a k-overlapping fragmentation of G.
Proof- Denote the standard O-overlapping fragmentation of (G) by F= (P, and the induced fragmenta- tion as F" = (Q, A), where P = {U4) A E g Q [Vi) V U C(v). We will prove that Fx is nEU,
k-overlapping fragmentation of G.
Ü V. Ü C(U.) = V, so Q is a vertex cover of G. Since G [U,] is connected, and every two adjacent nodes in L (G) share at least k nodes in G, so G is also connected.
m
Thus, F' is a fragmentation of G.
2. If Aij 1, then there exists u, Up and uj Uj, us and uj are adjacent in Lt) By Lemma (C(,,) n ((u)) k. Thus,
MOvi = ( (())) n c(m)) = U
c())n(())) 2 (z)))
NEt;
e'EUs
LEUi.''U
Combining 1 and 2, conclude Ft is k-overlapping fragmentation of G.
Disconnected Graph For a disconnected graph, we can apply the above theorem to each connected component. If a connected component contains only one node, we keep unchanged in the line graph transformation.
23

---

Published in Transactions on Machine Learning Research (04/2025)
+H2N
F
NH
H2N
3
?
2
6
Non-overlapping Fragment Graph
Atom Graph
Figure 6: An example of connection information loss in non-overlapping fragmentation.
x
(a)
(b)
(c)
Figure 7: (a): 3MR dataset. The blue nodes indicate the 3MR substructure. The last molccule does not contain the 3MR substructure. (b) Benzene dataset. The green nodes indicate the benzene substructure. The last molecule does not contain the benzene substructure. (c) Mutagenicity dataset. The red nodes indicate the substructure alerts. All molecules are mutagenic in ames test.
B Connection Information Loss
We use example to illustrate the connection information loss in the non-overlapping fragmentation, In Figure 0 we segment the molccule into six non-overlapping fragments and get the fragment graph. Within the fragment graph, the blue fragment linked to the green fragment, but it's indistinguishable whether the connection is established through atom 1 and atom 2, or through atom and atom 3. Similarly, the red fragment is connected with the yellow fragment, yet remains unclear this connection formed by atom and atom 5, or possibly atom 4 and atom 6. This specific kind of connection information is unfortunately lost within the context of non-overlapping fragment graph. Based on k-degree overlapping fragmentation, can identify the connection information between fragments through the overlapping atoms. Since the connection information is preserved in the fragment graph, the model can access and implicilly utilize the connection information between fragments. We will explore more explicit ways to utilize the connection information the future work.
C Interpretability Benchmark: DrugXAI
3MR 3MR dataset a synthetic dataset that predicts the existence of 3MR substructure in the molecule. 3MR is a ring that consists of two carbon atoms and one oxygen atom, all connected by single bonds. The examples are given in Figure aa). The dataset is collected from ZINC15 lead-like subset and the ground truth attribution is the 3MR substructure.
24

---

Published in Transactions on Machine Learning Research (04/2025)
Table Dataset size of 3MR, Benzene and Mutagenicity,
Train Set Size
Test Set Size
3MR Benzene
2521
631
9600
24000
Mutagenicity
5204
1302
Table 8: Performance (AUROC) of CMPNN, GraphSAGE, GraphNet, GAT with CAM, GradCAM, Gradluput, IG on 3MR dataset.
3MR
CMPNN
GraphSAGE
GraphNet
GAT
CAM
0.858
0.771
0.832
0.734
GradCAM
0.794
0.783
0.754
0.745
GradInput
0.951
0.844
0.967
0.877
IG
0.966
0.932
0.942
0.905
Best
0.966
0.932
0.967
0.905
Benzene Benzene dataset is a synthetic dataset designed to predict the presence of the benzene substructure within a molecule. Benzene ring consisting of six carbon atoms connected by aromatic bonds. The examples are illustrated in Figure (b). The dataset is also sourced from the ZINC15 lead-like subset, with the ground truth attribution being the benzene substructure.
Mutagenicity Mutagenicity dataset aims to predict the ames mutagenicity of molecules (Ames al. ground truth fragments.
1973 We use the 46 substructure alerts from Sushko et al. 2012
The size of Irain and Lest set after dataset splitting are shown in Table 7 We report the test set attribution AUROC (McCloskey et al.) 2019) of CMPNN, GraphSAGE, GraphNet, GAT with CAM, GradCAM, GradInput. in Table (3MR dataset), Table g (Benzene datase), and Table 10 (Mutagenicity dataset). These results refer to DrugXAI |ao et al. 2021) We list them here for the convenience of the reader.
Other Alternatives of FragCAM We adopt DeepLIFT (Shrikumar el al. 2017 and Integrated Gradients (IG) (Sundararajan et al.) 2017] on the initial fragment vectors, which is the output of subgraph encoder, and denote these methods FragDeepLIFFT and FragIG. We set steps=50 for IG and
the default setting for DcepLIFT in Captum (Kokhiikyan et al. 2020 The results DrugXAI are shown in Table 11 and Table 12 FragCAM outperforms FragDeepl LITT and FragIG on all datasets in terms of AUROC and fidelity, which demonstrates the effectiveness of FragCAM in explaining the prediction results with fragments.
D Experimental Details
Atom Features The composition of atom features is shown in Table 13
Profile of PharmaBench The profile of the PharmaBench dataset is shown in Table 14
Finetuning Details In the fine-tuning stage, we add a sum pooling on top of the pre-trained model and apply a linear predictor at last. We use Adam optimizer with a learning rate of 3e - and a batch size of 32. We train the model for 50 epochs and report the mean performance on the test set over three runs with different random seeds. The model is implemented in DGL (Wang et al. 2019) with PyTorch (Paskke et al., 2019) as backend and trained on single NVIDIA A800 GPU.
Time Measurement on Mutagenicity Dataset We measure the time per sample of FragCAM and SubgraphX the mutagenicity dataset on single NVIDIA 4090 GPU.
25

---

Published in Transactions on Machine Learning Research (04/2025)
Table 9: Performance (AUROC) of CMPNN, GraphSAGE, GraphNet, GAT with CAM, GradCAM, GradIuput, IG on Benzene dataset.
Benzene
CMPNN
GraphSAGE
GraphNet
GAT
CAM
0.831
0.798
0.859
0.776
GradCAM
0.566
0.582
0.657
0.604
GradInput
0.894
0.859
0.934
0.867
IG
0.906
0.801
0.865
0.798
Best
0.906
0.859
0.934
0.867
Table 10: Performance (AUROC) of Mutagenicity, GraphSAGE, GraphNet, GAT with CAM, GradCAM, GradInput, IG on Benzene dataset.
Mutagenicity
CMPNN
GraphSAGE
GraphNet
GAT
CAM
0.539
0.542
0.539
0.494
GradCAM
0.560
0.542
0.539
0.503
GradInput
0.607
0.607
0.654
0.608
IG
0.742
0.742
0.759
0.683
Best
0.742
0.742
0.759
0.683
E More Ablation Study on PharmaBench
Comparison between Different Fragmentation Hierarchies We first construct three hierarchy of fragmentation: All Atom, Junction Tree (JT) (Jin et al. 2018), and DOVE-1. "All Atom' takes each atom as a fragment. JT decomposes the molecule into rings and bonds. The number of fragments follows All Atom > JT > DOVE. We then evaluate the performance of FragFormer with different fragmentation methods, as shown in Table 15 The results show that the overall performance of FragFormer with DOVE is better than JT, which is better than All Atom.
Comparison between DOVE and JT- VA- JT-VAE (Jin et al. 2018 decomposes the molecule into rings and bonds. DOVE decomposes the molocule based on itcrative line graph and principal subgraph mining (Kong et al., 2022). Compared with JT-VAE, DOVE can learn succinet and expressive fragment library from molecule database by mining frequent substructures. The chemical functional groups in DOVE are more diverse and ficxible than the rings and bonds in JT-VAE. Morcover, DOVE can decompose the molecule into fragments with tunable overlapping degree, which is not supported by JT-VAE. We constuct a baseline model called FragFormer-JT, which replace the DOVE with the Junction Tree fragmentation (JT-VAE) and keep all other settings the same. The results on PharmaBench are shown in Table 15 FragFormer-JT achieves lower performance than FragFormer on Pharmabench, which demonstrates the effectiveness DOVE.
Comparison between Different & with the Same Vocabulary Vocabulary construction inherently depends on k, as is performed on (G). We conduct. an ablation experiment to isolate the difference of vocabulary. We first build the vocabulary for k 0, 1, 2,3 with vocab. size=500 and merge the vocabularies to get the final vocabulary. Then, we pre-train the model for k = 0, 2,3 with the final vocabulary separately and evaluate the performance on Pharmabench. The results are shown in Table 16, The optimal results for each dataset occur at k = 1 or & = 2, which is consistent. with the findings in Table [ Interestingly, under the same vocabulary, & = 2 yields superior performance more frequently than & = 1. The trend is reversed in
Table 0
Effect of Key Components Through an Incremental Ablation Study We begin with the bascline model, which employs mask atom prediction without incorporating knowledge fusion ("FragFormer-Atonn w/o K'). Next, introduce DOVE fragmentation with ('FragFormer w/o K & Nested MFP'). Subsequently,
26

---

Published in Transactions on Machine Learning Research (04/2025)
Table 11: Performance (AUROC) of FragCAM, FragIG, and FragDeepLift on DrugXAI.
Dataset / Method
FragCAM
FragIG
FragDeepLift
3MR Benzene
0.981
0.971
0.934
0.935
0.886
0.875
Mutagenicity 0.793
0.711
0.704
Table 12: Fidelity(sparsity) of FragCAM, FragIG, and FragDeepLift on DrugXAI.
Dataset/Methnn FragCAM
FragIG
FragDeepLift
3MR
0.87().85)
0.87(0.78)
0.13(0.75)
Benzene
0.50(0.67)
0.46(0.64)
0.45(0.64)
Mutagenicity
0.35(0.72)
0.36(0.67)
0.19(0.66)
we incorporate the nested MFP pre-training criterion ("FraFFormer w/o K'). Finally, we enhance the model by adding the knowledge fusion layer ("FragFormer'). The results are shown Table 17 Each component contributes to the performance improvement.
F Results on MoleculeNet
We test FragFormer (DOVE-1) on the MoleculeNet benchmark (Wu et al. 2017). We follow the experimental settings in KPGT (Li et al.) 2023). We split the dataset into 80% training, 10% validation, and 10% test with scaffold splitting. We report the mean performance over three runs with different random splits. The results are shown in Table 18 All baseline results of alom-based models are taken from KPGT. FragFormer outperforms all fragment--aase based baselines and achieves the SOTA performance on 7 out of 11 datasets. We find that FragFormer excels at the target-related tasks, e.g., BBBP, BACE, Estrogen, while underperforms previons SOTA method on toxicity-related tasks, e.g., Tox21, ToxCast. We hypothesize that this could be due differences task characteristics, and we plan to investigate the underlying reasons for this phenomenon in future work.
27

---

Published in Transactions on Machine Learning Research (04/2025)
Table 13: Composition atom features.
Feature Description
Feature Dimension
One hot encoding for the atom type, c-g., C, N,
101
One hot encoding for the atom degree
12
Formal charge
1
One hot encoding for radical electrons One hot encoding for atom hybridization Whether the atom is in aromatic ring
6
6
1
One hot encoding of number of H Whether the atom is chiral center One hot encoding of chirality type normalized atom mass
6
2
1
Table 14: Datasets Profile for PharmaBench.
Dataset Name
Dataset Size
Task Type
Category
Task Description
AMES
9,139
classification
Toxicity
The result of AMES test (Ames et al. mutagenic potential of the molecule.
1973 which evaluates the
BBB
8,301
classification
bbsorption
Predict the Blood-Brain Brain Barrier (BBB) penetration. dataset is labeled according to threshold log BB -1.
CYP2C9
999
regression
Metabolism
Binding affinity to CYP2C9 (unit: log 10pM).. Binding affinity to CYP2D6 (unit: log 10j.M).
CYP2D6
1,214
regression
Metabolism
CYP3A4
1,980
regression
Metabolism
Binding affinity to CYP3A4 (unit: log 10
HLMC
2,286
regression
Clearance
The clearance speed (unit: log 10(mLL /lm/m/mn//)))
liver microsomal system human
MLMC
1,403
regression
Clearance
The clearance speed in the liver microsomal system in mouse (unit: 10mml.min//g),
RLMC
1,129
regression
Chearance
The clearance speed the liver microsomal system rat (unit: log 10(mL. //ii//mm///)))
LogD
13,068
regression
Physochemical
LogD is the logarithm of the distribution coefficient (D), which
measures PH-adjusted lipophilicity of the molecule.
PPB
1,262
rogression regression
Distribution
Percentage of the molecule in the plasma that is bound.
Sol
11,701
Physochemical
Water solubility the molocule (unit: log 10nM).
Table 15: Comparison between different hierarchy of fragmentation.
Classfication (AUROC f)
Regression (RMSE J)
Method/Dutaset Duttaset
AMES
BBB
CYP2C9
CYP2D6
CYP3А4
MLMC
RLMC
LogD
PPB
FragFormer- Atom
U.867
0.916
17.849
16.133
16.663
0.606
0.755
0.718
0.800
0.192
1.045
FragFormer-JTT
0.863
0.919
17.413
15.666
16.603
0.550
0.746
0.711
0.709
0.185
0.938
FragFormer DOVE
0.889
0.927
16.855
14.425
15.894
0.514
0.702
0.596
0.667
0.157
0.895
Table 16: Comparison between different overlapping degree & with the same vocabulary.
Clussfication (AUROC
Regression
(RMSE L)
Method/Datasee
AMES
BBB
CYP2C9
CYP2D6
CYP3A4
HLMC
MLMC
RLMC
LogD
PPB
Sol
FragFormerd
0.858
0.918
17.358
17.241
16.859
0.625
0.854
0.603
0.780
0.193
0.952
TragFormers
0.865
0.922
17.316
14.833
15.773
0.517
0.734
0.565
0.695
0.175
0.910
Frrmeror
0.870
0.920
16.732
14.538
15.302
0.519
0.708
0.548
0.687
0.177
0.905
Former3
0.860
0.917
17.809
16.986
16.910
0.544
0.754
0.577
0.725
0.199
0.959
28

---

Published in Transactions on Machine Learning Research (04/2025)
Table 17: Effect of key components.
Classiiction (AUROC
Regression (RMSE 4)
Method/ Dataset
AMES
BBB
CYP2C9
CYP2D6
CYР3А4
HLMC
MLMC
RLMC
LogD
PPB
Sol
PragFormer-Ateee w.o K.
0.837
0.913
18.261
16.691
17.190
0.621)
0.996
0.937
0.891
0.296
1.387
FragFormer (K&Nested MFP)
0.856
0.921
17.919
15.659
16.891
0.563
0.877
0.774
0.740
0.211
1.134
Former w.o K
018822
0.921
17.851
14.815
16.055
0.512
0.799
0.680
0.706
0.166
0.981
FragFormer
0.889
0.928
16.855
14.425
15.894
0.514
0.702
0.596
0.667
0.157
0.895
Table 18: Results on MoleculeNet. Baselines labeled with 't' are fragment-based methods.
Classification (AUROC 1)
Regression
(RMSE 4)
Method
BACE
BHRP
Clin' Tox
SIDER
Estragen
MetStab
Tox21
ToxCast
FreeSnlv
PSOI.
Lipo
Inforax
Veliekovic et al.
2019
0.839
0.840
0.661
0.616
0.888
0.837
0.816
0.690
4.119
1.462
0.978
Edgepred
Hamilkon es
2017b
0.817
0.873
0.730
0.603
0.881
0.844
0.818
0.712
3.849
2.272
1.030
Masking
al.
2020
0.864
0.729
0.573
0.869
0.868
0.663
3.646
1.063
Contexipred
0.840
0.877
0.732
0.609
0,882
0.857
0.806
0.714
1.349
0.969
Infomax
sup
Fu
2021
0.839
0.873
0.754
0.622
0.864
0.864
0.826
0.713
3.017
1.238
0. 0.729
Edgepred
sup
Hi
21720
0.8:17
0.859
0.715
0.620
0.890
0.852
0.829
0.721
2,889
1.133
0.707
Masking_ssi sup
fiu a
2120
0.824
0.859
0.796
0.606
0.888
0.872
0.827
0.715
3.210
1.387
0. 0.725
Contextpred
stip [1t ct
2020
0.855
0.875
0.802
0.620
0.885
0.859
0.840
0.724
3.105
1.477
0.754
GraphLoG
Xu
al.
2022
0.830
0.846
0.667
0.615
0.871
0.850
0.796
0.677
4.174
2.335
1.104
GraphCL.
al 4120
0.887
0.691
0.587
0.875
0.821
0.696
1.835
JOA0 You
20121
0.826
0.879
0.640
0.861
0.837
0.823
3.466
1.771
0.933
GROVER VER
Rong cl
2020
0.840
0.887
0.874
0.638
0.892
0.876
0.838
0.696
2.991
0.928
0.752
3DInfomax
Stark
2012
0.811
0.877
0.887
0.585
0,880
0.806
0.805
0.716
2.919
1.906
1.045
GraphMVP
Liu el ai
2122
0.818
0.860
0.719
0.584
0.865
0.820
0.799
0.689
2.532
1.937
0.990
MolFormer
Roes et ml 202
0.791
0.866
0.810
0.578
0.806
0.651
0.764
0.687
2 2,322
0.821
0.673
ImageMol
Zeng et al.
2022
0.786
0.881
0.885
0.625
0.839
0.874
0.816
0.710
2.634
1.869
0.765
GEM [Fnng et al A121
0.895
0.905
0.894
0.863
2.389
0.803
0.663
GraphMAE
2022
0.878
0.597
0,881
0.853
1.378
0.746
MoleBERT
Xm
2025
0.843
0.851
0.797
0.615
0.887
0.868
0.832
0.720
2.801
1.185
0.690
KPGT Li al. 2023
0.855
0.908
0.946
0.649
0.905
0.889
0.848
0.746
2.121
0.803
0.600
GraphFP" Luong Sungh
2023
0.839
0.869
0.832
0.598
0.879
0.857
0.813
0.707
3.228
1.805
0.803
FraGAT! Zimug al. 202
b
0.813
0.841
0.726
0.577
0.867
0.841
0.771
0.682
4.049
2,043
0.913
PharmIITT
Jiang al.
2023
0.846
0.891
0.895
0.625
0.892
0.874
0.824
0.715
2.111
1.276
0.779
PragFormerr
0.642
29


Reviews:
[{"review_id": "TbjzKGfbvu", "reviewer": "Reviewer_8aDe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**: Overall the paper is well written with detailed descriptions of their method and comprehensive experiments.\n\n- DOVE overcomes the limitation of other methods being unable to model overlapping fragments using an iterative line graph method. This approach is strongly motivated by the domain and has theoretical justification, preserving the connectivity as well for these overlapping fragments.\n\n- MFP is a well thought out method for self-supervised learning of hierarchical representations where fragments can be nested or overlapping and overcomes the simplified formulation of exclusive fragment membership for atoms in the molecular graph. This method can therefore capture structural information at varied scales as reflected by the diversity of fragment sizes.\n\n- FragCAM is a helpful addition to enhance the attribution of molecular properties to functional fragments instead of atoms. Though its a simple formulation, experiments demonstrate its efficacy.\n\n- Comprehensive experiments are conducted to compare FragFormer with existing molecular graph representation methods and demonstrate its effectiveness. Ablations are also conducted to isolate the improvements resulting from the modifications introduced by this method.\n\n**Weaknesses**: There are a few points which could use some clarification.\n\n- Choice of the hyperparameter **k** in DOVE: While experiments are conducted with k=0,1,2 it is unclear how to choose the best value of k for a given problem. The results in table 1 broadly show that k=1,2 works better than k=0 but there is variation across datasets and sometimes k=1 is better than k=2. While larger values of k may be more expensive computationally, I think it merits additional investigation to establish clear trends that can be used by practitioners to inform their choice of k.\n\n- Fragment vocabulary construction's relationship with k - If a new vocabulary is being built for each k, it can be hard to isolate the improvements in the method to the vocabulary size vs the potentially more expressive graph structure. Ablations along this direction would be appreciated and be helpful to practitioners similar to the previous point.\n\n- Knowledge fusion layer implementation: The choice of using a GRU to aggregate multiple knowledge features is confusing. GRU is sensitive to the ordering of inputs, so a shuffled ordering of the same features would lead to a different result. You may want to replace this layer with one which is permutation invariant like a self-attention layer without any positional encodings.\n\n- FragCAM method: While this method is shown to be effective, CAM is an older interpretation method and there were better alternatives developed including DeepLIFT and Integrated Gradients which overcome some of its limitations. Both these are cited in the paper and are foundational to other graph based interpretation methods, so a justification for CAM (like efficiency / similar performance) would be nice.\n\n- FragCAM evaluation: Table 3 and 4 only report AUROC, but it is critical to include the Fidelity metric (as used in SubgraphX) to accurately measure if the explanations are actually important to the model's predictions.\n\n- Ablations: Given that FragFormer introduces multiple methodological improvements, the ablation studies only highlight the importance of each component as part of this larger system. While these experiments are insightful, it would be more helpful to take a baseline graph representation method and then introduce individual critical components like DOVE / MFP to demonstrate their utility over the traditional representation methods while controlling for vocabulary and pretraining method (if that is not the target of the ablation).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "- This paper introduces a novel method for representing molecular graphs using a k-degree overlapping fragmentation method (DOVE). This new representation aims to address a fundamental shortcoming of previous fragment based representation methods which cannot model an atom as a member of more than one functional fragment.\n\n- To further enhance fragment based representation learning, this paper describes nested masked fragment prediction (MFP) which aims to capture hierarchical relationships since fragments can be nested. The masked prediction task involves predicting all masked substructures, not just one of them, as a self-supervised representation learning objective.\n\n- A complementary model interpretation method, fragCAM is also introduced to help attribute predictions to functional fragments instead of atoms - this can enable a better analysis of which molecular substructures drive property predictions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The weaknesses listed above need to addressed to strengthen the work.\n\nCritical changes: Discussion of the hyperparameter k in DOVE with experiments for larger values to establish a clearer trend and highlight the tradeoff between compute efficiency vs better performance so that practitioners can make an informed choice. Fidelity results for FragCAM are also required for tables 3 and 4.\n\nOther changes: It is strongly recommended to include ablative experiments which study the improvements caused by DOVE/MFP in comparison to baseline method without DOVE and MFP while controlling for all other factors (such as fragment vocabulary).\n\nMinor issues: The cited paper by Shrikumar et al. introduces DeepLIFT, not sure why it is referred to as GradInput in the text. Authors are requested to double check their references for accuracy.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "C23l5JC2Y9", "reviewer": "Reviewer_BNJd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The problem of interest, molecular property prediction, is an important topic in chemistry, e.g., drug discovery.\n\n2. The motivation is reasonable; incorporating hierarchical nature of fragments can improve the understanding of molecules.\n\n3. Fragment-level interpretation seems intuitive; chemical features are encoded in fragments, rather than atoms themselves.\n\nWeakness:\n\n1. I have concerns on evaluation setups. Previous molecular property prediction works mainly utilize MoleculeNet, PCQM4Mv2 QM9 benchmarks. I think that evaluation on those benchmarks is essential to validate the effectiveness of FragFormer.\n\n2. Analysis is weak. The authors claim the importance of non-overlapping fragmentation, but it seems that there is insufficient support on the claim. For example, analysis of fragments in different hierarchy is required.\n\nQuestions:\n\n1. When the fragments are not in the vocabulary, how are they treated?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose FragFormer, a framework for molecular property prediction. The approach consists of (1) k-Degree Overlapping Fragmentation and (2) nested masked fragment prediction. The authors also propose a interpretation technique, FragCAM, to improve the interpretability of molecular representation. The main focus is to fully utilize overlapping fragmentations, which is yet under-explored in previous fragment-based approaches. Experimental results show that FragFormer achieves decent performance in various molecular property prediction tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See Weaknesses and Questions.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "i2IsZLQpfs", "reviewer": "Reviewer_BMQK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\n- The paper is very well-written - related work sufficiently covers the background and motivates the work; The method section is concise and clear. \n- The interpretability at fragment level for molecular property is highly useful contribution to the field; FragCAM report the best results among all baselines. \n- Ablation results sufficiently motivate various design choices. Overlapping fragmentation exhibits substantially superior performance in contrast to the non-overlapping fragmentation - proving the effectiveness of DOVE.\n- FragFormer outperforms exiting methods in most molecular property prediction tasks.\n\nWeaknesses:\n- One the main contribution of this work is proposing overlapping fragmentation of the molecular. However overlapping fragmentation has been purposed previously in JT-VAE in 2018. Bonding information is preserved via overlapping atoms in clusters. DOVE with degree 1- the best performing setting for molecular setting, seem to be produce fragment graph that is identical to JT-VAE’s method. There’s no discussion or ablation against the fragmentation method from JT-VAE in the paper.\n\n\n- It is claimed that one of the advantage for the proposed fragmentation method is that it preserves bonding information in the fragment graph - because there’s overlapping atoms in a pair of connected fragment nodes. However, the model architecture cannot directly access the bonding information - since the Fragment-level Graph Transformer just accesses the fragment embedding as input - which is computed independently for each molecular subgraph. It doesn't seem possible for the model to recover the bonding information between fragments from the subgraph pair.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "- FragFormer is a self-supervised learning framework for molecular representation. It leverage the prior that a molecule is consisted of overlapping molecular fragments. \n- This work develops a new k-Degree Overlapping fragmentation (DOVE) method - to account for overlapping functional groups. \n- The fragment-level interpretation method FragCAM is proposed and outperforms all baseline methods.\n- The framework is proved effective for molecular prediction tasks - reaching state-of-the-art in 8 out of 11 tasks in PharmaBench.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- It would be ideal to see a ablation and discussion on how the overlapping fragmentation method purposed in the method compare to the fragmentation method in JT-VAE. Is the newly proposed fragmentation method essential for FragFormer’s success?\n\n- Clarification on whether the model can directly access and utilize the bonding information between fragments from DOVE. The claim that bonding information is preserved may need to be adjusted.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes FragFormer, a self-supervised framework for molecular representation learning based on molecular fragments and masked fragment prediction. All the reviewers find the paper well-written, technically sound, and supported by comprehensive experiments. As the concerns of the reviewers have been also well addressed, the AE recommends accepting the paper.
Decision: Accept (poster)