$\mathcal{O}$-GNN: incorporating ring priors into molecular modeling
Authors: Jinhua Zhu
Kehan Wu
Bohan Wang
Yingce Xia
Shufang Xie
Qi Meng
Lijun Wu
Tao Qin
Wengang Zhou
Houqiang Li
Tie-Yan Liu
Venue: ICLR 2023 poster (2023)
Keywords: Graph Neural Network
Ring
Molecular Modeling

Abstract: Cyclic compounds that contain at least one ring play an important role in drug design. Despite the recent success of molecular modeling with graph neural networks (GNNs), few models explicitly take rings in compounds into consideration, consequently limiting the expressiveness of the models. In this work, we design a new variant of GNN, ring-enhanced GNN ($\mathcal{O}$-GNN), that explicitly models rings in addition to atoms and bonds in compounds. In $\mathcal{O}$-GNN,  each ring is represented by a latent vector, which contributes to and is iteratively updated by atom and bond representations. Theoretical analysis shows that $\mathcal{O}$-GNN is able to distinguish two isomorphic subgraphs lying on different rings using only one layer while conventional graph convolutional neural networks require multiple layers to distinguish, demonstrating that $\mathcal{O}$-GNN is more expressive. Through experiments, $\mathcal{O}$-GNN shows good performance on $\bf{11}$ public datasets. In particular, it achieves state-of-the-art validation result on the PCQM4Mv1 benchmark (outperforming the previous KDDCup champion solution) and the drug-drug interaction prediction task on DrugBank. Furthermore, $\mathcal{O}$-GNN outperforms strong baselines (without modeling rings) on the molecular property prediction and retrosynthesis prediction tasks.

Published as a conference paper at ICLR 2023
O-GNN: INCORPORATING RING PRIORS INTO MOLECULAR MODELING"
Jinhua Zhu, 'Kehan Wu, Bohha Wang. YYingee Xia, SShufang Xie, ?Qi Meng, 2 Lijuun Wu, Taoo Qin, 'Wnngang Zhou, "Houqiang Li, Tin-Yan Liu 1 University of Science and Technology of China, "Microsoft Research A14Science
Gaoling School of Artificial Intelligence, Renmin University of China
[ftelaahu, wu_201 B) emaiil ust edu. cn, Towwangfy@gg @@@gmi com, {zhwg, lihq]euste. edu. en, "shufangxie@ruc edu. en, 2 (yingce. xia, meg, li lijuwu, taoqin, eyliijemiiin www.mcoo.....ooommmmmmmmoooooome com
ABSTRACT
Cyclic compounds that contain least one ring play an important role in drug design. Despite the recent success of molecular modeling with graph neural net- works (GNNs), few models explicitly take rings in compounds into consideration. consequently limiting the expressiveness of the models. In this work, we design a new variant of GNN, ring-enhanced GNN (O-GNN), that explicitly models rings in addition to atoms and bonds in compounds. In O-GNN, each ring is represented by latent vector, which contributes to and is iteratively updated by atom and bond representations. Theoretical analysis shows that O-GNN is able to distinguish two isomorphic subgraphs lying on different rings using only one layer while conven- tional graph convolutional neural networks require multiple layers to distinguish, demonstrating that O-GNN is more expressive. Through experiments, O-GNN shows good performance 1 1 public datasets. In particular, it achieves state-of- the-art validation result on the PCQM4Mv1 benchmark (outperforming the previ- ous KDDCup champion solution) and the drug-drug interaction prediction task on DrugBank. Furthermore, O-GNN outperforms strong baselines (without model- ing rings) the molecular property prediction and retrosynthesis prediction tasks. The code is released at https: /ttp//gttttb/bt com/ OOCNNNO--NNN.
INTRODUCTION
Cyclic compounds, which refers the molecules that have at least one ring in its system, naturally exist in the chemical space. According to our statistics on 109M compounds from PubChem (Kim al., 2019) which is widely used chemical library, more than 90% compounds have at least one ring. The rings could be small/simple (e.g., the benzene is a six mexberreer carbon ring, and the pentazole is five-member nitrogen ring) or large/complex (e.g., the molecule shown in Figure 1). Rings are important in drug discovery, for example: (1) Rings can potentially reduce the flexibility of molecules, reduce the uncertainty when interacting with target proteins, and lock the molecules to their bioactive conformation (Sun et al., 2012). (2) Macrocyclic compounds, which usually have a ring with more than 12 atoms, play important roles in antibotics design (Venugopal & Johnson, 2011) and peptide drug design (Bhardwaj et al., 2022).
Recently, deep neural networks, especially graph neural networks (denoted as GNN) (Kipf & Welling, 2017; Hamilton al., 2017a), have been widely used in molecular modeling. A GNN takes a graph as input, and messages of different nodes are passed along edges. GNNs have made great success in scientific discovery: (1) Stokes et al. (2020) train GNN to predict growth inhibi- tion of Escherichia coli and find that Halicin is a broad-spectrum bactericidal antibiotic. (2) Shan al. (2022) leverage GNN model the interactions between proteins, and they eventually obtain possible antibodies for SARS-CoV-2. In addition, GNNs are widely used in drug property predic- tion (Rong et al., 2020), drug-tan target interaction modeling (Tomg Altman, 2019), retrosynthesis
"This work was done when Jinhua 7hu, Kekan Wu and Bohan Wang were interns Microsoft Research AI4Science. Correspondence Yingee Xia.

---

Published as a conference paper at ICLR 2023
(Chen & Jung, 2021), etc. However, none of the above work explicitly models the ring information into GNNs. From the application's 's perspective, they miss an important feature for their tasks. From the machine learning's 's perspective, Loukas (2020) points out that existing message-passe passing- based GNNs cannot properly capture the ring information when the product of network width and height is not large enough (see the Table 1 in Loukas (2020)). Therefore, with the classic GNNs, the ring information in compounds is not well leveraged.
To tackle this issue, in this work, we propose new model, ring-enhanced GNN (denoted as O-GNN), that explicitly models the ring information in compound. The O stands for the rings in molecules and is pronounced as "O". Generally speaking. O-GNN stacks I. layers, and each layer sequentially updates edge representations, node representations and ring representations by aggre- gating their neighbourhood information. We mainly use self-attention layer for adaptive message passing, and use feed-forward layer introduce non-linearity to repre- sentations.
OH
NH
O
T6
O
T2
ÖH
OH
T3
We first demonstrate the advantage of O-GNN through theoretical analysis. O-GNN is able to distinguish two isomorphic sub-graphs lying different rings using only one layer (see Figure 2 for the example). On the con- trary, if we remove the ring-modeling components from O-GNN, such a distinguishability would require multi- ple layers (see Section 2.3 for detailed analysis). These results demonstrate that O-GNN is more expressive than conventional graph convolutional networks in the absence of explicitly modeling rings.
Figure 1: Paclitaxel, a compound with 7 simple rings. Kampan et al. (2015) summarized that the intact taxane ring (i.e., r4, F5. r6) and four-membered oxetane side ring (i.c., r7) is essential to induce cytotoxic activity.
We then conduct experiments on 11 datasets from three tasks, including molecular property prediction, drug-drug interaction prediction and retrosynthesis:
(1) For molecular property prediction, we first conduct experiments on PCQM4Mv1, which is to predict the HOMO-LUMO gap of molecules. Our method outper- forms the champion solution of KDDCup tion set (Shi et al., 2022) (note that test set labels are not available). Next, we verify O-GNN six datasets from MoleculeNet (Wu et al., 2018), which is to predict several pharmaceutical related properties of molecules. O-GNN outperforms the corresponding GNN baselines without rings. Finally, we conduct experiments on FS-Mol (Stan- ley et al., 2021). a few-shot property prediction task, and shows that modeling rings can also improve the predic- tion accuracy. (2) For drug-drug interaction prediction, which is to predict whether two drugs interacts with each other, we test O-GNN on DrugBank following the previous settings (Nyamabo et al., 2021; Li al., 2022), and achieve state-of-the-arr results. (3) For retrosynthesis, we apply O-GNN to LocalRetro (Chen & Jung, 2021), a strong GNN-based method for retrosynthesis. On USPTO-50k, our method significantly boosts the accuracy.
H,N
An illustrative example of theoretical results. The three substruc- tures in the red circles are isomorphic. The second and third substructures lie on different rings (a Cyclooctane and an Azocane). regular GNN requires multiple layers to distinguish the three substructures while O-GNN requires only one layer due to the ring represen- tations.
Figure 2:
the valida-
METHOD
2.1 NOTATION AND PRELIMINARIES
Let G (V, E) denote molecular graph, where V and E are the collections of nodes/atoms
and edges/bonds' Let R denote the collection of rings in G. Define V = {vi, 02, v1} and
When the context is clear, we nodes/atoms and edges/bonds alternativey in this work.
2

---

Published as a conference paper at ICLR 2023
= (e where 2s is the i-th atom and Gjj is the bond connecting e and vy. When the context is clear, we use i to denote atom U;, and use e(v;, 1;) to denote edge Eij. (V() denote the neighbors of atom i, (i) = { Cij E E}. Define R = {ri, r2. 12 ) where each r, is simple ring. A simple ring does not contain any ring structure. For example, for the molecule in Figure 3, it has two simple rings as we marked (r1 and r2). The ring 2,, 4, 2,445 6, 7,8, 9, 1) is not simple ring. Let A(e:) and R(e;)) denote the rings that the atom or the bond eij lies on, and V(r) and E(r) denote all the atoms and the bonds lying on ring For example, in Figure 3, [R(vs) = {ri, r23 while R(vg)= = R(ea9) = {ri. r2) while R(e7s) = V(r1) fea. Us, U6 27, 0g } and E(r1) {ess, E36, C67, 678, Cgg, C
A graph neural network (GNN) is usually stacked by several identical GNN layers. Each GNN layer composed of an Aggregate function and an Update function,
h Update (h, Aggregate N())))
where h; is the representation of atom and hi is its updated representation. Different GNNs have different Aggregate functions and Update functions, Details are summarized in Appendix D.
HB-D H&-1) H&-D yb-n)
( H
H
U
input:
Update bond
Update atom
Update ring
Update compound
9
0
8
7
T1
12
2
11
12
12
Figure 3: The workflow of our method.
H
HV
and U denote the representation collec-
tions of bond, atom, ring and the global compound at the 1-th layer.
2.2 MODEL
Our model consists of L identical layers with different parameters. The architecture of each layer is shown in Figure 3. Let ",""," h. and h, denote the output representations of atom Us, bond eij and ring r at the (-th layer, respectively. Let (1)) denote the compound representation at the (-th layer. We initialize (0) via learnable embedding layer which indicates its atomic type, chirality, degree number, formal charge, hybridization type, and so on. Similarly, we initialize n/ with a learnable embedding wich indicates its bond type, stereoisomerism type and whether the bond is conjugated.
Then we initialize (0) by concatenating the node and edge embedding and then transform it with non-linear layer. Last, we initialize the compound representation with a learnable embedding. In each layer, we update representations of nodes, bonds, rings and the compound sequentially. We will frequently MLP(...) ), multi-layer perception network with one hidden layer, build our model. The inputs of MLP are concatenated as a long vector and processed by the network.
(1) Update bond representations: The representation of bond is updated via the connected atoms, the rings that the bond belongs to and the compound representation from the last layer:
h: (1-1) (1-1)) (1-1) EveRleg)) (G-1) (0-11)
h (1-1) + MLP
R(e;))
(2) Update atom representations: We use an attention model to adaptively aggregate bond represen- tations into the centralized atoms. Mathematically,
RM= = Σ a, w.conncatb")) n 1(-"1);
JEN(B
ay x exp(a LeakyReLu(WaA!"
Waconcat(hh-)) (1 11)))))s
(3)
h(D) = n (1-1) + MLP hn. (111) T
Σ 11-1) (.()))
[(ss)
reR(e.)
3

---

Published as a conference paper at ICLR 2023
In (3). the W's are the parameters be learned, and concat denotes concatenating the input vectors as a long one.
(3) Update ring representations: The ring representations updated using MLP networks:
h h +MLe
Σ
Σ
(Q,U)-))
VVEV
eiyE((r)
(4) Update the compound representation:
U = -0--) +LL
U))
hp (0-1)))
(5)
V
E
R
PER
After stacking L O-GNN layers, we get the graph representation by a simple average pooling layer, i.e. hg = 1
M MM n which could be utilized by graph classification tasks. For node classifica-
tion tasks, can add classification head h
2.3 THEORETICAL ANALYSIS
In this section, we compare the distinguishability between standard GNN (without ring representa- tions) and O-GNN, In addition to the notations defined Section 2.1, we define the valued version of graph G (V, E) as a triplet VALUE,(G) = (V, f), where is mapping storing feature information and mapping node or an edge to its corresponding input feature (e.g., 256-dimension representation). We call f as a feature mapping G.
Definition (k-neighbourhood node). For a molecular graph G = (V, E) and two nodes v E V, we say is a k-neeighoourhoo of if there exists a path in G connecting and with length no larger than k. More formally, is k-neighbourhood of if and only if there exists set of such that, S vo v. U2 u, and for any i E {0....,t 1},
nodes {vo, "
04+1 E (wv).
We highlight here that v is a O-neighbourhood node (and thus k-neighhourhood node with any k 2 0) of itself.
(V, =V, and a node in G, we define the k-neighbourhood sub-graph of as the sub-graph composed of all o's k-neighhourhood node. More formally, we slightly abuse the notations and denote the k- neighbourhood sub-graph of as G(v, k) (V(v, E(u, k)), where
Definition (k-neighbourhood sub-graph). For a molecular graph G
((u A {uEV us a k-neighhourhood node of v}.
01 F(U,k) & fe(v1, v2) E E: : E (v,, k)}-
Definition 3 (Equivalent valued graph). For two valued graphs VALUE,, (G1)= (V1, fi) and VALUE,, (G2) = (V2, E2, f2), we say that they are equivalent if (i). G1 and G2 are isomorphic, i.e. there exists a one-to-one mapping P V1 + V2, such that the edges preserved; (ii). P also preserves the value of edges and the the value of nodes, iee, VU,UE U
v(u,v) E Eg + ((P(u),P(u)) E2, = fs(e(P(u),
With all the preparations above, we are now ready to define the graph feature extractor and its discriminatory ability.
Definition 4 (Graph feature extractor and its discriminatory ability). We say mupping & is graph feature extractor, if maps valued graph VALUE, We further allow & to be parameterized as 2a, and call 29 a parameterized graph feature extractor.
new feature mapping f on G.
For parameterized graph feature extractor D., say Da has the discriminatory ability for k- neighbourhood sub- sub-graphs. if for any valued graphs (G, f) and any two nodes u, v G. if the valued k-neighbourhood sub-graph of u and (i.e.. (G(u, k), f) and (G(v, k), f)) are equivalent, there exists 8* such that o ((G, ()) + Ln- ((G. f)) In this case, we also say that Dy- can distinguish and
4

---

Published as a conference paper at ICLR 2023
We point out that ) ( big defined by Eqn. (2, 4, is parameterized feature extractor, and thus above provides a formal definition O-GNN's discriminatory ability.
The next proposition shows that without the ring representation, the O-GNN needs least k layer to have the has the discriminatory power for k-neighbourhood sub-graphs.
Proposition Without the ring presentation, O-GNN with no more than k layers does not have the discriminatory ability for k-neighbourhood sub-graphs.
Note that Proposition 1 can be easily extended to the conventional graph convolutional neural net- works, which only aggregate information from 1-neighborhood nodes. We then show that with the ring representations, O-GNN with only one layer has the discriminatory power.
Proposition If u and v lie on different rings, O-GNN with only one layer can distinguish them.
The proofs are deferred Appendix B due to space limitation. From Proposition 1 and 2, can see that O-GNN more expressive than the regular GNN that does not model rings. The regular GNN requires least & layers to distinguish different rings, while O-GNN only requires one layer for this purpose (see the example in Figure 2). Comparing O-GNN regular GNN with the same number of layers, modeling ring presentations constantly increases the percentages of parameters (irrelevant to k). However, regular GNN may require layers achieve the discriminatory power for 8-neighborhood sub-graphs. When is is large, O-GNN will be much more parameter efficient. More discussions are in Appendix C.5.
isomorphic k-neighborhood sub-graphs
3 EXPERIMENTS
To validate the effectiveness of our method, we test O-GNN on the following three tasks: molecular property prediction, drug-drug interaction prediction and retrosynthesis. The first two tasks are graph classification tasks, and the third one is node/link prediction task.
3.1 APPLICATION TO MOLECULAR PROPERTY PREDICTION
Datasets. We work on three datasets for this application:
(1) The HOMO-LUMO energy gap prediction of the PCQM4Mv1 dataset (Hu et al., 2021). The input is a 2D molecular graph, and the target is its HOMO-LUMO energy gap, which is an essen- tial molecular property in quantum chemistry. PCQM4Mv1 has 3045360 and 380670 training and validation data (test labels are not available). The properties are obtained via density function theory.
(2) Molecular property prediction on MoleculeNet dataset (Wu et al., 2018). This is a dataset about the prediction of pharmaccutical properties of small molecules. We choose six molccular property prediction tasks (including BBBP, Tox21, Cli, HIV, BACE and SIDER), and the data ranges from 1.5k to 41k.
(3) Feo-shot molecular property prediction of the FS-Mol dataset (Stanley et al., 2021). FS-Mol made up of 5120 separate assays extracted from ChEMBL27 (https: www.ebbl ac. ac.uk/ chembl/ /). Each assay has 94 molecular-property pairs average.
Training configuration. For PCQM4My1, we set the number of layers as 12 and hidden dimen- sion as 256, which is selected by the cross-validation method on the training set. For FS-Mol, the number of layers are 6 and the hidden dimension is 256. The candidate number of layers and hid- den dimensions for MoleculeNet the hyper-pas hyper-paramenee parameters are selected according to validation performance. We train all these tasks on one GPU. The optimizer is AdamW (Loshchilov & Hutter, 2019). More detailed parameters are summarized in Table 5 of Appendix A.
{4, 6, 8, 12) and {128, 256). On FS-Mol and MoleculetNet.
Results on PCQM4Mv1 The results of PCQM4Mv1 are reported in Table 1. We compare GNN with the following baselines: (1) Conventional GCN/GIN with/without virtual node (marked by "vn"). The results are from Hu et al. (2021); (2) ConIDSS (Liu et al., 2021). which predicts quantum properties conditioned et al., 2021), which has a regression head and a classification head that learn from each other; (4) Graphormer (Ying et al., 2021; Shi et al., 2022), the champion solution of PCQM4Mv1. Since
low-cost conformer sets; (3) Two-branch Transformer (Xia
5

---

Published as a conference paper at ICLR 2023
Method
MAE (L)
0.130
O-GNN w/o ring
O-GNN
GCN
0,1684
GCN + vn
0.1510
MAR
0.125
GIN
0.1536
GIN + vn
0.1396
ConfDSS
0,1278
0.120
Two-branch Transformer
0,1237
Graphormerbase
0.1193
0.115
GraphormeT
0.1231
O-GNN (ours)
0.1148
Number layers
Table 1: Validation MAE on PCQM4Mv1.
Figure 4: MAE w.r.t numbers of layers.
peolicise ingrrveenn
I-2
Mean predetion Inproveuuet
Lm2
Moan prodectin inpporement
0.04
I-12
0.055
Lmm12
0.0
0.02
0.000
0.01
0.00
0.000
0.00
(0.3)
13.61
16.91
(9.12)
(12,15)
(16.18]
Number tins
Numioc of otoms bring on slngs
Number of abons in the max sing
(a) Number of rings.
(b) Number of atoms lying on rings.
(c) Maximum ring size.
Figure 5: Performance improvement over several ring properties.
the owners of PCQM4Mv1 did release labels of the test set, we can only compare the results on the validation set. The evaluation metric is the mean absolute error (MAE). From Table 1, we can see that O-GNN achieves the best results among the strong baseline models, which shows the effectiveness of our method. In addition, GIN ConIDSS and Graphormer do not explicitly use the ring information, and we will combine O-GNN with the strong methods in the future.
To investigate the significance of the ring information, ring modeling component from O-GNN, and denote this variant as "O-GNN w/o ring". Specifically, it is implemented by removing (4) and all the h, 's in (2335,. We conduct experiments for O-GNN and "O-GNN w/o ring" from 2 to 14 layers. The results are in Figure 4. We can see that by utilizing ring information, the performance is boosted regardless of the number of layers. In addition, we find that a 6-layer O-GNN is comparable with the 12-layer O-GNN w/o ring, which shows the great power of modeling rings GNN. We also have that O-GNN outperforms "O-GNN w/o ring" in terms of the number of parameters (see Figure 10). It is noteworthy to point out that validation MAE of the 14-layer O-GNN slightly drops compared to the 12-layer O- O-GNN. Note that this phenomenon is also observed in Graphormer (Shi et al., 2022) that larger models do not always lead to better validation results. We will explore how to train deeper models in the future.
study variant of O-GNN by removing the
On PCQM4Mv1, we also study the average performance improvement w.r.t. several ring properties. The performance improvement is defined as 6 c2, where 6 and 62 denote the validation MAE of "O-GNN w/o rings" and O--NNN. The ring properties include: (i) the number of rings in molecule; (ii) the number atoms lying on rings; (iii) the number atoms in the largest ring. We conduct experiments for the networks with different numbers of layers (L 2, 6, 12). Results are reported in Figure 5. We can conclude that overall, as the increase of number of rings, maximum ring sizes and the number of atoms lying rings, O-GNN achieves more improvement compared to the variant without modeling rings. More analyses are in Appendix C.4.
Results on MoleculeNet For MoleculeNet, we compare with both pretraining and non- conppretraining methods. For prettratgn methods, we compare with the following baselines: (i) GCN (Kipf & Welling, 2017) with virtual node (ii) GIN (Xu et al., 2018) with virtual node; (iii) O-GNN without using ring information (denoted as "O-GNN w/o ring"). For pre-training methods, select

---

Published as a conference paper at ICLR 2023
Dataset Molecules
BBBP 2039
Tox21
Clin'Tox
HIV 41127
BACE 1513
SIDER 1478
7831
1478
(Hu et al., 2020)
71.2 0.9
74.2 +: 0.8
73.7 =
75.8
78
60.4 0.6 58.4
G-Contextual (Liu et al., 2022)
70.3
75.2
0.3
19.9
75.9
79.2
G-Motif (Liu al., 2022)
3.4
73.2
2.0
73.8
73.4
60.6
GraphMVP (Liu et al., 2022)
72.4 1.6
75.9 0.5
79.1
77.0
8.9
63.9 1.2
GCN + vn
72.7 1.3
75.0
92.0
78
80.0
62.9 1.3
GIN + vn
71.7 0.6
0.6
89.4 3.2
79.3 1.0
82.0 1.0
60.8 - 0.8
O-GNN w/o ring
74.5 1.4
75.2 0.9
90.2 2.1
80.5 1.0
84.2
65.5
O-GNN (ours)
76.4
75.7 U.7
94.3 1.6
8133
85.8
66.2
Table Test ROC-AUC (%) performance of different methods on 6 binary classification tasks from MoleculeNet benchmark. The training, validation and test sets provided by DeepChem. Each experiment is independently run for three times. The mean and standard derivation are reported.
several representative graph-based methods: (i) Hu al. (2020) proposed to predict the masked attributes on graphs as well as maintaining the consistency between a subgraph and its neighbors; (ii) G- G-(Contextual, Motif) are variants of (Rong et al., 2020), which provided in Liu et al. (2022). (iii) GraphMVP (Liu et al., 2022), which is a joint pre-training between 2D molecules and its 3D conformation. The results of (Hu al., 2020), G- 6--(Cottettual, Motif) and GraphMVP all extracted from Liu et al. (2022), since Liu et al. (2022) use the same scaffold-based splitting as us.
The results are reported in Table We see that: (i) O-GNN outperforms the conventional network architectures like GIN and GCN with virtual nodes, which demonstrate the effectiveness of our new architecture; (ii) When comparing with G-(Contextual, Motif). GraphM (Liu al., 2022) and Hu et al. (2020) which are all pre-training methods, O-GNN still outperforms those methods. (More discussion about pre-training methods are left in Table 11 of Appendix C.5) This shows the great potential of O-GNN and we will combine it with pre-training in the future. (iii) Comparing O-GNN and O-GNN w/o ring, the average improvement overall the six tasks is 1.6. This shows the advantage of using ring information in molecular property prediction.
Results on FS-Mol Stanley et al. (2021) verify that prototyp- ical networks (PN) performs the best on FS-Mol compared with other methods like MAML (Finn et al., 2017), multi-task learning (MT) and random forest (RF). Stanley al. (2021) use Transformer-Ie
residual network for few-shot classifi- cation. We replace that backbone our O-GNN and "O-GNN w/o ring", and the other parts remain unchanged. Following Stanley et al. (2021), the results with different support set sizes (denoted as [Tu..upporr)) 1) are reported. A support set consists of few examples with input-label pairs used train models. The evaluation metric is AAUPRC, which is the difference be- tween the AUPRC (area under the precision-recal curve) and the ratio of the active compounds in that query set. A higher A AAPRRC score indicates better classification performance of the model.
11.55
0.20
PX
0.10
22
24
Figure Results on FS-Mol.
The results are Figure We report the mean and the standard derivations for different tasks across various support sizes. We have the following observations: (i) By using O-GNN as the backbone model for the prototypical network. the results are boosted for different support set sizes. (ii) The improvement is more significant when the support set size is large. When Tu.eupportl 128 and 256, the improvements are 0.014 and 0.016. When reducing the sizes to 16/32 332/66 64, their improvements are all around 0.008. We will further improve the results on limited data size in the future.
3.2 APPLICATION TO DDI PREDICTION
Drug-drug interaction (DDI) prediction is to predict therapeutic output of taking two drugs together, like increasing the risk of some side effects, the effect is enhanced to take them together. We

---

Published as a conference paper at ICLR 2023
focus on the classification task, where the inputs two drug molecules and one interaction (e.g., inhibition), and the output is 0 or 1 to indicate whether the two drugs have this specific interaction. Following Nyamabo et al. (2022) and Li et al. (2022), we work the inductive setting of the DrugBank dataset (Wishart et al., 2018), which has 1, 706 drugs, 86 interaction types, and 191, 808 triplets. To test the generalization ability of the model, we conduct experiments on two settings w.r.t. the drugs: the SI setting, where neither of the two drugs on the test set appears in the training set; the S2 setting, where one drug is seen in the training set and the other is not. Noting that the drug pairs in the test set do not appear in the training set. Hence, the DrugBank data is split into training and test sets by the visibility of the drugs, and the negative samples are offline generated. We directly use the data provided by Nyamabo et al. (2021; 2022). where 20% drugs are first hold as unseen drugs for formulating test set and the rest 80% drugs are used to create the training set.
S2 setting (1 known drug I unknown drug)
S1 setting (2 unknown drugs)
Method
ACC
AUROC
AP
F1
ACC
AUROC
AP
GAT-DDI (Nyamabo ct al., 2021)
69.83
77.29
75.79
73.01
62.63
70.92
73.01
45.81
MHCADDI (Deac et al., 2019)
70.58
77.84
76.16
72.74
65.40
73.43
75.03
54.12
MR-GNN (Xu et al., 2019)
74.67
83.15
83.81
69.88
66.50
72.53
71.06
67.21
SSI-DDI (Nyamabo et al., 2021)
76.38
84.23
84.94
73.54
66.31
72.75
71,61
68.68
GMPNN (Nyamaho et al., 2022)
77.72
84.84
84.87
78.29
68.57
74.96
75.44
65.32
MSAN-GCN (Zhu et al., 2022)
77.81
85.74
76.48
69.17
76.12
67.10
MSN-DDI (Li ct al., 2022)
81.92
91.01
91.09
80.18
73.42
81.79
81.82
70.34
O-GNN w/o ring
87.72
94.51
95.28
85.91
75.47
83.83
85,58
65,59
O-GNN (ours)
88.47
95.87
96.51
86.91
76.81
87.64
88,70
70.81
Table 3: Results of drug-drug interaction prediction on DrugBank.
To predict the interaction between two drugs, we use one 6-layer O-GNN to extract features for the two drugs. Specifically, for each drug, we average the node representations output by the last layer as the drug feature. We concatenate the two drug features together, and then multiply the interaction embedding to do the prediction. The detailed parameters are left in Table 6 of Appendix A.
The results are reported in Table 3. O-GNN significantly outperforms previous baselines in terms of accuracy (denoted as ACC), the area under the receiver operating characteristic (AUROC), the average precision (AP), and the Fl score. Most of previous works use GCN, GIN or GAT backbones, and they focus on designing comprehensive interaction module (Nyamabo et al., 2021: Li et al., 2022). By using the advanced O-GNN backbone, we can significantly improve the results without designing complex interaction modules. This shows the effectiveness of our method.
3.3 APPLICATION TO RETROSYNTIESIS
Retrosynthesis is to predict the reactants of a given product. Various GNNs have been applied to this task. For example, GLN (Dai et al., 2019) use GNNs to predict the distributions of candidate reaction templates and reactants. GraphRetro (Somnath et al., 2021) and G2G (Shi et al., 2020) use GNNs to predict where to break a bond and how to add the fragments to complete the syn- thons. To demonstrate the ability of our O-GNN, we combine our method with LocalRetro (Chen & Jung, 2021), the current best graph- based model for retrosynthesis (without using pre-train LocalRetro uses GNN predict the possible templates for each atom and each bond, and sort the predicted templates according to their probabilities. The top templates will be applied to the corre- sponding atoms or bonds via RDKit (Landrum al., 2016) generate the reactants. Chen & Jung (2021) use MPNN (Gilmer et al., 2017a) for prediction, and we replace the MPNN with O-GNN. We conduct experiments on the USPTO-50k dataset (Coley et al., 2017) that contains 50, 016 re- actions. Following Chen & Jung (2021), we partition the dataset as 45k training set, 5k validation set and 5k test set. The evaluation metric the the top-k accuracy, where = 1, 5, 10, 50. The results are summarized in Table 4. We can observe that O-GNN can predict reactions more accu- rately than the baselines without ring information. Especially, when the reaction type is known, we improve the 1 accuracy for 1.8 points and the top-3 accuracy for 1.6 points. These results show the importance of modeling ring structure and the effectiveness of our method.
The performance for different number of rings. To study the prediction performance of molecules with different number of rings, we group the USPTO-50k test set by the number of rings in the product molecules and compute the top-1 accuracy for each group. More specifically, divide
8

---

Published as a conference paper at ICLR 2023
Reaction type unknown
Reaction type known
Method
Top-1
Top-3
Top-5
Top-10
Top-50
Top-1
Top-3
Top-5
Top-10
Top-50
G2G
48.9
67.6
72.5
75.5
61.0
81.3
86.0
88.7
GI.N
52.5
69.0
75.6
83.7
92.4
64.2
79.1
85.2
90.0
93.2
GraphRetro
53.7
68.3
72.2
75.5
63.9
81.5
85.2
88.1
LocalRetro
53.4
77.5
85.9
92.4
97.7
63.9
86.1
92.4
96.3
97.9
O-GNN (ours)
54.1
77.7
86.0
92.5
98.2
65.7
87.7
93.4
96.9
98.3
Table Results on USPTO-50k datasets with reaction type known/unknown.
Loca Reto ouppi
O-GNN odge (graux) buth
20.0
07.4
67.0m
Locd aniu
O-CAN atar
0 D
61.0
07.9
53.0
Retm 3
D-GNN adpu
g
Duu
51.
(a) Accuracy w.r.t #rings.
(b) Case study molecule with complex rings.
Figure 7: Study of O-GNN retrosynthesis task. (a) The top-1 accuracy w.r.t. number of rings in product molecules. (b) The one-step retrosynthesis prediction of product molecule with five rings. The first O-GNN output is the same as the ground truth (marked green).
the test set into four groups with ring numbers [0, 2), (2,4), [4, 6). [6, 12). and those groups have 808, 2347, 1617, 235 reactions, respectively. The results in Figure 7(a) where the blue bars represent the LocalRetro baseline and the green bars represent O-GNN. The results show that O- GNN has better accuracy on all groups, where the improvements are 0.99, 0.85, 1.30 and 5.96. Overall speaking, the improvement is larger when there are more rings in molecule. Especially, when there are least rings in group (i.e., the last column), O-GNN increases the accuracy for 5.96 points, demonstrating that our method can better leverage ring structures.
Case study. In Figure 7(b), we show an example prediction of a product molecule with rings. The reactions in the left panel are the top-3 predictions from LocalRetro baseline and the ones the right are from O-GNN. Our method successfully predicts the correct reactants in its first output (marked green), but the baseline fails to give a correct prediction. More importantly, the baseline system even fails to identify the correct bond to change. These results suggest that modeling ring structures is crucial to predict reactions accurately, and O-GNN is effective algorithm for retrosynthesis.
CONCLUSIONS AND FUTURE WORK
In this work, we propose new model, ring-enhanced GNN (briefly, O-GNN) for molecular mod- eling. We explicitly incorporate the ring representations into GNN and jointly update them with atom and bond representations. We provide theoretical analysis O-GNN and prove that by using O-GNN, the node representations are more distinguishable than the variant without using ring repre- sentations. We conduct experiments on molecular property prediction, drug-drug interaction (DDI) prediction and retrosynthesis. O-GNN outperforms strong baselines these tasks and achieves state-of-the-art results on the validation performance PCQM4Mv1 and DDI prediction. For fu- ture work, first, we will combine with pre-training obtain stronger O-GNN. Second, need to further improve our model when the training data is very limited (e.g., when the support set size 16 fewer). Third, how to efficiently identify and incorporate the representations with more complex structures is another interesting direction to explore. Fourth, we will apply our model to more real world scenarios, like the synthesis and generation of natural products with large rings.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
This work was supported in part by NSFC under Contract 61836011, and in part by the Fundamental Research Funds for the Central Universities under contract WK33490000077.
REFERENCES
Ravichandra Addanki, Peter Battaglia, David Budden, Andreea Deac, Jonathan Godwin, Thomas Keck, Wai Lok Sibon Li, Alvaro Gonzaez, Jacklynn Stott, Shantanu Thakoor, and Petar Veličković. Large-scale scale graph representation learning with very deep gnns and self-supervision. arXiv preprint arXiv:2107.0942 2021.
Gaurav Bhardwaj, Jacob O'Connor, Stephen Rettie, Yen-Hua Huang, Theresa A. Ramelot, Vikram Khipple Mulligan, Gizem Gokce Alpkilic, Jonathan Palmer, Asim K. Bera, Matthew J. Bick, Maddalena Di Piazza, Xinting Li, Parisa Hosseinzadeh, Timothy W. Craven, Roberto Tejero, Anna Lauko, Ryan Choi, Calina Glynn, Linlin Dong, Robert Griffin, Wesley C. van Voorhis, Jose Rodriguez, Lance Stewart, Gaetano T. Montelione, David Craik, and David membrine-traverrinn
Baker. Accurate de novo design of membrane- traversing macrocycles. Cell, 185(19) 3532.e26, 2022. ISSN 0092-8674, doi: / //www. sci encedi rect. com wwcienne//atiicl/eppppieee/e///e
URL https: httpc//eeeppee/002//////02000000
Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks? arXiv preprint arXiv:2 105. /449/, 2021.
Shuan Chen and Yousung Jung. Deep retrosynthetic reaction prediction using local reactivity and global attention. JACS Au. 1(00)1612-1620, 2021. doi: 10. 1100////aaau..c0022220
Connor W. Coley, Luke Rogers, William H. Green, and Klavs F. Jensen. Computer- Assisted Ret- rosynthesis Based on Molccular Similarity. ACS Central Science, 3(12)11277-1245, Decem- ber 2017. ISSN 2374-7943, 2374-7951. doi: 10. hat.a//cttsii.bb00335. //pubs.... acs wwt.orgg//ooooo doi/ /10 1tt.://.....1///seeeeeeeeee o2tp////ce/sscciiiicccccccaccccc 7b00355.
URL https:
Weilin Cong, Morteza Ramezani, and Mehrdad Mahdavi. On provable benefits of depth in train- ing graph convolutional networks. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL https: //openzeview
Hanjun Dai, Chengtao Li, Connor Coley, Bo Dai, Le Song. Retrosynthesis prediction with conditional graph logic network. In Advances in Neural Information Processing Systems, pp- 8870-8880, 2019.
Andreea Deac, Yu-Hsiang Huang, Petar Veličković, Pietro Liò, and Jian Tang. Drug-drug adverse effect prediction with graph co-attention. arXiv preprint arX1v:1905.00534, 2019.
Jörg Degen, Christof Seggschiid-Gerlaee Andrea Zaliani, and Matthias Rarey. On the of compil- ing and using' drug-like' chemical fragment spaces. ChemMedChem:: Chemistry Enabling Drug Discovery, 3(10): 550- 2008
Xiaomin Fang, Lihang Liu, Jieqiong Lei, Donglong He, Shanzhuo Zhang, Jingbo Zhou, Fan Wang. Hua Wu, and Haifeng Wang. Geometry-enhanced molecular representation learning for property prediction. Nature Machine Intelligence, 4(2): 2022.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. agnes-ag meta-learning for fast adaptation of deep networks. Doina Precup and Yee Whye Teh (eds.). Proceedings of the 34th International Conference on Machine Learning. volume 70 of Proceedings of Machine Learning Research, pp- 1126-1135. PMLR, 06-11 Aug 2017.
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry, 2017a. URL https: arxiv. org/ www/aa///. 01 01212.
10

---

Published as a conference paper at ICLR 2023
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp- 1263-1272. PMLR, 2017b.
Will Hamilton, Zhitao Ying, and Jure Leskovee. Inductive representation learning on large graphs. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017a. URL https: httpt::/ppocceeeeeeeeeeee (proceedingg neure neuri ps. cc/ cctppaaper/211//f1ee Paabee99PPpe--- -df..
5dd9db55 9c
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017b.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitmik, Percy Liang. Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In International Confer- ence on Learning Representations, 2020. URL https://p//......t /reeeetetttttteeeeeeeeeeeeooooo HJIWWJSFDH.
net / forum?ic iid=
Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, and Jure Leskovec. Ogb-Ise: A large-scale challenge for machine learning on graphs. arXiv preprint arXiv:2103.09430, 2021.
Nirmala Chandralega Kampan, Mutsa Tatenda Madondo, Orla M McNally, Michael Quinn, and Magdalena Plebanski. Paclitaxel and evolving role in the management of ovarian cancer. Biomed Res. Int., 2015:41300
June 2015.
Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Ben- jamin A Shoemaker, Paul A Thiessen, Bo Yu, Leonid Zaslavsky, Jian Zhang, and Evan E Bolton. PubChem 2019 update: improved access to chemical data. Nucleic Acids Res., 47(01):D1102- D1109, January 2019.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. ICLR, 2017.
Greg Landrum et al. Rdkit: Open-source cheminformatics software, 2016. URL http://www. rdkit, org/. https://github. com/rdkit/rdkir, 149:150, 2016.
Guohao Li, Chenxin Xiong, Ali Thabet, and Bernard Ghanem. Deepergen: All you need to train deeper gens, 2020.
Junying Li, Deng Cai, and Xiaofei He. Learning graph-level represeentionoooo for drug discovery. arXiv preprint arXiv:1709, 171,001109,,741,,, 2017.
Zimeng Li, Shichao Zhu, Bin Shao, Tie-Yan Liu, Xiangxiang Zeng, and Tong Wang. Multi-view substructure learning for drug-drug interaction prediction, 2022. URL wtt.httss//aaxii.oooo abs/ 220.3.203.22 14513.
Meng Liu, Cong Fu, Xuan Zhang, Limei Wang, Yaochen Xie, Hao Yuan, Youzhi Luo, Zhao Xu, Shenglong Xu, and Shuiwang Ji. Fast quantum property prediction via deeper 2d and 3d graph networks, 2021. URL https: /ttp///....oa///o stanford. www/ppppee/kcccup22222 htwp://22220/ppma2m11/ DIVE. pdf.
Shengchao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. Pre-training molecular graph representation with 3d geometry. In International Confer- ence Learning Representations, 2022. URL https:: //tpp//v////peeiieeww.. net ienpenrrriee.nee/fouunieeeee 1OKe1p
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2019. URL https: /ttp:///////eeeeeeeee/me net fontt/formmiit
2kg6Ricqq"
Andreas Loukas. What graph neural networks cannot lear: depth width. In International Confer- ence on Learning Representations, 2020. URL https:/ http///ppnneee..eeeee net wtt.:/wrreteeeeeeeeeee wtr.mmrrrmiicoommmmmm
B1 12b44YWS.
11

---

Published as a conference paper at ICLR 2023
Arnold K Nyamabo, Hui Yu, and Jian- Yu Shi. Ssi-ddi: cesstructure-substuuutctte interactions for drug- interaction prediction. Briefings in Bioinformatics, 2021.
Arnold K Nyamabo, Hui Yu, Zun Liu, and Jian-Yu Shi. Drug-drug interaction prediction with
learnable size-adaptive molecular substructures. Briefings in Bioinformatics, 2022.
Trang Pham, Truyen Tran, Hoa Dam, and Svetha Venkatesh. Graph classification via deep learning with virtual nodes. arXiv preprint arXiv:1708.043557 2017.
Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang- Self-supervised graph transformer on large-scale molecular data. Advances in Neural Information Processing Systems, 13:12559-12571, 2020.
Sisi Shan, Shitong Luo, Ziqing Yang. Junxian Hong, Yufeng Su, Fan Ding, Lili Fu, Chenyu Li, Peng Chen, Jianzhu Ma, Xuanling Shi, Qi Zhang, Bonnie Berger, Linqi Zhang, and Jian Peng. Deep learning guided optimization of human antibody against sars-cov-2 variants with broad neutralization. Proceedings of the National Academy of Sciences, 119(11), 2022. doi: 10.1073/ pnas.212295411
Chence Shi, Minkai Xu, Hongyu Guo, Ming Zhang, and Jian Tang. A graph to graphs framework for retrosynthesis prediction. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 2818-8827, PMLR, 13-18 Jul 2020.
Yu Shi, Shuxin Zheng, Guolin Ke, Yifei Shen, Jiacheng You, Jiyan He, Shengjie Luo, Chang Liu, Di He, and Tie Yan Liu. Benchmarking graphormer on large-scale molecular modeling datasets, 2022. URL https: htt/axiivv.o.. org/ abs 2200. 04810.
Vignesh Ram Somnath, Charlotte Bunne, Connor Coley, Andreas Krause, and Regina Barzilay. Learning graph models for retrosynthesis prediction. In M. Ranzato, A. Beygelzimer, Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 9405-9415. Curran Associates, Inc., 2021.
Megan Stanley, John F Bronskill, Krzysztof Maziarz, Hubert Misztela, Jessica Lanini, Marwin Segler, Nadine Schneider, and Mare Brockschmidt. FS-mol: A few-shot learning dataset of molecules. In Thirty-fitth Conference Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL https: httts::/openreviiee http///ppnwweee.eeeeeeeeeeee.eep net forum?ids 701FtuyLlAd.
Jonathan M. Stokes, Kevin Yang. Kyle Swanson, Wengong Jin, Andres Cubillos-Ruiz, Nina M. Donghia, Craig R. MacNair, Shawn French, Lindsey A. Carfrae, Zohar gloom-Ackermann, Vic- toria M. Tran, Anush Chiappino-Pepe, Ahmed H. Badran, Ian W. Andrews, Emma J. Chory, George M. Church, Eric D. Brown, Tommi S. Jaakkola, Regina Barzilay, and James J. Collins. A deep learning approach to antibiotic discovery. Cell, (00()):688-722ee)) 2020. ISSN 0092-8674. doi: https://doi.orge com/ wtt.:/o000011.10...o....oo 1ell1000....l..220.0000.0 021. URL https: : www.. sci wecieenneiireee rect.
Hongmao Sun, Gregory Tawa, and Anders Wallqvist. Classification of hooppng approaches. Drug Discov. Today, 1777-8)3310322 April 2012.
Ruoxi Sun, Hanjun Dai, and Adams Wei Yu. Does GNN pretraining help molecular representation? In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https: /oppr//iiepeeeeeeeeeeeeeee net / forum? id=uytgM9N0v12,
Wen Torng and Russ B. Altman. Graph convolutional neural networks for predicting drug-target interactions. Journal of Chemical Information and Modeling, 59(10):4131-4444 2019. doi: 10. 10.102//acxjccim.b00006
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv: 17/0. 10903, 2017.
12

---

Published as a conference paper at ICLR 2023
Anilrudh A. Venugopal and Stuart Johnson. Fidaxomicin: A novel macrocyclic antibiotic approved for treatment of clostridium difficile infection. Clinical Infectious Diseases, 54(4):568 5774, 12 2011. ISSN 1058-4838. doi: 10.1093/cii//irr83.. URL https: /ttp::://dooioooo wtwp::.......................... cir000.
David S Wishart, Yannick D Feunang, An C Guo, Elvis Lo, Ana Marcu, Jason R Grant, Tanvir Sajed, Daniel Johnson, Carin Li, Zinat Sayeeda, al. Drugbank 5.0: major update to the drugbank database for 2018. Nucleic acids research, 46(DI))D1004--11022 2018.
Zhengin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S. Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learning. Chemical Science, 9:513-530, 2018. //dx. 1rri.orrg10.100000
doi: 10.70/2477A URL http:
Yingce Xia, Jinhua Zhu, Lijun Wu, Yang Fan, Shufang Xie, Yutai Hou, and Tao Qin. When trans- former meets graph neural networks. 2021. URL https: ogb. stanford. htw.foor.ddduu/ppprrr.ee/ kkddup2021 http0///1pppmmmmmm AeeeeeeenGGeeerrrr pdf.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful graph neural networks? arXiv preprint arXiv:1810.00826. 2018.
Nuo Xu, Pinghui Wang, Long Chen, Jing Tao, and Junzhou Zhao. Mr-gan: Multi-resolution and dual graph neural network for predicting structured entity interactions. arXiv preprint arXiv:1905.09558, 2019.
Chengxuan Ying. Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie- Yan Liu. Do transformers really perform badly for graph representation? In Advances in Neural Information Processing Systems, volume 34, 28877-28888 2021.
Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Chee-Kong Lee. Motif-based graph self- supervised learning for molecular property prediction. Advances in Neural Information Process- ing Systems, 34:15870-15882, 2021.
Xinyu Zhu, Yongliang Shen, and Weiming Lu. Molecular substructure-aware network for drug-drug interaction prediction. CIKM, 2022.
A DETAILED EXPERIMENT CONFIGURATIONS
The hyperparameters for the molecular property prediction, drug-drug interaction prediction and retrosynthesis are summarized in Table 5, Table and Table 7 respectively.
PCQM4Mv1
FS-Mol
MoleculeNet
Number of Layers
12
6
[4, 6, 8, 12) {128, 256}
Hidden dimension
256
256 AdamW
Optimizer
AdamW
AdamW
Dropout
0.0
{0.0, 0.1, 0.2) {0.0001, 0.0002, 0.0003)
{0.0, 0.1, 0.2, 0.3, 0.5)
Learning rate
0.0003
{0.00005,0 1.00005500000000 1.0001 0.00002.0.0005)
Training steps
300 epochs
10000 iterations
(50, 100} epochs
Batch size
512
16 {0.01,0.1 0.1} Cosine
(32,64)
Weight Decay
0.1
0.01
Learning Rate Decay
Cosine
Linear
Table 5: Detailed hyper-parameters for molecular property prediction tasks.
13

---

Published as a conference paper at ICLR 2023
Number of Layers
Hidden dimension
512
Optimizer
Adam
Dropout
{0.2, 0.5)
Learning rate
0.0003
Training steps
100 epochs
Batch size
128
Weight Decay
0.01
Learning Rate Decay
Cosine
Table 6: Detailed syper-parameters for drug-drug interaction prediction.
Number of Layers
Hidden dimension
512
Optimizer
AdamW
Dropout
0.1
Learning rate
0.0003
Training steps
200 epochs
Batch size
64
Weight Decay
0.1
Learning Rate Decay
Cosine
Table 7: Detailed haperepas for retrosynthesis.
B PROOFS OF THE TWO PROPOSITIONS
Proof of Proposition I. We start the proof by explicitly writing down the ring-free variant of O-
GNN.
Specifically, the bond representations are given by
(1-1) + MLP (1 (1-1) 11 11 11-1), ((0-)))
(1-1)
The atom representations are given by
n,60
Σ
aj @,W.cenca((h.. h (1-1));
JEN())
0j x exp(a LeakyReLU (1-1) + Waconcat(h) (1-1) " M));
(6)
6 n1 ((-1)) ML. (1 (())) (()-)))
The compound representations are given by
U = ((1-1) MLP
(1) h ((-1)
E
Given the above notations, Proposition 1 can then be translated the following claim:
Claim. For any valued graphs (G, f) and any two nodes Va, in G, if the valued k-neighbourhood sub-graph of and (i.e.. (G(ua, k), f) and (G(vs, k), f)) are equivalent, we have that h hgg for any E f1. }} holds regardless of the parameter.
We denote the equivalent mapping between (G(v,, k), and (G(v,,)) f) as P. We will slightly abuse the notations by letting P(i) if P(x:) V..
We will prove the above claim by induction. Specifically, we will prove that for any for (0. 1.... ... k). we have My, = hp((e) for any Ve, E V(va, h - l), and Me ((ee))( any uo Veg V(va, 1).
14

---

Published as a conference paper at ICLR 2023
Base case: for 1 = 0. by the definition of f, we have f(ve)) f((((v)) )) for every Uay E V(ua: k) and f(Ve, Ve,) = (((()e,) P(Uc.)) for every Ve, Vos E (e,, k). The claim immediately follows h = f(v,,) (()))) (P }), 22ge2 f(Ves Voz) and h ((e)))(())) = =(P(ci)) (((z))))((z))
Induction step: suppose the claim is true for I = (0,... k 1}. Then, for i + 1, we have that for every Vag Vog E (Va,, -1).
(0 01
(h--") + MMLE(K 11-1) (1-1) /1102 U(l-!))
( 1 P(o)))(((2) + MLP (h P(())) P(o))) U(-1)
(1-1) (111) 11-1)
=1 P(c))(())))
(8)
where (+) is due the induction hypothesis. as Vey, Vez E V(ua, k- k 1)).
Similarly, for every Ve, E ((va, k - -(),
h0 = Σ
ay ay h
( 1-11)
JEV(a))
(0)
Σ
(1) 0g www.conaat((77...))a. "() (
jEN(e)
Σ
(P6) WPcnncat ( P()) (
jEN(e)
ayW,concat 11 jEN(a)
Σ
P((s)) 11 1-11)
P(ss)"
where Eq. (o) is due the induction hypothesis and Eq. (8). Eq. (o) is due to
aj x exp((a T LeakyyReU ((- + (k (1-1) ()))
(1-1) 10) exxp(a LeakyReLU(W, (t-1) Waconcat (b) ((())))))))
and thus aj = ap() for any (cc)..
We then have
h =h(-11) + MLP (((-1) 1,, v(-1))
= ((-)) MLP
(14-1) hT((o))= U((-1))
p(a)'
Thus, the claim holds for + 1, and the proof for the induction claim completes. Thus, the claim true for every 1 {0, ........) k))
For every E ... k} u e Gi(va. 0) c. G(va, k - (). Therefore, we have h0 11 = h and the proof is completed.
(1)
Proof of Proposition 2. For two equivalent valued sub- sub-graph (G(ua, k). and (G(u), k), f). v2 and V/ lie on different rings, have
Σ 5o +
R(va) rER(
R(vb)] (ER())
15

---

Published as a conference paper at ICLR 2023
Therefore, there exists choice of MLP, such that
1 =h MLP 1 h
1
Σ
1 )
R(wa))
rER(Ua)
b (0) + MLP 0 (1))
Σ
14 1 U(
((bb)
rER((a)
(2)
bbb
The proof is completed.
C MORE ABLATION STUDY
C.I NODE REPRESENTATION POOLING V.S. COMPOUND REPRESENTATIONS
We explore the difference between using average pooling hig = 1 representation U1L) for classification. We try networks with different numbers of layers (L and 12). We conduct experiments on PCQM4Mv1 dataset. The validation mean absolute errors (MAE) are reported in Table We can see that using average node pooling is better than using compound representation. This is consistent with the discovery of using virtual node in GIN (Hu al., 2021). A virtual node can regarded as compound representation, which connects to all nodes in graph. When using virtual nodes, it is common practice to use the average or sum pooling node representations to represent a graph. One can refer to https: hithhb.thhhu...gggg. com/ snap http-staaforrd//ggggggg angge-staatforr/ooo/o///////2172 2ttp://06..222222222222222222222 examples /lsc/
Mn 2. and the compound
hatmpp//es/sscc/pcmm//////////// http://cpmmmm/mm//m 16#160 for the detailed implementation.
L =
L = 12
Average node pooling
0.1171
0.1149
Compound representation
0.1196
0.1167
Table 8: Comparison between using average node representations VS compound representations.
C.2 ATTENTIVELY AGGREGATE THE INFORMATION FROM RINGS
In (4), we concatenate the sum pooling of atom representations, the sum pooling of bond repre- sentations and compound representations to update ring representations. An alternative solution is to use attention models to aggregate the atom and bond representations. We study a variant which updates the ring representations as follows:
1 = 11 + MLP [ (1-1))
Σ (EV(()
n ai
Σ (iEE((r
B ( vQ-1)),
(9)
In (9).
a exp (Wgihf()) Wnih!() and D exp (Wrphf( + Waah! (1)
(10)
where the four W's are parameters to be learned. The results are reported in Table 9, We can see that although our method is simple, it can effectively leverage the ring information, and outperform this attention-basee variant.
C.3 O-GNN WITH BRICS
The ring representation used in our method could be considered as a special motif. One might wonder whether other types of new motifs would be helpful. To the effect, (Degen et al., 2008) to decompose molecules into fragments. BRICS designs 16 rules to break bonds that can match set of chemical reactions. The ring representations in (2,, replaced by
BRICS model
16

---

Published as a conference paper at ICLR 2023
L=
I. 12
O-GNN O-GNN with attention models when updating ring representations
0.1171
0.1149
0.1179
0.1160
Table 9: Comparison between our method and using attention models when updating ring represen- tations.
2
L = 4
L = 6
L =
L = 12
O-GNN O-GNN w/o rings BRICS
0.1247
0.1201
0.1181
0.1172
0.1
0,1325
0.1243
0.1222
0.1221
0.1204
0.1294
0.1239
0.1219
0.1208
0.1193
Table 10: Comparison between using simple rings (i.e., our method) and using BRICS- based frag- ments.
these motif representations. The remaining parts remain unchanged. We conduct the experiments on PCQM4Mv I dataset, and the results shown Table 10. Due time and computation resource limitation, all the models are trained for 200 epochs.
we can conclude that: (1) using simple ring representations achieves better results than using BRICS; (2) in general, using BRICS is better than the variant without using any ring or ring-based fragmentation information. We will keep exploring more segmentation methods.
From Table
C.4 MORE COMPARISON BETWEEN O-GNN AND O-GNN w/O RINGS
As complementary to the analysis of the MAE w.rt. the number of rings in molecules in Figure 5, also report the predicted error (i.e., mean absolute error, MAE) of O-GNN and the variant "O- GNN w/o rings" in Figure 8. We can observe that when molecules have no rings, the two methods perform similar. As the number of rings increases from 1 to 6, the MAE increases, and O-GNN always outperforms the "O-GNN w/o rings" variant.
C.5 ADDITIONAL DISCUSSIONS
About over-smoothing One might be curious that since we build 12-layer network, whether it suffers from over-smoothing. Actually, Cong et al. (2021) point that "over-smoothing does not necessarily happen in practice, deeper model is provably expressive, converge global op- timum with linear convergence rate, and achieve very high training accuracy as long as properly trained." (The words from (Cong al., 2021) for accurate expression). In addition, Li et al. (2020) and Addanki et al. (2021) both successfully trained 50+ layer networks. Our method follows
O-GNN rings
rags
0.5
w/lo rings O-GNN
0.3
O-GNN
O-GNN
MAR
MAR
0.1
0.1
0.0
0.0
0.0
Number rings
Number rings
Number of rings
(a) L L= 2
(b) L =
(c) L = 12
Figure 8: Predicted MAE categorized by different properties. --axis denotes the number or rings, and y-axis denoted the mean absolute error (MAE) the validation set.
17

---

Published as a conference paper at ICLR 2023
the architecture of (Addanki et al., 2021), therefore do not think that our model suffers from over-smoothing.
Modeling k-neighborhood: If want to explicitly use the k-neighborhood information, might need additional modules to process them, like
net (1 neighbor nodes) + n neighbor nodes) + ................................ + net (k neighbor nodes).
(11)
To ensure expressiveness, we usually do not share parameters. Therefore, the parameters are & times larger than regular GNN. O-GNN constantly increases the percentages of parameters (irrelevant to k). When is large, O-GNN will be much more parameter efficient. On the other hand, the optimal k" easy determine. For example, in DrugBank, the ring maximum ring sizes range from (e.g.. DB00658) to 53 (e.g.. DB05034). Which is the best is hard to determine.
About invariant constraints In O-GNN, the features of atoms, bonds and rings all invariant. Specifically, the features of atoms and bonds are about their types, number of correlated electrons, number of neighbors, etc (please refer to hitps: / /tttttbttttu///g//// www.cccm//ooooo. GNN/O cot/ GNN/blob/ 5b70a 4 for details). The ring representations are obtained via atom and bond representations (please kindly refer to (4), which are also invariant. The variant features (like coordinates) are not encoded.
7eeeeleeeelllee @eeeaeeeeec3e3e00 3048 9eb8/ / /ogb//////// bogb/ur ut f py#L2
Comparison about the convergence speed: The validation MAE curves of PCQM4Mv1 are shown in Figure 9. The results of 6-layer O-GNN (with/without rings) 12-layer O-GNN (with/without rings) are reported. We can see that:
(1) by training the 6-layer O-GNN for 175 epochs, the results are almost the same as training the 12-layer "O-GNN w/o ring" for 275 epochs;
(2) by training the 12-layer O-GNN for 75 epochs, the results are almost the same as training the 12-layer "O-GNN w/o ring" for 275 epochs.
These results demonstrate that O-GNN has better convergence speed.
O-GNN ring,
0.145
O-GNN ring,
O.GNN, L-6
0.140
O-GNN, L-12
0.135
0.130
0.125
0.120
175
225
Figure 9: Comparison about the convergence speed O-GNN and "O-GNN w/o ring". r-axis denotes the training epoch and y-axis denotes the validation MAF.
Comparison between different number of parameters. Figure 4 shows the validation MAE of O- GNN and "O-GNN w/o ring" w.r.t. the number of layers. We also visualize the validation MAE w.r.t. the number of parameters in Figure 10. We observe that when aligned with the number of parameters, O-GNN still outperforms the variant without modeling rings.
Pre-training baselines MoleculeNet. We summarize the pre-training baselines on MoleculeNet in Table 11. Sun et al. (2022) have demonstrated different data splitting method could result in significantly different results. We follow the common practice to scaffold based splitting, and we cite the results of Rong et al. (2020) from Fang et al. (2022). Note that the results O-GNN is not pre-trained on unlabeled molecules. We that in terms of the average score, method
18

---

Published as a conference paper at ICLR 2023
O-GNN w/O ring
0.130
O-GNN
MAE
0.125
0.120
0.115
20
40
60
# #Parameters (M)
Figure Validation MAE of PCQM4Mv1 w.r.t. the number of parameters.
comparable with those strong baselines, which demonstrate the effectiveness of our method. We will combine our method with pre- pre-training in the future.
Dataset
Tox21 7831
Clin'Tox 1478
BACE 1513
SIDER 1478
Avg
Molecules
2039
41127
(Hu al., 2020)
71.2 0.9
74.2
0.8
73.7 73.7.::411 4.0
75.8
78.6
0.6
72.3
G- Contextual (Liu al., 2022)
70.3 1.6
75.2 0.3
8.2
75.9
0.9
+.3
58.4
69.8
G-Motif (Liu et al., 2022)
3.4
73.2
7.
73.8
73.4
60.6
70.9
GraphMVP (Liu al., 2022)
1.4
0.3
2.8
1..0
812
1.2
71.9
MGSSL (Zhang et al., 2021)
70.5
0.3
2..7
79.5 1.1
79.7
6.8
71.8
GROVER (Rong et al., 2020)
70.0 7..0:+0.1
0.1
8122
6..5
8.7
64.8
4.66
72.6
GROVER (Rong et al., 2020)
0.1
1.1 3.7
8..
0.1
GEM (Fang et al., 2022)
0.4
0..1
133
80.6
85.6 1.1
0..2
79.0
O-GNN (ours)
7.44
94.3 7.7
81.
85.8
6..2
80.0
Table 11: Pre-training baselines on MoleculeNet.
D RELATED WORK SUMMARY
GCN (Kipf Welling, 2017) aggregates its neighbor information according to the adjacency ma- trix and degree matrix, and then updates the aggregated information with linear transformation and a non-linear activation layer. GraphSAGE (Hamilton et al., 2017b) aggregates the neighbors information by element-wise average. GAT (Veličković et al., 2017) introduces the attention mech- anism into GNN. by which it can adaptively aggregates the representations of the neighbors. Brody al. (2021) propose the GATv2 improve the attention mechanism
more expressive way. Xu et al. (2018) develop simple aggregate function which involves an parameter and multi-layer per- ceptrons (MLPs) which is provably as powerful the Weisfeiler-Lehman graph isomorphism test. Besides, Gilmer et al. (2017b); Li et al. (2017); Pham et al. (2017) propose to augment the graph with a virtual node to capture the global information of the graph. virtual node connects to all the other nodes in the graph and is jointly updated during training. Its effectiveness is validated in series of graph classification tasks.
the ring R in graph neural networks. Complementary to these work, we consider how to incorporate the ring information, which is another important component top of the node and edge information, into molecular modeling. These advanced Aggregate and Update functions are also applicable to our work.
However, these work do not explicitly
19


Reviews:
[{"review_id": "sA5BeqhM8Nv", "reviewer": "Reviewer_X4DK", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In the context of predictive tasks which involve molecules, the paper presents a GNN variant called $\\mathcal{O}$-GNN that explicitly models rings in compounds, besides the usual atom/bond modeling. The paper presents a theoretical analysis that justifies why it is preferable to account for rings in molecules, as well as a series of experiments to show that the model performs well in practical usage. Notably, this model is able to outperform the winner of the KDDCup on the PCQM4Mv1 benchmark (although on the validation set).", "summary_of_the_review": "**Disclaimer**\n\nI did not check the proofs.\n\n**Overall judgment**\n\nWith the doubts I have exposed above, I consider this paper borderline, and I'm giving a 5 for the moment. I'd really like the author to respond to my points in order to be able to raise my score. Also, I **highly** recommend getting this paper proofread for the rebuttal. It would be a shame if I had to leave my score as-is just because nothing was done to improve the writing.\n\n**Questions/comments to authors**\n\n- I think an image will help the reader understand the \"two isomorphic sub-graphs lying on different rings\" in Section 1. Also, this statement is a bit misleading since two graphs with different rings are not isomorphic. Do you mean isomorphic in structure, but not in the node/bond features that make up the ring? I think this difference is addressed in Section 3.2, however, reading up to this point, the reader is confused.\n- In Eq. 4, why the ring representation at the previous layer $h_r^{(l-1)}$ is inside the $\\texttt{MLP}$, when the other representations (Eq. 2, 4, 5) are outside it?\n- For the graph classification tasks, why do you do an average pooling to get the graph representation, when you have all the compound representations at each layer already available?\n\n\n**Minors**\nPage 7: define multi-task learning as MT and random forest as RF in the text, so they are readily understandable when looking at Figure 4.\n\nPage 7: although it is clear from context, you should specify that $\\Delta$-AUPRC is a metric where higher values are better. You should also explain what a \"support set\" is.\n\n\n\n**EDIT**\n\nAfter 1st rebuttal, I am changing my score to 6 given the impressive effort put by the authors to answer my enquiries.\n\n\n**EDIT**\n\nAfter a successful rebuttal, I am now convinced this paper deserves acceptance, and I'm therefore raising my score to 8.", "strength_and_weaknesses": "**Strengths**\n\n- It makes sense from a chemical point of view to have a separate modeling \"circuit\" for rings in molecules.\n- Really impressive results across different tasks.\n\n**Weaknesses**\n\n- Not really well-written (although well-organized).\n- I am a bit confused by some of the experimental results. The PCQM4Mv1 and MolNet benchmarks are somewhat okay. On FS-Mol, $\\mathcal{O}$-GNN replaces a backbone transformer-based residual network. However, the results using \"$\\mathcal{O}$-GNN without rings\" as a backbone are not shown, so it is not clear if the improvement is due to the use of GNN or because of the ring modeling. A similar pattern happens in the drug-drug interaction prediction task.\n- The theoretical analysis shows that few layers of $\\mathcal{O}$-GNN are at least as expressive as many more layers of a regular-GNN. But this does not necessarily qualify as an advantage in practice, unless **1)** $\\mathcal{O}$-GNN layers uses fewer parameters than a regular GNN with the same expressivity; **2)** $\\mathcal{O}$-GNN is faster to train than a regular GNN with the same expressivity. To put it in a different way, why do I have to use few layers of $\\mathcal{O}$-GNN, when I can get the same expressivity with many regular-GNN layers (and perhaps also using fewer parameters or being computationally faster)? I wonder if the authors can provide evidence that using $\\mathcal{O}$-GNN is really more advantageous than using regular-GNN once the expressivity is similar.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "jY6VVPupYiG", "reviewer": "Reviewer_Q5iY", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors propose a ring-enhanced GNN called \\mathcal{O}-GNN to learn ring representations for molecular modeling. The ring representations are updated by concatenating the atom representations and bond representations in the ring, as well as the overall compound representation. Experiments on 11 public datasets present good performance of \\mathcal{O}-GNN.\n", "summary_of_the_review": "In this paper, the authors propose to improve the discriminative ability of GNN in the field of molecular modeling by updating the ring representations. Although the proposed model performs better than the baseline approach on several public available datasets, there is still space for improvement in model innovation. In addition to the weaknesses mentioned above, I have the following concerns:\n\n1. I'm curious how the proposed method overcomes the oversmoothing problem by stacking so many layers?\n2. Why not consider more other types of subgraphs, such as hinge.\n3. Some operations should be supported by adding references or experiments.\n* Why is it necessary to concatenate compound representations when updating atom, bond and ring representations?\n* Why not use the learned compound representations for graph classification tasks but use the average pooling results of atomic representations?\n4. In Table 1, why does graphormer-large perform worse than graphormer-base?\n5. In Table 2，is there any explanation as to why wo ring's O-GNN works better than baseline?\n", "strength_and_weaknesses": "Strength:\n1. \\mathcal{O}-GNN achieves better performance than baseline methods for both molecular property prediction and retrosynthesis prediction tasks.\n2. It is interesting to model rings separately in the molecules since rings play an important role in molecular properties.\n\nWeaknesses:\n1. The innovation of the model is not good enough. It simply updates the ring representations by concatenating atom and bond representations.\n2. The theoretical analysis is not convincing. In fact the ring representations are obtained by aggregating the representations of multi-hop nodes on the ring. A similar discriminatory ability can be achieved by sampling multiple nodes in the k-neighborhood to update the central node.\n3. The proposed method is used for molecular modeling but ignores the equivariant constraints among atoms, bonds and rings.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "E7ncQLPOVy", "reviewer": "Reviewer_4qh9", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new graph neural network O-GNN for reasoning over molecular graphs that specifically encodes ring structures and claims and shows that this improves upon regular GNNs in terms of number of layers required. This is achieved by extending regular message passing GNNs by updating  ring encodings (using simple feed-forward networks) along with the usual update of node and edge encodings.\n", "summary_of_the_review": "Altogether, I think the authors have a good idea but do not justify it in terms of investigation and evaluation for an ML venue focusing on representation learning. A detailed analysis would probably make the paper rather different and go beyond minor corrections. I tend to rejection at the current point. I am open to discussion since I think the paper makes an interesting contribution.\n", "strength_and_weaknesses": "(+) I think it is worth to study molecular ring encoding in an ICLR paper since it concerns an important part of molecules, an important target domain, and representation learning.\n\n(+) The proposed inclusion as first-order components into the MPNN makes totally sense to me and is nice since simple and straightforward.\n\n(-) From an ML perspective, I don't think, the topic and model are investigated fairly. In this form, I would consider the paper to target a more general audience and it could be presented, e.g., at a chemistry venue. More specifically, I am missing a more detailed evaluation and comparison how it improves upon the regular GNN. For example:\n- How do the samples look like (in terms of numbers and size of rings) on which the proposed model is particularly better than other models. Are there samples with rings which the other models get right and how do they look like?\n- Molecular rings tend to not as many atoms and we have technology to construct deeper GNNs. The question is how and if the proposed model improves over those as well (e.g., what happens after 12 layers in Figure 3).\n- Rings can be considered as motifs so it would be interesting to include some GNNs considering motifs into the evaluation.\n\nNote that there is a small section \"The performance for different number of rings.\", but I think much more such analysis should be given.\n\n================================================================================\n\nMinor Comments\n\n- Sec 3.2 (theoretical analysis) seems unnecessary to me. It's a lot of definitions and reading where, in my opinion, you don't learn much in the end given that it's rather obvious that encoding the rings directly adds expressivity.\n\n- p. 3 \"We initlaize\"\n\n- p. 2: (I think)\n\"BACKGROUND AND PRELIMINARY\" -> \"PRELIMINARIES\"\n\"Notations:\" -> \"Notation:\"\n\n- below Eq (1), the entire paragraph about GNNs is redundant for this paper I would say.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster