ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphs
Authors: Limei Wang
Yi Liu
Yuchao Lin
Haoran Liu
Shuiwang Ji
Venue: NeurIPS 2022 Accept (2022)

Abstract: Many real-world data can be modeled as 3D graphs, but learning representations that incorporates 3D information completely and efficiently is challenging. Existing methods either use partial 3D information, or suffer from excessive computational cost. To incorporate 3D information completely and efficiently, we propose a novel message passing scheme that operates within 1-hop neighborhood. Our method guarantees full completeness of 3D information on 3D graphs by achieving global and local completeness. Notably, we propose the important rotation angles to fulfill global completeness. Additionally, we show that our method is orders of magnitude faster than prior methods. We provide rigorous proof of completeness and analysis of time complexity for our methods. As molecules are in essence quantum systems, we build the \underline{com}plete and \underline{e}fficient graph neural network (ComENet) by combing quantum inspired basis functions and the proposed message passing scheme. Experimental results demonstrate the capability and efficiency of ComENet, especially on real-world datasets that are large in both numbers and sizes of graphs. Our code is publicly available as part of the DIG library (\url{https://github.com/divelab/DIG}).

ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphs
Limei Wang" Texas A&M University College Station, TX 77843 limei mameittam... edu
Yi Liu"
Florida State University Tallahassee, FL 32306 liuy@cs www.fs...eee..
Yuchao Lin Texas A&M University College Station, TX 77843 Bruskallin@tane edu
Haoran Liu
Texas A&M University College Station, TX 77843 1iuhr990tamu. edu
Shuiwang Ji Texas A&M University College Station, TX 77843 sji@tamu, edu
Abstract
Many real-world data can be modeled 3D graphs, but learning representations that incorporates. 3D information completely and efficiently is challenging. Existing methods either use partial 3D information, suffer from excessive computational cost. To incorporate 3D information completely and efficiently, we propose a novel message passing scheme that operates within 1-hop neighborhood. Our method guarantees full completeness of 3D information on 3D graphs by achiev- ing global and local completeness. Notably, we propose the important rotation angles to fulfill global completeness. Additionally, we show that our method is orders of magnitude faster than prior methods. We provide rigorous proof completeness and analysis of time complexity for our methods. As molecules are in essence quantum systems, we build the complete and efficient graph neural network (ComENet) by combing quantum inspired basis functions and the pro- posed message passing scheme. Experimental results demonstrate the capability and efficiency of ComENet, especially real-world datasets that are large in both numbers and sizes of graphs. Our code is publicly available as part of the DIG library (https: / ///gitinn giihhu..... com/ com/divelabe divelab/DIG).
Introduction
In machine learning, structured objects such molecules [14, 53, 52], proteins 16, 35, 39, 30], materials [54], and quantum systems [33, 17] are usually modeled as graphs. Original modeling shows basic connections between units and the resulted data type known 2D graphs. Accordingly, 2D graph neural networks (GNNs) have been intensively studied [14, 21, 20, 55, 59, 34] and the message passing scheme [24, 7, 51] is shown effective to realize 2D GNNs. However, in the modern machine learning era, it is increasingly accepted that modeling real-world data like molecules as 2D graphs leads to inborn defects for succeeding learning models. practice, 3D information
"Equal contribution
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

crucial, based on which some important geometries can be derived, such as chemical bond lengths in molecular modeling. This essentially raises the need of a new data type, known as 3D graphs.
Formally, a 3DD graph contains the original 2D graph as well as Cartesian coordinates for all nodes. In this work, follow invariant 3ID GNNs [43, 22, 32, 45, 36, 23] that take relative 3D information, like distances and angles, as inputs to networks. Such relative geometries are naturally SE(3))invarian) The main challenge comes from what geometries should be computed such that 3D information is incorporated completely. A most recent work SphereNet [36] shows that distance, angle, and torsion information is necessary to incorporate more comprehensive 3D information. However, SphereNet is only complete in local neighborhood, failing to achieve global completeness and distinguish wide range of molecular structures such as conformers. Let n and denote the number of nodes and the average degree in a 3D graph. Existing methods also exhibit excessive time complexity of O(nk?) or even O(nh:"), ), severely preventing their scalability in real-world applications.
In this work, we propose ComENet as complete and efficient graph neural network for 3D molecular graph learning. We first formally provide the definition of completeness. Intuitively, geometric transformation is considered as complete if it generates distinct representations for any two different 3D graphs. Based on this, we propose novel message passing scheme by faithfully fulfilling global completeness via the important rotation angles. In addition, we design novel strategies to achieve local completeness, largely reducing the computing complexity to (nk)). To elucidate the merits of the proposed methods, we provide rigorous proof of geometric completeness achieved by our method. Combining the novel message passing scheme and quantum inspired features, ComENet is developed for 3D) molecular graphs. We apply ComENet to two large-scale datasets including OC20 and Molecule3I,, and a commonly used dataset QM9. Experiments show that ComENet performs similar to existing best methods, but accelerates the training and inference by 6-10 times on various datasets. We summarize the contributions of ComENee as below. (i). To our best knowledge, is the first rigorously complete pipeline for 3D molecular graph learning. Theoretically, it is guaranteed to incorporate 3D information completely without information loss. Practically, can distinguish all molecular structures in nature. (ii). It is highly efficient. The message passing is shown to be orders of magnitude faster than existing methods in terms of time complexity. (ii). The great capability and efficiency of ComENet allow its scalability to real-world molecule datasets that are large in both numbers and sizes of graphs. (iv). It achieves similar or better performance compared with existing methods, and dramatically accelerates the training and inference by 6-10 times.
Relations with Prior Work. SphereNet [36] is recent method that achieves local completeness with complexity of (nk)²) In summary. SphereNet is not complete and not efficient enough for processing large-scale molecular graphs. However, ComENet is complete and much more efficient with complexity of O(n)). Technically, a primary difference is that, ComENet proposes to use the important rotation angles to achieve global completeness. As a result, ComENet is provably complete as shown in Sec. 2.2. Practically, ComENet is able to identify wide range of real-world structures such as conformers, achieving completeness at the conformer level, as introduced in Sec. 2.3. In addition, ComENet and SphereNet both use (d, 0, Φ) in the spherical coordinate system (SCS) to obtain local completeness. Indeed, the fact that (d. 0, Φ) can specify the location of a point in SCS is widely known. The key difference lies in that, SphereNet operates within 2-hop neighborhood, while ComENet operates within h-hop neighborhood. When coupled with rotation angles, ComENet can achieve provable completeness with a reduced complexity of Onk)). This different message passing for local completeness in ComENet entails many differences with SphereNet, including building coordinate systems, defining z-axis, choosing reference nodes, and computing (d, 0, Φ). as detailed in Sec. 2.4. All the computing procedures for ComeNet are described in detail in Algorithm 1 of Appendix A.I.
2
The Proposed Message Passing Scheme
2.1 Notations & Definitions
We first formally define notations and the concept of completeness used in this paper.
Notations. A 3D graph G can be represented as G AV, The node feature matrix V =
vn.'T [V1, V2. ".,'.,.'" E I" xd, with each v, E IRd., The adjacency matrix A E R=xn, based on which we additionally define there is an edge ey if A|]][s] 1. The position matrix P = .2 'P,,'" g2x33 where p.- (I Uss 21) 3 is the position vector for node i given in the Cartesian coordinate
2

---

system (CSC). The relative position Pú of node j to node is defined as Pij P; P;- Particularly, throughout this paper, we define the average degree for G.
We then formally define completeness given a geometric transformation T. In particular, we aim at incorporating 3D information in 3D molecular graphs. Hence, our definition of completeness is set from the geometric view. Generally, T maps a 3D graph G = (V, A, P) to a geometric representation with size m ht where m is the number of transformed geometric features and h is the feature size. T can be different dependent on different methods, resulting in different m E N or h E N+ For example, SchNet [43] only computes the distance for each edge based on the coordinates of the two nodes connected by this edge. Thus. SchNet maps G to a representation with size m x h, where m is the number of edges in G and 1. However, such geometric transformation not complete. We provide the definition of completeness as below:
Definition 1 (Completeness). For two 3D graphs G = AV, and G2 (V, A, P2), a geometric transformation T (R=xd, R"x" R=x3)
) + Rmxd is considered as complete when
T(G1) = T(G2) = ER E SE(3), R(22).
Here SE(3) denotes the Special Euclidean group in dimensions. include all rotations and translations in 3D [1, 26, 36, 41]. Thus R is transformation that combines rotation and translation. A rotation transformation can be represented with a 3 x rotation matrix, and translation transformation can be represented by 3 x 1 vector. Matrix form of SE(3) is provided in Appendix A.2. The reason why we introduce SE(3) lies in that, combination of rotation and translation will not change the 3D conformation of a 3D graph. In Def. 1, if P1 and P2 are in the same SE(3) group, then G1 and G2 would share the same 3D conformation. As a result, G1 and G2 would be the same 3D graph. Intuitively, complete geometric transformation T can distinguish any two different 3D graphs. This is to say, as long as two 3D graphs differ in 3D conformations, their outputs from T would be different.
2.2 Global Completeness via Rotation Angles
Existing studies focus on the complete representation learning of local neighborhood. Earlier methods like SchNet [43] and DimeNet [22] cannot achieve local completeness. In more recent method SphereNet, completeness is guaranteed within edge-based 1-hop local neighborhood. but fails to hold in the whole 3D graph. In this section, we move a step forward to formally fulfill global completeness for given 3D molecular graph. Particularly, for the purpose of clear illustration, we safely assume local completeness is already obtained by exiting methods like SphereNet.
Without loss of generality, start by describing our method to attain full completeness within 2- hop neighborhood, as illustrated in Fig. (a)). Formally, for cen- ter node i, we let N₁ and N?² de- note two sets of indices of i's 1- hop and 2-hop neighboring nodes, respectively. We also define any node i and its 1-hop neighbor- hood as a local structure. Then the whole 2-hop neighborhood of node can viewed as + [Mi local structures centered in and Nr. defined as L, and Lij.EE,, respectively. As shown in Fig. 1(a), L, is the local structure centered in i, and Lij is the local structure centered in j. Apparently, each local structure Lij.EE, shares the common edge ei with the local structure C, Given the complete representation for each local structure, for the structure C, U Lij. the only remaining degree of freedom is the rotation angle of edge eij. denoted as Tiy. With the rotation angles for all the JN] common edges specified, we can obtain a complete representation for 2-hop neighborhood of node i. Achieving completeness beyond 2-hop neighborhood is similar. Overall, after considering rotation angles, the global completeness be easily guaranteed when gradually generalizes from n- (n + 1)-hop neighborhood.
feu
C
Eyj
(a) Illustration of 2-hop local structure.
(b) Illustration of computing the rotation angle for edge eij-
Figure 1: Illustrations of how to achieve global completeness in our proposed methods.
Essentially, each edge an input graph can treated as a common edge between different local structures. We reveal how to compute the rotation angle Tú for each edge ei in Fig. 1(b). Specifically,
3

---

Energy/eV
A
CHs
B
GH3
0.3
D
0.2
0.1
B
HgC
H
CH3
180"
120"
60"
0"
C
D
Clly
11
CHa II
HCHs
H₂C
CHs
II
HaC CH2
H
CH3
H
H
CH2
H₂C
H
(a)
(b)
Figure 2: (a). Illustration that the relative conformation energy of butane is a function of the rotation angle of the C-C bond. (b). A 3D view of the four conformers in (a).
choose two reference nodes whose indices are fry and fyy for node i and j. respectively. fay denotes the index of i's nearest neighhoring node except j. and lزVx denotes j's nearest neighboring node except i. Then Taj for edge Cij is the angle from the plane formed by feyj to the plane formed by j, Jy As analyzed previously, the global conformation of the input graph can be identified based on all local structures and rotation angles. As result, given fixed local structures, the global completeness the input 3D molecular graph is fulfilled by additionally considering the rotation angle of each edge, introduced in this section. Note that in terms of the selection of reference nodes for computing rotation angle, we think a selection strategy is valid as long as it is applied to every edge consistently. In this work, for an edge eıj. we choose the nearest neighboring nodes for and j as reference nodes, which are fily and fy respectively. We apply this selection strategy to all edges in 3D graph for computing corresponding rotation angles. With such selection strategy. we can prove that our method is complete as shown in Sec. 3.1. We also show in Appendix A.3 that it is easier to prove completeness using nearest neighboring nodes as reference nodes.
2.3 Rotation Angles for Conformer Identification
The proposed rotation angles in Sec. 2.2 play a crucial role in identifying some important molecular structures, such as conformers. In nature. real molecule exits as an ensemble of interconventing 3D structures, known as conformers [5, 4, 19]. Different conformers posses the same 2D molecular graph, but differ in 3D structures. Generally, conformer for molecule exits with a certain probability and may exhibit distinct properties [5, 4]. As shown in Fig. 2(a), different conformers for the molecule butane (C4H2a) show varying conformation energy. To this end, is important to design complete 3D GNNs for identifying molecules at the conformer level. From the geometry perspective of view, a conformer distinguishes itself from others mainly through varying rotation angles of chemical bonds [19]. As shown in Fig. 2(b), given the fixed ethyl (-C2H5) of both sides, the only degree of freedom is the rotation angle the C-C bond. In literature, the ethyl is formulated as a local 3D graph. Existing studies focus on the complete representation of such local structures, failing identify the whole 3D graph globally. Essentially, they can only distinguish different molecules, trying to achieve completeness at the molecule level rather that the finer comformer level. By integrating the rotation angles as in Sec. 2.2 into the message passing scheme, our methods can fulfill rigorous completeness at the conformer level and can distinguish all conformers in nature.
2.4 Local Completeness with Improved Efficiency
Formally, each node and its local neighborhood can be viewed as a local structure. Existing studies focus the learning of local structures and SphereNet achieves local completeness. However, SphereNet induces the complexity of Onk??), restricting its scalability on large molecules in practice. Here, we design novel strategy to guarantee local completeness with the computing cost of O(nk).
Specifically, we follow SphereNet and perform on the spherical coordinate system. It is commonly known that the location of each node can be completely determined using the tuple (d, 8, Φ) in SCS. SphereNet employs the directional message passing (DMP) fashion that operates within hop neighborhood. It first updates messages over edges thus the center edge is z-axis in SCS. For node i, the computing of the tuple (d, 0, Φ) involves 2-hop information. However, we view 1-hop neighborhood as local structure, which requires all strategies in our local completeness to be

---

different from SphereNet, including building local coordinate systems, defining z-axis, picking reference nodes, and computing (d, o). First, we build a light local coordinate system for any node i's corresponding local structure. Similarly, the center node i serves as the origin. Then z-axis is defined as the direction from i its nearest neighbor fa, and zzz-plane further formed by z-axis and i's second nearest neighbor 8;. Finally, the tuple (d, 0, σ) is computed within 1-hop neighborhood with a complexity of ((nk). Particularly, we analyze efficiency versus model expressiveness in Sec. 3.2. We show that compared with the DMP fashion used by SphereNet, our method operating within l-hop neighborhood hurts the model expressiveness a bit by largely improves the efficiency.
2.5 Message Passing Scheme
Based upon global completeness achieved in Sec. 2.2 and improved local completeness introduced in Sec. 2.4, the complete geometrie transformation T required by Def. I should be formulated based on 4-tuple as (d, f, o, r). Specifically, we build such transformation within 1 1-hop neighborhood, x
and a 4-tuple is computed for each edge. Hence, for a 3D graph G (V, E, P). the full expression for 'T is TG)) = [dij. Øij. Túj "y)ii=......jJ... E R= where m is the number of edges in G. Especially, as T converts absolute Cartesian coordinates in P to relative information, is naturally SE(3)-invariant as required in Def. 1. To this end, we formally build our message passing scheme as
V = 9 vi. diy 6zg. JEN.
where 9 and / can be implemented by neural networks or mathematical operations. Intuitively, our message passing is established in 1-hop local neighhorhood and all edges connecting regions beyond. Essentially, dıj. Ozj. and Φij specify the 1-hop local neighborhood, and Tij determines the orientation of the local neighborhood. By doing this, the complete representation for whole 3D molecular graph is eventually achieved. The formulas of computing of dig. Oij. and Φig, and Túj are shown in Algorithm in Appendix A. along with detailed description of the complexity of O(nk). Overall, our formal analysis in Sec. 2.2, Sec. 2.3, and Sec. 2.4 lead to the proposed message passing scheme defined in Eq. 1. It is the first fully complete scheme with great efficiency of O(nk). We also provide rigorous proof on completeness and analysis on efficiency of our message passing in Sec. 3. Note that achieve efficiency, our message passing scheme adopts novel strategy that computes all the needed geometries within 1 1-hop neighborhood. Hence, it can not be directly applied to existing architectures built in 2-hop neighborhood, such as DimeNet++ [32] and SphereNet [36]. To this end, we design new network to implement the proposed message passing scheme, as detailed in Sec. 4.
3 Merits of Our Methods
3.1 Geometric Completeness
Proposition 1. For a strongly connected 3D graph G = (V, its geometric transformation
T(G)- [(deg: 0ig, deiy Tig [.j])i=,,...:s..., is complete.
Proof. We employ mathematical induction and assume the number of nodes in 3D graph is n. Note that the 3D graph we consider is strongly connected, which means that there exist a path between any two nodes in the graph. All the molecules in nature can be constructed as strongly connected graphs.
Base case: It is obvious that the 3D structure G can be identified when n = 1. Hence, we let be the base case, where the completeness can be achieved by only considering d and @ in T.
Inductive hypothesis: The claim that TT is complete holds for the node numbers of n up to k: 2 3.
Inductive step: Let n = k + 1. Without loss of generality, among the existing A nodes, we safely assume and its neighboring nodes 1 form the local region of interest. Then j is the index of the newly (k + 1) node connected to the center node i. To show global completeness of the whole graph, based on Def. 1, we only need to prove that the relative position of the new node j uniquely determined given T. We propose the following lemma for this as
Lemma 1. Assume a strongly connected 3D graph G = EV,
E, P) with more than nodes is fully identified. If a new node j is connected to a node i of G following the geometric transformation T(G)- = [[dky 01g, dig, Tzy
then Piş is uniquely determined.
5

---

The proof of Lemma 1 is provided in Appendix A.3. With Lemma successfully proved, we show that such geometric transformation T can determine a unique 3D graph. Hence, the if condition in Def. 1 holds. In addition, as T renders purely relative 3D information like distance and angle, it's naturally SE(3) invariant. Hence, the only if condition in Def. holds. Overall, based on Def. 1, we complete the proof of Prop. 1.
L
Intuitively, since molecular graph is strongly connected, two arbitrary nodes are connected by at least one path. Hence, starting from the existing structure, can restore the relative position of any new node step by step along path with finite length. As a result, geometric completeness for the whole 3D graph with any number of nodes can be guaranteed in our message passing scheme. Theoretically, 3D information of the input 3D molecular graph is fully captured without information loss. practice, our method can distinguish all structures in nature.
3.2 Efficiency
Efficiency versus Model Expressivenes:: Our method induces the complexity of ((k)) by operating within 1-hop neighborhood. Existing methods, such as like DimeNet and SphereNet, employ the DMP fashion that update edges within 2-hop neighborhood, inducing the complexity of O(nA2). Notably, DMP [48, 57, 22, 36] incorporates 2-hop information in one single layer but (n + - when stacking n layers. However, stacking n proposed message passing layers is already able to incorporate information from n-hopp neighborhood. Obviously, compared with a network containing several DMP layers, network with the same number of our proposed message passing layers merely hurts the model expressiveness bit, but significantly improves the model efficiency.
Efficiency via Less Torsion Angles: In addition, method achieves completeness by computing O(nk) torsion angles, which is efficient especially compared with methods including Gasteiger et al. [23], Adams al. [1], Ganea et al. [19]. Given scenario where global region is the union of two local regions with ni and f22 nodes. As introduced in Sec. 2.2 and Sec. 2.4, our method computes the same number of torsion angles as nodes in local region, then employs rotation angle (torsion angle essentially). Hence, the number of torsion angles that our method computes is + 722 + 1. Ganea al. [19] computes a torsion angle based on one pair of nodes, each of which is from separate local region. Hence, the number of torsion angles needed is n1 X n22 Apparently, our method reduces the number of torsion angles from Onk-3) (nk)), which significant considering the computing of torsion is excessively expensive.
4 ComENet
z
0,
d, r
Based upon the message passing scheme introduced Sec. 2.5, we propose the complete and efficient graph neural network (ComENet) as shown in Fig. 3. Existing invariant 3D GNN methods [42, 22, 32, 36, 23, 44] share the similar architecture pipeline, which contains an input block, sev- eral interaction blocks, and out- put block. Our ComENer also fol- lows such architecture fashion along with several novel components, such as self-atom layers and specifically designed local and global graph con- volution layers, to better fulfill our proposed message passing scheme in Eq. 1. Generally, ComENet consists of an embedding layer, multiple inter- action layers, a self-atom layer, and a pooling layer. To be in line with the message passing scheme in Sec. 2.5,
Embodding
Interaction Block
TBF
MLP
SBF
Interaction
GlobalConv
Interaction
Concat
Interaction
Down-Project
Self-Atom
MLP
Pooling
Output
Figure 3: Illustration of ComENet with overview (left) and the interaction layer (right). TBF and SBF denote the basis functions for tuples (d, 0, Φ) and (d, 7). LocalCony and GlobalConv denote the proposed local and global con- volution layers. Concat is the concatenation operation and Down-Projj is a linear layer to reduce feature dimensions. + denotes the element- wise sum operation.
take the updating process for node

---

i's feature vector v, as an example to describe the network. In practice, feature vectors for all the nodes in a graph are updated simultaneously. In particular, we omit all indices in Fig. and below for clear presentation. Specifically, embedding Layer converts atom type to an initial node feature vector v via learnable atom type embeddings [43, 22]. Interaction Layer updates node feature vector v based on features of the neighboring nodes and geometric features (d. 0, é, r) in Eq. 1 using the local and global graph convolution layers. A detailed description of interaction layer is provided in Appendix A.4. Self-atom layer is used to update each node feature and project the feature dimension into I, And pooling layer is a sum-pooling performing on all node features to obtain final predictions.
5 Related Work
We consider how to represent 3D information in 3D molecular graphs [3, 10]. One category of methods are equivariant 3D GNNy that directly use coordinates in the CSC as inputs to networks [49, 2, 18, 44, 8]. These methods are efficient but suffer from several setbacks. Firstly, each network component needs to be carefully designed to be rotation equivariant of input graphs. Secondly, the reason why this category of methods are efficient lies in that they only use the type-1 basis in Spherical harmonics, which is an approximation essentially. It is proved in [49] that theoretically, l is infinite in terms of type-l basis. In practice, I should be at least 2 for achieving satisfactory performance. Type-1 basis essentially coerces the cony kernel to be in a narrow learning space, which imposes a hard constraint on the network capability. However, when using type-2 basis, the conv kernel can be more expressive while the efficiency issue would emerge as a new bottleneck. Thirdly, the performance of such equívariant GNNs is shown to be worse than invariant 3D GNNs [36].
In this work, we follow invariant 3D GNNs to inherit the merit of SE(3)-invarianee by investigating relative 3D information, which is used in both representation learning tasks [43, 22, 32, 45, 36. 23] [4,,22,
and coordinates generation tasks [19, 46, 47, 56, 30, 6]. Since the relative information is SE(3)- invariant of input 3D graphs, the employed networks favorably achieve the invariance merit. We focus on the 3D graph learning problem and existing methods either capture partial 3D information or suffer from high computational cost. For example, SchNet [43] only considers distance information and DimeNet [22] further incorporates angles between bonds. They both integrate incomplete 3D information that the network capacity is limited in practice. SphereNet [36] generates approximate complete 3D representations by using distance, angle, and torsion information but the complexity is Onnk²). GemNet [23] is based on quadruplets of nodes, which is more expensive. Our objective is to build a fully complete 3D graph net with a much lower computational budget.
There also exist some methods in literature using both absolute and relative 3D information [25, 28, 58). In this work, we only use relative information as input to avoid inherent limitations of equivariant 3D GNNs. Moreover, we design a novel message passing such that the computational cost is comparable with that of equivariant 3D GNNs.
6 Experiments
We examine the power and efficiency of ComENet on two large-sacle datasets in-
Table I: Statistics of the datasets.
Dataset
OC20
Molecule3D
QM9
cluding Open Catalyst 2020 (OC20) [12] and Molecule3D [56], and the mostly commonly used datastet QM9 [40]. The statistics of three datasets are provided in Table 1. Detailed descriptions of the datasets are provided in Appendix A.S. In particular, the Molecule3D contains about 4 million 3D
# Graphs
660,010 Split Ratio
3,899,647
130,831
Split Type
Pre-defined
Random/Scafiold 6:2:2
Random
70:15:15
84:8:8
# Nodes/Graph
77.75
29.11
18.02
molecular graphs, and the OC20 has the average graph size of 77.75. Baseline methods include GIN- Virtual [27], CGCNN [54], SchNet [43], PhysNet [50], MGCN [37], DimeNet [22], DimeNet++ [32],
SphereNet [36], PaiNN [44], GemNet [23]. Unless otherwise specified, the values for baseline methods are taken from the referred papers. For the ComENet, we use data loader in the PyTorch
Geometric library [15] to load the datasets. All the models are trained using the Adam optimizer [31] and the optimal hyperparameters are obtained on validation sets using grid search. Experimental setup and search space for all models are provided in Appendix A.6. Code is integrated in the DIG library [34] and available at https: ///ggtiii //github. com dimd/ieelab/DI
7

---

Table 2: Results on IS2RE including computing cost in training&inffrence and performance in terms of energy MAE and the percentage of EwT of the ground truth energy. Training time is the average time per epoch during training using GPU. Performance is reported for models trained on the All training dataset. The best performance is shown in bold and the second best is shown with underlines.
Time
Energy MAE [eV]]
EwT
Model
Train
Infor.
ODD
0OD Cat
OOD Both
Average
ODD
0DD Cat
OOD Both
Average
OGCNN
18min
Imin
0.7426
0.6718
3.30%
2.11%
2.29%
2,82%
SchNet
10min
Imin
0.6465
0.7074
0.6475
0.6626
0.6660
2.96%
2.225%
3.03%
2.38%
2.65%
DimeNetee
250min
4min
0.5636
0.7127
0.5612
0.6492
0,6217
4.25%
2.48%
4.41%
2.56%
3.42%%
GemNet-T
200min
4min
0.5561
0.7342
0.5659
0.6964
06632
4.51%
2.24%
2.38%
3.38%
SphereNet
290min
5min
0.6682
0.6190
0.6023
2.20%
2.70%
3.64%
ComENet
20min
Imin
0.5558
0.6602
0.5491
0.5901
0,5888
4.17%
2.71%
4,53%
2.83%
3,56/8
6.1 OC20
The Open Catalyst 2020 (OC20) dataset is a newly released large-scale dataset with millions of DFT relaxations model and discover catalysts. In this work, we focus on Initial Structure to Relaxed Energy (IS2RE) task, which is the most common task in catalyst discovery. Descriptions of the data and tasks are provided in Appendix A.5. The ground truth of the test set is not publicly available, therefore, we compare the results of different methods on the validation set. The evaluation metrics include the energy MAE and the percentage of Energies within Threshold (EwT) of the ground truth energy. The values for the baseline methods are taken from Chanussot et al. [11]. Liu al. [36]. Notably, aim to predict relaxed energy directly from initial structure and do not compare with some methods using relaxation [12], trajectory information, or relaxed structures. Using relaxation [12, 23, 45] is computationally expensive during prediction while the relaxation trajectory and relaxed structures [25, 58) are hard to obtain in practice.
Table 2 shows that ComENet outperforms all the baseline methods in terms of energy MAE, which is also used as the main evaluation metric in the Open Catalyst Challenge [13]. ComENet achieves best performance two splits and the second best the other two splits in terms of EWT. Specifically, ComENet reduces the average energy MAE by 0.0135, which is 2.2% of the second best model. Note that ComENet achieves the best results the OOD Both split in terms of both energy MAE and EWT. In practice, it is common that test data is in the different domain with the training data. Hence, OOD Both test the generalization capability of learning models. More importantly, ComENet is much more efficient than methods like DimeNet++ and SphereNet. For example, Sphere needs hours per epoch while ComENet only requires 20 minutes using the same computing infrastructure (NVIDIA RTX A6000 48GB). Overall, compared with existing best methods, ComENet achieves better performance and largely reduces training by least 10 times.
6.2 Molecule3D
The Molecule3D dataset [56] is newly proposed large-scale dataset, in- cluding around 4 million molecules with precise ground-state 3D informa- tion derived from DFT and molecu- lar properties. We focus on the pre- diction of the HOMO-LUMO gap as it is one of the most practically- relevant quantum chemical properties of molecules. A detailed description of the Molecule3D dataset provided in Appendix A.5. As this is a newly proposed dataset, we run baseline methods including GIN- Virtual [27].
Table 3: Comparisons between ComENet and other models
in terms of computing cost and Molecule3D for both random and scaffold splits. Train time is the average training time per epoch.
gap MAE on
Time
MAE
Model
Train
Inference
Random
Scaffold
GIN-Virtual
15min
2min
0.1036
0.2371
SchNet
14min
3min
0.0428
0.1511
DimeNet++
133min
16min
0.0306
0.1214
SphereNet
182min
28min
0.0301
0.1182
ComENet
22min
3min
0.0326
0.1273
SchNet [43], DimeNet++[32] and SphereNet [36], among which GIN-Virtun Virtual a powerful baseline for 2D graphs while the others are for 3D graphs. All the models are trained using the same computing infrastructure (Nvidia GeForce RTX 2080 Ti 11GB).
8

---

Table 4: Comparisons between ComENet and other models in terms of MAE and the overall mean std. MAE on QM9.
Property
Unit
SchNet
PhysNet
MGCN
DimeNet
DimeNet++
PaiNN
SphereNet
ComENet
μ
D
0.033
0.0529
0.0560
0.0286
0.0297
0.012
0.0245
0.0245
ag
0.235
0.0615
0.0300
0.0469
0.0435
0.045
0.0449
0.0452
meV
41
32.9
42.1
27.8
24.6
27.6
22.8
23.1
FLUMO
meV
34
24.7
57.4
19.7
19.5
20.4
18.9
19.8
Ac
meV
63
42.5
64.2
34.8
32,6
45.7
31.1
32.4
R2))
an²
0.073
0.765
0.110
0.331
0.331
0.066
0.268
0.259
ZPVE
me'V
1.7
1.39
1.12
1.29
1.21
1.28
1.12
1.20
Up
meVV
14
8.15
12.9
8.02
6.32
5.85
6.26
6.59
U
meV
19
8.34
14.4
7.89
6,28
5.83
6,36
6.82
meV
14
8.42
14.6
8.11
6.53
5.98
6,33
6.86
G
meV
14
9.4
16.2
8.98
7.56
7.35
7.78
7.98
6
mol
K
0.033
0.038
0.025
0.023
0.022
0.024
std. MAE
%
L.76
1.37
1.86
1.05
0.98
1.01
0.91
0.93
Table shows that ComENet dramatically reduces training time by 6-9 times compared with DimeNet++ and SphereNet, and costs similar time as GIN-virtual and SchNet that only considers distance information. terms of performance, our ComENet performs much better than SchNet for both random and scaf- fold splits with similar time and computing costs, but a little worse than DimeNet++ and SphereNet. This may be due to the molecules are relatively small in Molecule3D compared with OC20, the structures that our complete strategy can distinguish may not exist in the dataset. However, considering the comparable performance and high efficiency, our ComENet is more practically useful than other methods on such large datasets. In addition, Fig. also shows ComENet either converges much faster in terms of total training time or performs much better compared with other baselines.
Gin-Virtual
SchNet
DimeNet++
SphereNet
ComENet
2000
4000
6000
8000
Time
Figure Total training time for different methods on Molecule3D.
6.3
QM9
The QM9 dataset is a widely used dataset for predicting various properties of molecules. The evaluation metries include the MAE for each property and the overall mean standardized MAE (std. MAE) for all the 12 properties. detailed deseription of the dataset is provided Appendix A.5. Notably, we do list results for PPGN [38] and Cormorant [2] since they use different train/val/test sizes. Table 4 shows that ComENet is much better than the methods operating in 1-hop neighborhood like SchNet, PhyNet, and MGCN. Compared with DimeNet, DimeNet++, PaiNN, and SphereNet, ComENet achieves similar results on all properties and the overall std. MAE. Consistently, compared with methods operating in 2-hop neighborhood like DimeNet, DimeNet++, and SphereNet, ComENet is much more efficient.
6.4 Ablation Study for Identifying Conformers
As mentioned Sec. 2.3, rotation angles are the main difference be- tween different conformers [19, 29]. We investigate the contribution of our proposed rotation angles т to demonstrate the effectiveness of our global complete representations. We conduct experiments on the OC20 dataset since there exist different con- formers for molecules in this dataset.
pisc
22
Frequency
2000
4200
650
1090
addorbates
11500 catalvsts
Figure 5: Distributions of adsorbates and catalysts in OC20. For y-axis, frequency counts the number of conformers for each individual adsorbate and catalyst.

---

Specifically, there are 660,010 data samples in the OC20 dataset (IS2RE), where each sample is a combination of two parts, namely, adsorbate and catalyst. There are 82 adsorbates and 11,500 catalysts used in the datasets. Fach adsorbate inevitably corresponds to different conformers in the dataset, and it is similar to catalyst. We show the number of conformers for each adsorbate and each catalyst in Fig. 5. We remove the rotation angle from ComENet and denote it as "ComENet w/o r". The results in Table 5 show that removing rotation angles T can harm the performance of ComENet, demonstrating the effectiveness of our global complete representations for identifying conformers.
Table 5: Comparisons between ComENet and the model without rotation angles T on OC20.
Energy MAE [eV]
E
Model
OOD Ads
OOD
OOD
Average
OOD Ads
OOD
Average
ComFNet
0.5558
0.6602
0.5491
0.5901 0.6186
0.5888
4.17%
2.71% 2.65%
4.53% 4.13%
2.81% 2.75%
3,56%
ComENNet w
0.5585
0.6851
0.5574
0.6049
4.13%
3.42%
7 Conclusions, Limitations, Outlook, and Societal Impacts
3D information is crucial for 3D molecular graph learning. Existing methods either learn partial 3D information or induce high time complexity. We propose ComENet that is both complete in incorporating 3D information and efficient with time complexity of (nk)). Particularly, we propose the novel rotation angles to fulfill global completeness. ComENet can generalize to large-scale datasets, accelerating training and inference by 6-10 times with superior or comparable performance. Even though ComENet is the first complete and the most efficient 3D GNN model, there exists one major limitation, which not only for ComENet but also for existing 3D GNNs. Basically, existing 3D GNN models are centered on the fact that 3D information is given in data. However, acquiring 3D information itself is difficult and expensive in practice. Current methods rely on experiments or DFT-based computing, which is extremely time-consuming It would be significant that machine learning models can be developed to tackle this problem. Looking forward, we can derive two directions for fulfilling such objective. Firstly, we can study to generate 3D graphs either from 2D) graphs or from scratch by developing generative models, such as VAE, flow and diffusion models. Especially, in some real-world applications like drug discovery, 2D molecules This raises the need of developing new generation methods from scratch. Secondly, we can target at research case where we have minimal set of training data with 3D information, but the vast unseen data or new data lack such 3D information. We may develop novel contrastive learning components to force correspondence and consistency between 2D graphs and their 3D geometric views, then integrate such components into end-to-end learning systems for application based on 2D graph data. ComENet can facilitate a plethora of important real-world applications, such as drug discovery and material discovery. It can be used in several research domains including quantum chemistry and physics, material sciences, molecular dynamics simulations, etc. Any negative societal impact associated with those applications and domains can be applied to our method.
usually not given.
Acknowledgments and Disclosure of Funding
This work was supported in part by National Science Foundation grant -90-290 and National Institutes of Health grant U01700701
References
[1] Keir Adams, Lagnajit Pattanaik, and Connor W. Coley. Learning 3D representations of molec- ular chirality with invariance to bond rotations. In International Conference on Learning Representations, 2022. URL /tsttp:://ppereeteee wwtpeeewwwwweeeeeeeeeeeeeeeeeeee www./forum/miiiie http://oruummtddmmmmtttdeeeee hnDtdddda
[2] Brandon Anderson, Truong -oong-Sonnon Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks. In Proceedings of the 33st International Conference on Neural Information Processing Systems, pages 14537-14546, 2019.
[3] Kenneth Atz, Francesca Grisoni, and Gishert Schneider. Geometric deep learning on molecular representations. Nature Machine Intelligence, pages 1-10, 2021.
10

---

[4] Simon Axelrod and Rafael Gomez- Bonbaz-lmmbarllii.. Molecular machine learning with conformer ensembles. arXiv preprint arXiv:2012.08452..00 2020.
[5] Simon Axelrod and Rafael Gomez- Bonnz--ommbareli.. GEOM, energy-annotane molecular conforma- tions for property prediction and molecular generation. Scientific Data, 9(1):1-14, 2022.
[6] Minkyung Baek, Frank DiMaio, Ivan Anishchenko, Justas Dauparas, Sergey Ovchinnikov, Gyu Rie Lee, Jue Wang. Qian Cong, Lisa N Kinch, Dustin Schaeffer, Accurate prediction of protein structures and interactions using three-track neural network. Science, (73(6557):871-8766 2021.
[7] Peter W Battaglia, Jessica Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint 1xXiv:806.00166 2018.
[8] Simon Batzner, Albert Musaclian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E(3)-cquivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):
1-11, 2022.
[9] Karsten M Borgwardt, Cheng Soon Ong, Stefan Schönauer, SVN Vishwanathan, Alex Smola, and -eter Kriegel. Protein function prediction via graph kernels. Bioinformaties, 21 (supp__):i47-i56, 2005.
[10] Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint 13445 2021.
[11] Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Mor- gane Riviere, Kevin Tran, Javier Heras-Doming Caleb Ho, Weihua Hu, et al. Correction to "the open catalyst 2020 (oc20) dataset and community challenges". ACS Catalysis, 1 11(21):
13062-13065, 2021.
[12] Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Mor- gane Riviere, Kevin Tran, Javier Hieras-Dominge Caleb Ho, Weihua Hu, al. Open catalyst 2020 (OC20) dataset and community challenges. ACS Catalysis, 11110)6059-6072, 2021.
[13] Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Mor- gane Riviere, Kevin Tran, et al. Open Catalyst Challenge. https:/ http:///poeenaatlysrrpojeee org/ htg/challenge html, 2021.
[14] David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Sazukk-Guzik, and Ryan P Adams. Convolutional networks on graphs for learning molecular fingerprints. Advances in Neural Information Processing Systems, 28:2224-2233 2015.
[15] Matthias Fey and Jan E. Lenssen. Fast graph representation learning with Py PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
[16] Alex Fout, Jonathon Byrd, Basir Shariat, and Asa Ben-Hur Hur. Protein interface prediction using graph convolutional networks. In Proceedings of the 31 International Conference on Neural Information Processing Systems, pages 6533 (642,,652,,00 2017.
[17] Cong Fu, Xuan Zhang, Huixin Zhang, Hongyi Ling, Shenglong Xu, and Shuiwang Ji. Lattice convolutional networks for learning ground states of quantum many-body systems. arXiv preprint arXiv:2206.0r 07370, 2022.
[18] Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. SE(3) Transfrrrasss:mer::: 3D roto-translation equivariant attention networks. Advances Neural Information Processing Systems, 33, 2020.
[19] Octavian-Fugen Ganea, Lagnajit Pattanaik, Connor W Coley, Regina Barzilay, Klavs F Jensen, William Green, and Tommi S Jaakkola. GeoMol: Torsional geometric generation of molecular 3D conformer ensembles. Advances in Neural Information Processing Systems, 2021.
11

---

[20] Hongyang Gao and Shuiwang Ji. Graph U-Nets. In International Conference on Machine Learning, pages 2083-2092. PMLR. 2019.
[21] Hongyang Gao, Yi Liu, and Shuiwang Ji. Topology- aware graph pooling networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(12):4512-451 2021.
[22] Johannes Gasteiger, Janek GroB, and Stephan Günnemann. Directional message passing for molecular graphs. In International Conference on Learning Representations, 2020. URL https://openrevieg http:/iwwweeeeeeeeeeeeeeeeeeeeee www.ooorummdddddddiiiooee
[23] Johannes Gasteiger, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. Advances Neural Information Processing Systems, 34: 2021.
[24] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals. and George E Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 1263-1272. JMLR. org, 2017.
[25] Jonathan Godwin, Michael Schaarschmidt, Alexander L Gaunt, Alvaro Sanchez-Gonzake Yulia Rubanova, Petar Veličković, James Kirkpatrick, and Peter Battaglia. Simple GNN regularisation for 3D molecular property prediction and beyond. In International Conference on Learning Representations, 2022. URL httph::t///o/t /ttps///////peevee/e/ee wwwpperrviee.eeteffeooeoeoeeeeee wtt.oommmndaaaeeeaaooooo
[26] Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignae, and Max Welling. Equivariant diffusion for molecule generation in 3D. In International Conference on Machine Learning. pages 8867-8887. PMLR, 2022.
[27] Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, and Jure Leskovec. OGB- LSC: A large-scale challenge for machine learning on graphs. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL http://www....tee..oooooorrtraaa
(28) Weihua Hu, Muhammed Shuaibi, Abhishek Das, Siddharth Goyal, Anuroop Sriram, Jure Leskovec, Devi Parikh, and C Lawrence Zitnick. ForceNet: graph neural network for large-scale quantum calculations. arXiv preprint arXiv:2103.01456 2021.
[29] Bowen Jing, Gabriele Corso, Regina Barzilay, and Tommi S. Jaakkola. Torsional diffusion for molecular conformer generation. In ICLR Workshop on Deep Generative Models for Highly Structured Data, 2022. URL https: hwwpr//eeeeeeeeeeeeeeee/ee
[30] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, et al. Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873):583-555
2021.
[31] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. The Interna- tional Conference on Learning Representations, 2015.
[32] Johannes Klicpera, Shankari Giri, Johannes T Margraf, and Stephan Günnemann. Fast and uncertainty-awar directional message passing for -quiiiiiiiii molecules. In NeurlPS-W W, 2020.
[33] Dmitrii Kochkov, Tobias Pfaff, Alvaro Sanchez-Gonzalez, Peter Battaglia, and Bryan K Clark. Learning ground states of quantum hamiltonians with graph networks. arXiv preprint arXiv:21 10.0639,, 2021.
[34] Meng Liu. Youzhi Luo, Limei Wang, Yaochen Xie, Hao Yuan, Shurui Gui, Haiyang Yu, Zhao Xu, Jingtun Zhang, Yi Liu, Keqiang Yan, Haoran Liu, Cong Fu, Bora M Oztekin, Xuan Zhang, and Shuiwang Ji. DIG: A turnkey library for diving into graph deep learning research. Journal of Machine Learning Research, 22(240):1-9) 2021. URL http: http://mmmrrrrrrr// pttrgpapprs//22/
21-0343. 1tm0333.html.
[35] Yi Liu, Hao Yuan, Lei Cai, and Shuiwang Ji. Deep learning of high-order interactions for protein interface prediction. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 679-687, 2020.
12

---

[36] Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3D molecular graphs. In International Conference on Learning Representations, 2022. URI. https: opwnrevieeeeee..eeeeee net/forumm www.orrnmdddiiisss0t0000....
[37] Chengqiang Lu, Qi Liu, Chao Wang, Zhenya Huang, Peize Lin, and Lixin He. Molecular property prediction: A multilevel quantum interactions modeling perspective. In Proceedings of the AAA/ Conference on Artificial Intelligence, volume 33, pages 1052-1060, 2019.
[38] Haggai Maron, Heli graph networks. Advances in Neural Information Processing Systems, 32:2153-21644 2019.
Hadar Serviansky, and Yaron Lipman. Provably powerful
[39] Alex Morehead, Chen Chen, and Jianlin Cheng. Geometric transformers for protein interface contact prediction. In International Conference on Learning Representations, 2022. URL
[40] Raghunathan Ramakrishnan, Pavlo Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1-7, 2014.
[41] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference on machine learning, pages 9323-9332. PMLR, 2021.
[42] Jürgen Schmidhuber. Deep learning in neural networks: An overview. Neural Networks, 61: 85-117, 2015.
[43] Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. SchNet: A continuous-filte convolutional neural network for modeling quantum interactions. In Advances in Neural Information Processing Systems, pages 191-10011,000
[44] Kristof Schütt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, pages 9377-9388. PMLR, 2021.
[45] Muhammed Shuaibi, Adeesh Kolluru, Abhishek Das, Aditya Grover, Anuroop Sriram, Zachary Ulissi, and C Lawrence Zitnick. Rotation invariant graph neural networks using spin convolu- tions. arXiv preprint 1=XXi:2106.09577 2021.
[46] Gregor N. C. Simm, Robert Pinsler, and José Miguel Hernández-Lobate Reinforcement learning for molecular design guided by quantum mechanics. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 8959-8969. PMLR, 2020. URL http:/ http:://proceedings mlr. www..preessiii..e /t////mmmmmmm/. html.
[47] Gregor N. C. Simm, Robert Pinsler, Gábor Csányi, and José Miguel Herándezz- Symmetry-aware actor-critic for 3D molecular design. In International Conference on Learning Representations, 2021. URL /ttt:::///pppeeeiieee nwt. wett:/oorumiddd-E//ppp....
[48] Jonathan M Stokes, Kevin Yang. Kyle Swanson, Wengong Jin, Andres Cubillos-Ruiz, Nina M Donghia, Craig R MacNair, Shawn French, Lindsey A Carfrae, Zohar -oko--Aceemmnn,, et al. A deep learning approach antibiotic discovery. Cell, (80(4):68-770) 2020.
[49] Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and cannsaaiio-qquivriiaa neural networks for 3D point clouds. arXiv preprint /xXiv:/802.082) /9, 2018.
[50] Oliver T Unke and Markus Meuwly. PhysNet: A neural network for predicting energies, forces, dipole moments, and partial charges. Journal of chemical theory and computation, 15(6): 3678-3693, 2019.
[51] Clement Vignac, Andreas Loukas, and Pascal Frossard. Building powerful and equivariant graph neural networks with struetural message-passingg Advances Neural Information Processing Systems, 33:14143-114155 2020.
13

---

[52] Zhengyang Wang, Meng Liu, Youzhi Luo, Zhao Xu, Yaochen Xie, Limei Wang, Lei Cai,
Qi Qi, Zhuoning Yuan, Tianbao Yang, et al. Advanced graph and sequence neural networks for molecular property prediction and drug discovery. Bioinformatics, 38(9):257--258) 2022.
[53] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. MoleculeNet: a benchmark for molecular machine learning. Chemical science, 9(2):513-555 2018.
[54] Tian Xie and Jeffrey C Grossman. Crystal graph convolutional neural networks for accurate and interpretable prediction of material properties. Physical review letters, 120 (111:):445 145301, 2018.
[55] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2019.
[56) Zhao Xu, Youzhi Luo, Xuan Zhang, Xinyi Xu, Yaochen Xie, Meng Liu, Kaleb Dickerson, Cheng Deng, Maho Nakata, and Shuiwang Ji. Molecule3D: A benchmark for predicting 3D> geometries from molecular graphs. arXiv preprint arXiv:21
2021.
[57] Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Philipp Eiden, Hua Gao, Angel Guzman-Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, et al. Analyzing learned molec- ular representations for property prediction. Journal of chemical information and modeling, 59 (8):3370-3388, 2019.
[58] Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do transformers really perform badly for graph representation? Thirty-Fifth Conference on Neural Information Processing Systems, 2021. URL https://openreviis ntt//forumr1ce@ooooOOooOod
[59] Muhan Zhang. Zhicheng Cui, Marion Neumann, and Yixin Chen. An end-to-end deep learning architecture for graph classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
14

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix A.3.
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as a URL)? [Yes] See Appendix
A.6 and https: http:://ttthiee github..t.... www/diveelleeeeeeeeeeeoooooee.ee divelab/DGG.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix A.6.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [N/A] Error bars are not necessary for baseline methods and our method.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] All of these are reported in Appendix A.6.
4. If you using existing assets (e.g., code, data, models) or curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any assets either in the supplemental material as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "r_o9SeEHW_t", "reviewer": "Reviewer_eaXa", "summary": "This paper proposes ComENet designed for the representation learning on 3D molecular graphs. The framework involves the message passing schemes to capture local completeness and the rotation angle to help capture the global completeness. The paper shows the computational advantage of the proposed model over the existing one and the experimental results show its effectiveness.", "questions": "I hope the author could address the concerns raised in the above section.", "limitations": "The paper does not discuss the limitation and the social negative impact.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Instead of the intuition, the paper defines the global completeness and provides theoretical ground for the model to capture the global completeness.\n2. The experiments show the effectiveness of the model.\n\nWeakness:\n1. There are a lot of places that need to be further clarified:\n    a) Line 33, in O(nk^2), what are n and k? They should be explicitly defined before being used.\n    b) Line 82: what is m in m x d?\n    c) Line 88: What are the function R() and SE(3)? I would suggest the author to add those in the main text of the appendix to make the paper more readable.\n    d) Proposition 1: what is the definition of \"strongly connected\"? Does it mean the line 199 that two arbitrary nodes are connected by at least one path?\n    e) Line 245-246: it would be more clear to say \"node feature vector v\".\n    f) Line 113: What is the definition of L_{ij, j\\in N_i}? It seems that its notation is different from previously defined L_i.\n2. A tiny typo: line 94: It should be achieve rather that achieves.\n3. I'm confused about how to rotation angle. In the example of Figure 1 (b), the angle is calculated as the angle between two planes indicated in the figure. But why is it computed between plane f_{i\\j}-i-j and f_{j\\i}-i-j rather than other planes that contain the edge i-j (e.g., the plane formed by i, j and the node at the top right corner)? How to choose plane to calculate the angle?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "UJ-UBsR6XcN", "reviewer": "Reviewer_tzSo", "summary": "This paper introduces a message-passing paradigm on 3D molecular graphs with better completeness (bijectivity) and additionally other advantages including operating within a 1-hop neighborhood, lower computational complexity, and faster training and inference. Analysis and experiments show that the proposed method can perform comparably well on multiple datasets with obviously increased efficiency.", "questions": "Questions:\n- As \"unless otherwise specified, the values for baseline methods are taken from the referred papers\" (line 320), can I also assume that the (macro) network architectures for different baselines are not necessarily the same? I'd be interested in a comparison of the message-passing paradigm itself to other message-passing paradigms (e.g. the one in SphereNet[34]) under the same (macro) network architectures. If such a comparison is hard to conduct, some other details such as the network size of the baselines and ComENet may also help make the statements more convincing.\n\nSuggestions:\n- I may suggest first introducing the message passing scheme in Section 2.5 (e.g. Equation 1) and then showing its completeness (Section 2.2-2.4) -- reading the deductions with an overview of the final formulation and a summary of notations may help with faster comprehension. But just my two cents.", "limitations": "I probably missed it but I haven't found discussions of limitations and potential negative societal impact in the paper.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper brings up the problem of local completeness != global completeness which is kinda overlooked in prior works.\n- The paper proposes a useful message-passing paradigm on molecular graphs with better completeness and efficiency.\n- Systematical analysis and extensive experiments on multiple tasks under different evaluation metrics are provided.\n- Good experimental results and analysis support the effectiveness of the proposed method.\n\nWeaknesses:\n- To my understanding, the main arguments of the paper are \"completeness\" and \"efficiency\" of the proposed method, and my feeling is that the latter is better addressed than the former in the experiments. As stated in Section 2.3, completeness is most important in identifying structures like conformers. More specific experiments on this (in addition to the general tasks) or some relevant details (e.g. how many conformers exist in the dataset) for the datasets could make the argument stronger, as a higher number on a generic task can be affected by many factors.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "7IAvicH5ns", "reviewer": "Reviewer_SSbD", "summary": "The paper proposes a new neural network architecture to learn representations of 3D graphs, primarily\noriented towards applications to molecules.  The method uses a message passing scheme\noperating in 1-hop neighborhoods of each node to propagate information throughout the graph efficiently.\nIt is said to guarantee global completeness, i.e., distinct conformers of the same molecule yield\ndistinct representations.", "questions": "p.2, line 83: need a reference for SchNet at first mention; how is the distance generated for an edge?\n\np.3, line 89: missing definition/reference for SE(3).\n\np.3, line 91: what is included in \"differences in structures\"? what does \"structure\" refer to?\n\np.3, line 94: need a reference for DimeNet at first mention.\n\np.3, line 95: how large is a local neighborhood with guaranteed completeness by SphereNet?\n\np.6, Figure 3: how is the single-atom unit shown in the figure applied to the entire molecule?\nIt will be useful to show an example with, say, one of the conformers in Figure 2(b).\nIs each atom in a molecule to be passed through it one by one in turn?  Or are they concatenated\nto serve as input to another network for a \"downstream\" prediction task?\n\np.8, p.9: the list of Properties in Tables 4 and 5 requires spelled out names and also references for their definitions.\n\n", "limitations": "The datasets used in the experiments contain relatively small molecules (# nodes/graph < 100).\nIs it possible to directly apply this method to much larger molecules?\nSome comments on the challenges and limitations there are needed. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The proposal is sound and appears to have good reference value for practitioners.\nThe method is tested with large public datasets and compared to prior art to demonstrate its effectiveness.\n\nAt places the writing of the paper requires some clarification.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "3hTe-pkNXpf", "reviewer": "Reviewer_h4eH", "summary": "The authors propose ComENet, a new network for molecular property prediction with 3D information. The authors define a new message passing way (see equation (1)) and a new network backbone Figure 3.", "questions": "Please refer to the ``Strengths And Weaknesses'' section.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Originality\n\n1. The general framework is similar to PaiNN and DimeNet++, which is not novel to me.\n\n2. In Eqn.(1), seems more features are used in mesage passing. The method is new but not surprising. \n\n\n## Significance\n\n1. The accuracy in Table 3 and 4 are not as good as baselines. \n\n2. On OC20, the authors did not compare with the state-of-the-art algorithms.\n\n3. If efficiency is the most important part of this paper, I am not sure whether we can directly apply Eqn(1) to existing backbones like DimeNet++?\n\n## Clarify\n\n1. Section 2: What are the key points to deliver? And how & why they help the models?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept