Spherical Message Passing for 3D Molecular Graphs
Authors: Yi Liu
Limei Wang
Meng Liu
Yuchao Lin
Xuan Zhang
Bora Oztekin
Shuiwang Ji
Venue: ICLR 2022 Poster (2022)

Abstract: We consider representation learning of 3D molecular graphs in which each atom is associated with a spatial position in 3D. This is an under-explored area of research, and a principled message passing framework is currently lacking. In this work, we conduct analyses in the spherical coordinate system (SCS) for the complete identification of 3D graph structures. Based on such observations, we propose the spherical message passing (SMP) as a novel and powerful scheme for 3D molecular learning. SMP dramatically reduces training complexity, enabling it to perform efficiently on large-scale molecules. In addition, SMP is capable of distinguishing almost all molecular structures, and the uncovered cases may not exist in practice. Based on meaningful physically-based representations of 3D information, we further propose the SphereNet for 3D molecular learning. Experimental results demonstrate that the use of meaningful 3D information in SphereNet leads to significant performance improvements in prediction tasks. Our results also demonstrate the advantages of SphereNet in terms of capability, efficiency, and scalability.

Published as a conference paper at ICLR 2022
SPHERICAL MESSAGE PASSING FOR 3D MOLECULAR GRAPHS
Liu, Limei Wang', Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin & Shuiwang Ji Department of Computer Science & Engineering
Texas A&M University
College Station, TX 77843, USA
(yiliu, 1imei, mengliv, truskallin, xuan zhang, bora, sjil i @tamu. edu
ABSTRACT
We consider representation learning of 3D molecular graphs in which each atom is associated with a spatial position in 3D. This is under-explored area of research, and principled message passing framework is currently lacking. In this work, we conduct analyses in the spherical coordinate system (SCS) for the complete identification propose the spherical message passing (SMP) as novel and powerful scheme for 3D molecular learning. SMP dramatically reduces training complexity, enabling it to perform efficiently on sarge-scale molecules. In addition, SMP is capable of distinguishing almost all molecular structures, and the uncovered cases may not exist in practice. Based meaningful physically-based representations of 3D information, we further propose the SphereNet for 3D molecular learning. Experimental results demonstrate that the use of meaningful 3D information SphereNet leads to significant performance improvements in prediction tasks. Our results also demonstrate the advantages of SphereNet in terms of capability, efficiency, and scalability. Our code is publicly available as part of the DIG library (https:/ /tgi thub. com/ divelavllll IaG).
3D graph structures. Based on such observations, we
INTRODUCTION
In many real-world studies, structured objects such as molecules naturally modeled graphs (Gori et al., 2005; Wu et al., 2018; Shervashidze et al., 2011; ; Fout et al., 2017; 7; Liu et al., 2020; Wang et al., 2020). With the advances of deep learning, graph neural networks (GNNs) have been developed for learning from graph data (Kipf & Welling, 2017; Defferrard et al., 2016; Veličković et al., 2018; Zhang al., 2018; Xu al., 2019; Gao & Ji, 2019; Gao et al., 2018; 2020). Currently, the message passing scheme (Gilmer et al., 2017; Sanchez- Sanchez-Gonzalez et al., 2020; Vignac et al., 2020; Battaglia et al., 2018) is one of the commonly used architectures for realizing GNNs. In this work, we aim at developing a novel message passing method for 3D graphs. Generally, 3D molecular graph contains 3D coordinates for each atom given in the Cartesian system along with the graph structure (Liu et al., 2019; Townshend et al., 2019; Axelrod &
2020). Different types of relative 3D information can be derived from 3D molecular graphs, and they can be important in molccular learning, such as bond lengths, angles between bonds (Schütt et al., 2017; Klicpera et al., 2020b).
We first investigate complete representations of 3D molecules. This requires the graph structure to be uniquely defined by relative 3D information. To this end, conduct formal analyses in the spherical coordinate system (SCS) (Chen et al., 2019), and show that relative location of each atom in 3D graph is uniquely determined by three geometries, including distance, angle, and torsion. However, such completeness needs to involve edge-based based 2-hop information, leading to excessively high computational complexity. To circumvent the computational cost, we propose novel message passing scheme, known as the spherical message passing (SMP), for fast and accurate 3D molecular learning. Our SMP is efficient and approximately complete in representing 3D molecules. First, we design novel strategy to compute torsion, which only considers edge-based 1-hop information, thus substantially reducing training complexity. This enables the generalization of SMP to large-scale
"These authors contribute equally this work.

---

Published as a conference paper at ICLR 2022
molecules. In addition, we show that our SMP can distinguish almost all 3D graph structures. The uncovered cases seem rarely appear in practice. By naturally using relative 3D information and a novel torsion, SMP yields predictions that are invariant to translation and rotation of imput graphs.
We apply the SMP to real-world molecular learning, where meaningful physical representations are needed. Geometries (d, f. 4)) specified by SMP are then physically represented by & (d,, 0,4), which can be a solution to the Schrödinger equation, as described in Sec. 4. Based on this, we develop the spherical message passing neural networks, known as the SphereNet, for 3D molecular learning. We conduct experiments on various types of datasets including OC20, QM9, and MD17. Results show that, compared with baseline methods, SphereNet achieves the best performance without increasing the computing budget. Ablation study reveals contributions and necessity of different types of 3D information, including distance, angle, and torsion. Particularly, we compare with a complete message passing scheme that can distinguish all 3D graph structures but involves adge-based 2-hop information. Experimental results show that SphereNet achieves comparable performance but reduces running time by 4 times. This suggests the use of SehereNet in practice rather than the complete message passing scheme, whose computational complexity prevents its use on large molecules.
COMPLETE REPRESENTATIONS OF MOLECULES
Equivariant graph neural networks (EGNNs) represent one research area for 3D molecular graphs, as introduced in Sec. 5. These methods usually take coordinates in the Cartesian coordinate system (CCS) for all atoms as the raw input. Hence, all the network layers need to be carefully designed to be equivariant. The computing of some equivariant components is expensive, like spherical harmonics and Gaobsh-Grrrannn coefficients (Thomas et al., 2018; Fuchs et al., 2020). In addition, the complicated SE(3) group representations may not necessary for molecular learning where final representations are generally required to be invariant. In this work, we focus on the other category of methods that take relative position information purely as input to graph learning models. Relative 3D information could be distance angle, which is inherently invariant to translation and rotation of input molecules. is natural to consider such information in the spherical coordinate system (SCS). We start by investigating the structure identification of 3D molecules in the SCS. For any point in the SCS, its location is specified by a 3-tuple (d, 8, (,), where d, 8, and &o denote the radial distance, polar angle, and the azimuthal angle, respectively. When modeling 3D molecular graphs in the SCS, any atom i can be the origin of local SCS, and d, 0, and yp naturally become the bond length, the angle between bonds, and the torsion angle, respectively. Thus, the relative location of each neighboring atom of atom i can be specified by the corresponding tuple (d, 0, sp). Similarly, the relative location of each atom the 3D molecular graph can be determined, leading to the identified structure, which is naturally invariant to translation and rotation of the input graph. The SCS can be easily converted from the Cartesian coordinate system, thus, the tuple (d, 0, up) can be easily obtained.
As in Fig. I, we use the chemical structure of the hydrogen peroxide (H₂O2) to show how d, 0, and o are vital for the molecular structure identification. is obvious that the structure is uniquely defined by the three bond lengths d1, d2, d3, the two bond angles 01, f2, and the torsion angle yp. Note that the input may not contain all pairwise distances (all possible bond lengths). This is because the atomic connectivity is usually based on real chemical bonds and cut-off distances. The cut-off distance is usually set as a hyperparameter. It is hard to guarantee that the cut-off is larger than any pairwise distance in a molecule. Hence, in this example. H-H bond length may not be considered as input if the cut-off is small. Setting a proper cut-off is even harder for other complicated and large molecules where a distance between two atoms could be large. In addition, considering all pairwise distances will cause severe redundancies, dramatically increasing the computational complexity. The model also easily gets confused by excessive noise, leading to unsatisfactory performance. From the perspective of completeness, using all pairwise distance is not capable of recognizing the chirality property. To overcome the above challenges, we use a combination of distance, angle, and torsion for rigorous design and accurate learning. Apparently, the two O-H bonds can rotate around the O-O bond without changing any of the bond lengths and bond angles. In this situation, however, the torsion angle &p changes and the structure of the H₂Oz
d2
d
d3
H
H
Figure 1: The chemical structure of the H2 O2.
2

---

Published as a conference paper at ICLR 2022
rk
Tk
reference plane
42
Ck
41
e
Sk
95
(a)
Figure 2: (a). The message aggregation scheme for the spherical message passing. (b). An illustration for computing torsion angles the spherical message passing architecture.
varies accordingly. The importance torsion angle has also been demonstrated in related research domains. Garg et al. (2020) formally shows that the torsion along with the port numbering can improve the expressive power of GNNs in distinguishing geometric graph properties, such as girth and circumference, etc. Other studies (Ingrabam et al., 2019; Simm et al., 2020) reveal that protein sequences and molecules can be accurately generated by considering the torsion in the given 3D structures. In this work, we propose SMP to systematically consider distance, angle, and torsion for approximately complete representation learning of 3D molecular graphs. Note that by using angle and torsion, SMP can easily recognize the chirality property.
SPHERICAL MESSAGE PASSING
3.1 MESSAGE PASSING SCHEME
Currently, the class of message passing neural networks (MPNNs) (Gilmer et al., 2017) are one of the most widely used architectures for GNNs. Based upon the completeness analyses in Sec. we propose to perform message passing in the spherical coordinate system (SCS), resulting in a novel and efficient scheme known as spherical message passing (SMP). We show that message passing schemes used in existing methods, such as SchNet and DimeNet, are special cases of SMP.
We first formally define 3D molecular graph, which usually represented as 4-tuple G (u, V. E, P). The E R is a global feature vector for the molecular graph G. V= (V; the set of atom features, where each V; E R.d, the feature vector for the atom i. E {(ex, rk, 84)]==im is the set of edges, where each eş E Rede is the feature vector, Fkk is the index of the receiver atom, and 5% is the index the sender atom for the edge k. P = {1% J is the set of 3D Cartesian coordinates that contains 3D spatial information for each atom. In addition, let E, = {(ok. Tk: { fr=s,k=:mm denote the set of edges that point to the atom i, and N₁ denote the indices of incoming nodes of atom i. The outputs after message passing process include the updated global feature vector u' E Rdu, the updated atom features V Jim and the updated edges E' [(e',, Tk; 8k)) k=1mm
An illustration of the message aggregation scheme for SMP is provided in Fig. 2 (a). Apparently, the embedding of the atom тı obtained by aggregating each incoming message eg. The message e₁ updated based on E.,. the set of incoming messages pointing to the atom sj. Let q denote the sender atom of any message E.g. Hence, we can define a local SCS, where 5 serves the origin, and the direction of the message naturally serves as the z-axis. We define neighboring atom o of s as the reference atom. Thus, the reference plane is formed by three atoms $k, rk. and For atom q. its location is uniquely defined by the tuple (d, 0, sp), as shown in Fig. 2 (a). Specifically, d determines direction to update the message eg. The torsion angle yp
its distance to the atom 81:, 0 specifies is formed by the defined reference plane and the plane spanned by 5k. Tk. and Intuitively, as an advanced message passing architecture in spherical coordinates for 3D graphs, SMP specifies relative location for any neighboring atom q by considering all the distance, angle, and torsion information, leading to more comprehensive representations for 3D molecular graphs.
Generally, the atom sk may have several neighboring atoms, which we denote 91. qt- It is easy to compute the corresponding bond lengths and bond angles for these t atoms. The SMP computes torsion angles by projecting all the atoms to the plane that is perpendicular to eg and intersect
3

---

Published as a conference paper at ICLR 2022
with Sy. Then on this plane, the torsion angles are formed in predefined direction, such as the anticlockwise direction. By doing this, any atom naturally becomes the reference atom for its next atom in the anticlockwise direction. Notably, the sum of these t torsion angles is 2s. A simplified case is illustrated in Fig. 2 (b). The atom 85: has three neighboring atoms @1, 42, and qs: ya is the reference atom for ς1, and they form 90; 91 is the reference atom for 72, and they form (P2; similarly, 42 is the reference atom for Q3. and they form 423. It is obvious that the sum of (p1, P22, and yp is 2T. As the torsion defined relatively, qi can be picked arbitrarily, which will not affect the output of the message passing scheme, as we perform summation when aggregating information to Sy from its neighbors q1, 42, and gs. Notably, by designing each atom to be the reference atom of the next one in the predefined direction, invariance is effectively achieved because reference atom is naturally relative. addition, our method computes torsion within edge-based 1-hop neighborhood. Even though torsion angle involves four atoms, our design avoids the number of torsion angles to be exponential, but makes it the same as the number of neighboring atoms. Hence, is efficient and will not cause time or memory issues. Formally, the proposed SMP can be defined in the SCS
e Φ eee VoEa Vag D ((rA)) (IrA)eEmuuANN..)) v = 6" (v (E)) u' = x² pu, (V')),
where o², σ², and 6t are three information update functions on edges, atoms, and the whole graph, respectively. p² p²- and p" aggregate information between different types of geometries. Especially, in SMP, the 3D information in P is converted and incorporated
update each message e*. Hence, SMP employs another position aggregation function pr for message update. Notably, the main difference between our SMP scheme defined in Eq. and the GN framework in Battaglia al. (2018) is the inclusion of 3D information P. In line with the research area described in Sec. 5.1.2, we focus on such 3D information and develop a systematic solution to incorporate it completely and efficiently. Detailed description of these functions is given in Appendix A.
3.2 COMPLETENESS VERSUS EFFICIENCY
The identification criteria described in Sec. 2 can fully determine the structure of a 3D molecule, but involves edge-based 2-hop informa- tion. Hence, the computa- tional complexity is as size- able as O(nk:)), where the number of atoms, and k denotes the average num- ber of neighboring atoms for each center atom. Un- fortunately, such design can hardly generalize to large molecular graphs. To this end, we propose SMP as an efficient and scalable scheme to realize message passing in SCS. Our SMP only involves edge-based 1- hop information, thus the time complexity is reduced to O(nk²). This enables the application of SMP to large molecules, like the newly released OC20 data (Chanussot al., 2020). We rigorously investigate the completeness of SMP and show that it can distinguish even complex geometric properties such as chirality, as indicated by Fig. (a) and Fig. 3 (b). As SMP uses the last atom as the reference atom (like q2 is the reference atom for 43 in Fig. 2 (b)) in predefined direction, the relative order between adjacent atoms is considered while the absolute order is neglected. Hence, SMP can not distinguish the two molecules illustrated by Fig. 3 (b) and Fig. 3 (c). However, this scenario may not exist in nature. This is also demonstrated in experiments that our SMP achieves comparable performance with the complete representations, while the latter induces huge time complexity and severe memory issues.
d
80
90"
80"
90*
90
120
S
60-
60
sp*
120'
120"
4
(a)
(b)
(c)
Figure 3: An illustration of cases that SMP can and cannot distinguish. All the neighboring nodes of 8 are projected to the plate perpendicular to the message of interest. We assume all the distances and angles are fixed (the molecules can be more easily distinguished otherwise). Hence, all the angle shown are torsion angles and they are formed in the anticlockwise direction. (a) and (b) are chiral and SMP can distinguish them. This is because in (a), gf (90"), (60"). g (120%), La (90%); in (b), df (60%), d' (120%). o (90%). of (90%). SMP cannot distinguish (b) and (c) but this scenario may not exist in nature. Zgj suge in (b) and Lg swg's in (c) usually are different as 4/2 and g/ss are different atoms and the corresponding distances and angles are the same.
4

---

Published as a conference paper at ICLR 2022
3.3 RELATIONS WITH PRIOR MESSAGE PASSING METHODS
When developing message passing methods for 3D graphs, sphere message passing is an advanced scheme where the relative location of each atom is more specified. The development for 3D graphs with relative information is still in early stage. To our best knowledge, there exist several notable methods in literature, and they can be viewed as special cases of the SMP, as they capture partial 3D position information. For example, the SchNet and PhysNet consider distance, and the DimeNet encodes directional information. Formally, these methods can be perfectly fit the Spherical scheme defined in Eq. 1. We describe the details of these methods in Appendix C. Notably, compared with prior models, SMP provides a rigorous justification on its completeness with failure cases clearly described. Importantly, SMP is developed based on the identification analyses of 3D molecular graphs. Hence, it aims at learning complete data representations for 3D molecular graphs, rather than simply including extra 3D information (like angle or torsion).
4 SPHERENET
The obtained 3-tuple (d, 0, sp) indicates the relative location of any atom in a 3D molecular graph. However, it cannot serve as the direct input to neural networks as it lacks meaningful representations. Essentially, molecules are quantum systems thus the representation design needs to follow physics laws. An important aspect is to choose appropriate basis functions that transform the 3-tuple (d, 0, p) into physically-based representations. Several basis functions have been explored in Hu et al. (2021); Klicpera et al. (2020b), including MLP, Gaussian and sine functions, spherical Bessel basis, and spherical harmonics. Especially, spherical Bessel is shown to be the best basis for encoding distance, and spherical harmonics is the most appropriate one for encoding angle (Hu et al., 2021; Klicpera et al., 2020b). We denote the final representation as (cl, 0, pp). Referring to theories in Griffiths & Schroeter (2018); Cohen et al. (2019); Klicpera et al. (2020b), one form of the representation can be expressed as (d, 0, yp) = je Sem d (7m where je(-) is a spherical Bessel function of order Y!" is spherical harmonic function of degree m and order denotes the cutoff, 3rm. is the n-th root of the Bessel function of order E. We also have l E [0, ................................ L - 1J. m E [-6,-.., E] and [1..... NN. L and N denote the highest orders for the spherical harmonics and spherical Bessel functions, respectively. They are hyperparameters in experimental settings. In addition, we can derive two simplified representations L/ (d) and & (d, 0) from L (d, 0,4p).
Based upon the spherical message passing scheme described in Sec. 3 and physical representations, we build the SphereNet for molecular learning. Apparently, SphereNet can produce data representations that are both accurate and physically meaningful. By incorporating the position information in spherical coordinates, SphereNet also generates predictions invariant to translation and rotation of input molecules. Following the architecture design in the research line stated in Sec. 5.1.2, our network is composed of an input block, several interaction blocks, and an output block. For clear description, we assume the message ex for the edge & in Fig. 2 and Eq. (1) is the message for update. The update process and detailed architecture for the SphereNet are provided in Appendix B.
RELATED WORK
5.1 METHODS FOR 3D MOLECULAR GRAPHS
5.1.1
EQUIVARIANT GRAPH NEURAL NETWORKS
One research line for 3D molecular graphs equivariant graph neural networks (EGNNs) includ- ing tensor field networks (TFNs) (Thomas et al., 2018), trans-rrrsssormers (Fuchs et al., 2020), PaiNN (Schütt al., 2021). NequlP (Batzner et al., 2021), Noisy Nodes (Godwin et al., 2022), etc. The raw input of these methods usually contains the absolute information, such as coordinates in the Cartesian coordinate system. In intermediate layers, absolute information could be decomposed into partial absolute information and partial relative information needed. A simple example is that a vector can be decomposed into its orientation (absolute) and length (relative) (Thomas et al., 2018; Schütt et al., 2021). Apparently, network components of these methods should be carefully A designed to be equivariant. The preliminary work like TFNs were developed for 3D point clouds. However, it is demonstrated that for molecules whose downstream tasks usually require the systems
5

---

Published as a conference paper at ICLR 2022
to be invariant, the complicated SE(3) group representations not necessary but S representations are enough (Klicpera et al., 2021). Moreover, their performance on molecular tasks is not satisfactory.
5.1.2 INVARIANT GRAPH NEURAL NETWORKS
Another category of methods purely take relative 3D information as input, such as distances between atoms, angles between bonds, angles between planes, etc. Hence, the network is naturally invariant. The development of these methods is in early stage, and existing studies focus leveraging different geometries. The SchNet (Schütt et al., 2017) incorporates the distance information during the information aggregation stage by using continuous-filte convolutional layers. The PhysNet (Unke & Meuwly, 2019) integrates both the atom features and distance information in the proposed interaction block. The DimeNet (Klicpera et al., 2020b) is developed based on the PhysNet and moves a step forward by considering directional information in the interaction block. The GemNet (Kliepera et al., 2021) is proposed recently for universal molecular representations. The OrbNet (Qiao et al., 2020) combines distance information with the atomic orbital theory to design important SAAO features as inputs to GNNs. Generally, the use of 3D position information usually results in improved performance. However, existing methods simply include additional geometries such as distance and angle, and there lacks a rigorous justification on how different geometries contribute to the information aggregation process. We conduct formal analysis and show that all the distance, angle, and torsion are necessary for 3D molecular identification, based on which we propose SphereNet to generate more powerful molecular representations.
5.2 METHODS FOR OTHER OBJECTS MODELED AS GRAPHS
Besides molecules, many other data objects are also represented as graphs, such as 3D point clouds (Guo et al., 2020; Simonovsky & Komodakis, 2017; Shen et al., 2018; Landrieu & Si- monovsky. 2018) and meshes (Bronstein ct al., 2021; De Haan et al., 2020; Perraudin et al., 2019). When modeling 3D point clouds as 3D graphs, points represented as nodes and connections between points are directed edges. Existing methods mainly capture distance information from local neighborhood in 3D space. In DGCNN (Wang et al., 2019b), novel layer namely EdgeConv is proposed to aggregate distance- edges features for node learning. In Landrieu & Boussaha (2019), neighborhood radius along with spatial orientation is incorporated in local point embedding. The work Wang et al. (2019a) proposes a graph attention convolution for 3D point clouds. Generally, these methods can be fit into our message passing scheme defined in Eq. 1. The work De Haan et al. (2020) is an exemplary study that formulates meshes as graphs with considering geometrical information. The used convolutional kernel depends on the angle between a predefined reference edge and any other edge projected to the tangent plane for each vertex. It focuses on the design of gauge equivariance rather than the learning of complete geometry information. In this work, we study the complete learning of 3D molecules and leave extensive studies on other data types as future work.
6 EXPERIMENTAL STUDIES
6.1 EXPERIMENTAL SETUP
We apply our SphereNet to three benchmark datasets, including Open Catalyst 2020 (OC20) (Chanus-
sot al., 2020), QM9 (Ramakrishnan al.. 2014), and MD17 (Chmiela et al., 2017). Baseline methods include PPGN (Maron et al., 2019), SchNet (Schütt et al., 2017), PhysNet (Unke & Meuwly, 2019), Cormorant (Anderson et al., 2019), PaiNN (Schütt et al., 2021), NequlP (Batzner et al., 2021), 1), MGCN (Lu et al., 2019), DimeNet (Klicpera et al., 2020b), DimeNet++ (Kliepera et al., 2020a), GemNet (Klicpera et al., 2021), CGCNN (Xie & Grossman, 2018), and sGDML (Chmicla et al., 2018). Detailed configurations of all the models used in the following sections are provided in the supplementary material. Unless otherwise specified, for all the baseline methods, we report the results taken from the referred papers or provided by the original authors. For the SphereNet, all models are trained using the Adam optimizer (Kingma& & Ba, 2014). The optimal hyperparameters are obtained by grid search. Network configurations and search space for all models are provided in Appendix D.

---

Published as a conference paper at ICLR 2022
Table Comparisons between SphereNet and other models on IS2RE in terms of energy MAE and the percentage of EwT of the ground truth energy. Results reported for models trained on the All training dataset. The best results are shown in bold.
Energy MAE [
EwT
Model
ID
OOD Ads
OOD Cat
OOD Both
Average
ID
OOD Ads
OOD Cat
OOD Both
Average
CGCNN
0.6203
0.7426
0.6001
0.6708
0.6585
3.36%
2.11%
3,53%
2.29%
2.82%
SchNet
0.6465
0.7074
0.6475
0,6626
0.6660
2.96%
2.22%
3.03%
2.38%
2.65%
DimeNet++
0,5636
0.7127
0,5612
0.6492
0.6217
4.25%
2.48%
4.40%
2.56%
3.42%
GemNet-T
0.5561
0.7342
0,5659
0.6964
0.6382
4.51%
2.24%
4.37%
2.38%
3.38%
SphereNet
0.5632
0.6682
0.5590
0.6190
0.6024
4.56%
2.70%
4.59%
2.70%
3.64%
6.2 OC20
The Open Catalyst 2020 (OC20) dataset is newly released large-scale dataset for catalyst discovery and optimization (Chanussot et al., 2020). It comprises millions of DFT relaxations across huge chemical structure space such that machine learning models can be fully trained. We focus the IS2RE task in this work, and description of the data is provided in Appendix E. Results for CGCNN, SchNet, and DimeNet++ not contain results on the OC20 dataset, We use the publicly available code from the OC Project website to produce results for GemNet-T. We report evaluation results of fixed epochs for SphereNet. Following setting in Chanussot et al. (2020), we use the direct approach and all the training data for training models. The used metrics are the energy MAE and the percentage of Energies within a Threshold (EwT) of the ground truth energy. Table 1 shows that the SphereNet achieves the best performance on 3 out of splits and the average terms of energy MAE. For EwT, SphereNet the best on all the 4 splits. Specifically, it reduces the average energy MAE by 0.019, which is 3. 301% of the second best model. addition, it improves the average EWT from 3.42% to 3.64%, which is a large margin considering the inherently low EWT values.
provided in Chanussot al. (2020). The original GemNet paper does
(Hu ct al., 2021) and GemNet (Klicpera et al., 2021) are recently proposed for quantum system learning. A prominent advantage for ForceNet is its high efficiency and scalability to large molecules. ForceNet focuses on S2EF thus there are no original results for the IS2RE task. However, DimeNet++ is slightly better than ForceNet in terms of performance, and our SphereNet outperforms DimeNet++ significantly. GemNet has two variants GemNet-T and GemNet-Q. GemNet- T considers distance and angle information input, and contains an effective architecture with novel network components, such as bilinear layers and scaling factors. We can see GemNer-T is similar as DimeNet++ terms of performance. GemNet-Q Q is claimed be able to capture universal representations of molecules, However, it considers edge-base 2-hop information and the time complexity is extremely high. It may not be configured properly on the large catalyst molecules.
Notably, ForceNet
6.3
QM9
We apply the SphereNet to the QM9 dataset, which is widely used for predicting various properties of molecules. consists organic molecules composed of up heavy atoms. Thus, this test can examine the power of the SphereNet for similar quantum chemistry systems. The dataset is original split into three sets, where the training set contains 110,000, the validation set contains 10,000, and the test set contains 10,831 molecules. For energy-related properties, the training processes use the unit eV. All hyperparameters are tuned on the validation set and applied to the test set. We compare our SphereNet with baselines using mean absolute error (MAF) for each property and the overall mean standarized MAE (std. MAE) for all the 12 properties. The comparison results are summarized in Table 2. SphereNets achieves best performance on 3 properties. It also improves the overall mean std. MAE of the QM9 dataset from 0.98 to 0.91 and sets the new state of the art. Notably, the most recent method PaiNN uses the same data splits as SphereNet in terms of sample numbers. Its final performance is the average of three different runs three random splits. We follow such settings and run SphereNet four properties including еномО, ELUMO, Up. and p. The corresponding results are 22.9 + 0.2, 18.8 + 0.2, 6.28 + 0.05, and 0.0243 + 0.00, respectively. is obvious that these results are highly close those Table thus, we can draw consistent conclusions.
5 properties and the second best performance
http::://gitttbb..mmmmmm....mc Cata wrtpaaaalsstttojjt////ooo
7

---

Published as a conference paper at ICLR 2022
Table Comparisons between SphereNet and other models in terms of MAE and the overall mean std. MAE on QM9. '-' denotes no results are reported in the referred papers for the corresponding properties. The best results are shown in bold and the second best results are shown with underlines.
Property
Unit
PPGN
SchNet PhysNet Cormorant MGCN DimeNet DimeNet++ PaiNN SphereNett
0.047
0.033
0.0529
0.13
0.0560
0.0286
0.0297
0.012
0.0245
ag
0.131
0.235
0.0615
0.092
0.0300
0.0469
0.0435
0.045
0.0449
HOMO
meV
40.3
41
32.9
36
42.1
27.8
24.6
27.6
22.8
LUMO
meV
32.7
34
24.7
36
57.4
19.7
19.5
20.4
18.9
A
meV
60.0
63
42.5
60
64.2
34.8
32.6
45.7
31.1
ao²
0.592
0.073
0.765
0.673
0.110
0.331
0.331
0.066
0.268
ZPVE
meV
3.12
1.7
1.39
1.98
1.12
1.29
1.21
1.28
1.12
Uo
meV
36.8
14
8.15
28
12.9
8.02
6.32
5.85
6.26
U
meV
36.8
19
8.34
14.4
7.89
6.28
5.83
6.36
H
meV
36.3
14
8.42
14.6
8.11
6.53
5.98
6.33
G
meV
36.4
14
9.40
16.2
8.98
7.56
7.35
7.78
G
mol K
0.055
0.033
0.0280
0.031
0.0380
0.0249
0.0230
0.024
0.0215
std. MAE
%
1.84
1.76
1.37
2.14
1.86
1.05
0.98
1.01
0.91
Table 3: Comparisons between SphereNets and other models in terms MAE of forces MD17. WoFE indicates weight of force over energy in loss functions. Results of all baseline models are directed taken or adapted (if the unit varies) from the original papers, and SphereNet uses two WoFEs in line with the original papers of different baselines for fair comparisons. The best results are shown in bold and the second best results shown with underlines.
WoFF = 100
WoFF: = 1000
Molecule
sGDML SchNet DimeNet SphereNet NequlP GemNet-T GemNet-Q SphereNet
Aspirin
0.68
1.35
0.499
0.430
0.353
0.220
0.217
0.209
Benzene
0.20
0.31
0.187
0.178
0.186
0.145
0.145
0.147
Ethanol
0.33
0.39
0,230
0.208
0.204
0.086
0.088
0.091
Malonaldehyde
0.41
0.66
0.383
0.340
0.328
0.155
0.160
0.172
Naphthalene
0.11
0.58
0.215
0.178
0.105
0.055
0.051
0.048
Salicylic acid
0.28
0.85
0.374
0.360
0.242
0.127
0.125
0.113
Toluene
0.14
0.57
0,216
0.155
0,102
0.060
0,060
0.054
Uracil
0.24
0.56
0.301
0.267
0.173
0.097
0.104
0.106
std. MAE
1.11
2.38
1.10
0.97
0.79
0,45
0.45
0.44
6.4 MD17
The MD17 dataset is used to examine the expressive power of SphereNet for molccular dynamics simulations. Following the settings in Schütt et al. (2017); Klicpera al. (2020b), train separate model for each molecule to predict atomic forces. We use 1000 samples for training, and each of the eight molecules has both the validation and test sets. Note that all the baseline models employ joint loss of forces and conserved energy during training. In the original SchNet (Schütt et al., 2017) and DimeNet (Klicpera al., 2020b) papers, the authors the weight of force over energy (WoFB) to 100, while the NequiP (Batzner et al., 2021) and GemNet (Klicpera et al., 2021) papers use weight of 1000. As the WoFE tends affect the force prediction significantly, we perform SphereNet with both WoFE values for fair comparisons. PaiNN (Schütt et al., 2021) uses neither 100 nor 1000 as WoFE, so we not compare with it on MD17. The results for forces are reported in Table
Note that for Benzene, all models are evaluated on Benzene17, thus, the result for sGDML, is 0.20 rather than 0.06 (Benzene18) We can observe from the table that when WoFE is 100 for all models, SphereNet consistently outperforms SchNet and DimeNet by largin margins. Notably, sGDML is one of the original work that created the MD17 dataset with carefully -eiillydd features. Compared with sGDML, SphereNet performs better four and worse the other four molecules, which is similar to DimeNet. One reason is sGDML incorporates molecular symmetries to boost precision, and different molecules have different symmetries. However, sGDML. has poorer generalization
8

---

Published as a conference paper at ICLR 2022
power to larger datasets without hand-engineered features. In addition, SphereNet achieves much better overall std. MAE than sGDML. When using the same WoFE that 1000, SphereNet achieves similar results with GemNet in spite of that GemNet-T is of high complexity and contains carefully designed network components for performance boost.
6.5 COMPLETENESS VERSUS EFFICIENCY
Table 4: Comparisons bewtween SMP and Q-MP on MD17 using two backbone networks.
The message passing scheme Q-MP in GemNet represents the edge-based 2-hop geometric message passing and generate complete representations of 3D molecular graphs. We study the capability and efficiency of the pro- posed SMP by comparing with Q-MP. Specifically, We the same back- bone network for these two MP meth- ods for fair comparisons. We exten- sively use two backbones, which are the SphereNet backbone introduced in Sec. 4 and the GemNet backhone pro- posed in Kliepera et al. (2021). We conduct experiments on MD17, re-
SphereNet Backbone
GemNet Backbone
Molecule
SMP
Q-MP
SMP
Q-MP
Aspirin
0.209
0.247
0.225
0.231
Benzene
0.147
0.153
0.144
0.149
Ethanol
0.091
0.102
0.089
0.083
Malonaldehyde
0.172
0.168
0.169
0.176
Naphthalene
0.048
0,057
0.063
0,062
Salicylic acid
0.113
0,125
0.111
0.114
Toluene
0.054
0.043
0.052
0.063
Uracil
0.106
0.106
0.098
0.113
Time/Epoch (s)
324
1270
295
1185
porting performance and average running time for all the molecules per epoch using the same computing infrastructure (Nvidia GeForce RTX 2080 TI IGB)) Results are shown in Table 4, from which we can observe that on either backbone network, SMP achieves very similar results with Q-MP. However, the time cost is much less than SMP, which indicates it is much more efficient than Q-MP. Based on analyses in Sec. 3.2, SMP can distinguish almost all molecular structures, and the failure cases may not exist in nature. Hence, SMP performs similarly with Q-MP even though the latter is complete theoretically but not scalable in practice. We further compare the efficiency between SphereNet and other models in terms of parameters and time cost in Appendix F. SphereNet uses similar computing budget with others but achieves the best performance.
6.6 ABLATION STUDY
Table 5: Comparisons among three message passing strate- gies on the same SphereNet architecture on the partial MD17 dataset.
The proposed SMP considers all the distance, angle, and torsion, leading to more powerful data representations. We investigate contributions of differ- ent geometries to demonstrate the ad- vances of SMP. We remove tor- sion information from SMP which we Malonaldehyde
Molecule Ethanol
SMP w/o (8, (p)
SMP w/o y
SMP
0.249
0..22
0,208
0.550
0.360
0.340
Naphthalene
0.372
0,205
0.178
denote "SMP w/o =p"; we further remove angle information which we
Toluene
0.446
0.182
0.155
denote as "SMP w/o (6, 4p)". The three message passing strategies are integrated to the same architec- ture with other network parts remaining the same. We evaluate these models on four molecules of MD17. Table 5 shows that SMP outperforms SMP w/o y,, and SMP w/o & outperforms "SMP w/o (0, "p)" These results demonstrate the effectiveness of angle and torsion information used in the SMP. The best performance of SMP further reveals that SMP represents an accurate scheme for 3D graphs. In addition, provide visualization results for SphereNet filters in Appendix G to further show that all the distance, angle, and torsion information determine the structural semantics of filters.
7 CONCLUSIONS
3D information is important for molecules but there lacks a principled message passing framework to consider it. We first propose the spherical message passing unifying and efficient scheme that can achieve approximately complete representations molecules without increasing computing budget. Based SMP and meaningful physical representations, SphereNet is presented, and experiments on various types of datasets demonstrates its capability, efficiency, and scalibility.

---

Published as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
Detailed experimental setup is provided in Appendix D. Implementation hyper-parameters of SphereNet on all the three datasets OC20, QM9, and MD17 are given in Table 6, Table 7, and Table 8, respectively. Code is integrated in the DIG library (Liu et al., 2021) and available at https: /ttp://ggtttb/ com/ com/divelab/DI..
ACKNOWLEDGMENTS
This work was supported in part by National Science Foundation grant 19S-190811 198 and National Institutes of Health grant IR2INS102828, We thank Hannes Stärk for his valuable suggestions and discussions when developing the methods.
REFERENCES
Brandon Anderson, Truong-Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks. In Proceedings of the 33st International Conference on Neural Information Processing Systems, pp. 14537-14546, 2019.
Simon Axelrod and Rafael momez-Bombarrlli.. Geom: Energy- molecular conformations for property prediction and molecular generation. arXiv preprint arXiv:2006.05531, 2020.
Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint 1rXiv:1806.01261 2018.
Simon Batzner, Tess E Smidt, Lixin Sun, Jonathan Mailoa, Mordechai Kornbluth, Nicola Molinari, and Boris Kozinsky. Se (3)-eurvvr graph neural networks for data-efficient and accurate interatomic potentials. arXiv preprint arXiv:2101 100-2101003/666 2021.
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13475, 2021.
Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Morgane Riviere, Kevin Tran, Javier Heras-Domingo, Caleb Ho, Weihua Hu, et al. The open catalyst 2020 (oc20) dataset and community challenges. arXiv preprint arXiv:2010.0999 0.09900, 2020.
Chao Chen, Guanbin Li, Ruijia Xu, Tianshui Chen, Meng Wang, and Liang Lin. Clusternet: Deep hierarchical cluster network with rigorously rotation-invariant representation for point cloud analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4994-5002, 2019.
Stefan Chmiela, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof Schütt, and Klaus-Robert Müller. Machine learning of accurate energy-conserving molecular force fields. Science advances, 1653e15 2017.
Stefan Chmicla, Huziel E Sauceda, Mlaus-Robert Müller, and Alexandre Tkatchenko. Towards exact molecular dynamics simulations with machine-learned foree fields. Nature communications, 9(1): 1-10, 2018.
Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling, Gauge equivariant convolutional networks and the icosahedral enn. In International Conference on Machine Learning, pp. 1321- 1330, 2019.
Pim De Haan, Maurice Weiler, Taco Cohen, and Max Welling, Gauge equivariant mesh cnns: Anisotropic convolutions on geometric graphs. In International Conference Learning Represen- tations. 2020.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks graphs with fast localized spectral filtering. Advances in neural information processing systems, 29:3844-388 3852, 2016.
10

---

Published as a conference paper at ICLR 2022
Alex Fout, Jonathon Byrd, Basir Shariat, and Asa Bun-Hur. Protein interface prediction using graph convolutional networks. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pp.
2017.
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. Se (3)ttransformmrs:: 3d roto- translation equivariant attention networks. Advances in Neural Information Processing Systems, 33, 2020.
Hongyang Gao and Shuiwang Ji. Graph U-nets. In Proceedings of 36th International Conference on Machine Learning, pp. 1083-2092 2019.
Hongyang Gao, Zhengyang Wang, and Shuiwang Ji. Large-scale learnable graph convolutional networks. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1416-1424, 2018.
Hongyang Gao, Yi Liu, and Shuiwang Ji. Topology-aware graph pooling networks. arXiv preprint arXiv:2010.09834, 2020.
Vikas Garg, Stefamie Jegelka, and Tommi Jaakkola. Generalization and representational limits of graph neural networks. In International Conference on Machine Learning. pp. 3419-3430, 2020.
Justin Gilmer, Samuel Schoenholz, Patrick Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning- Searning-Volume 70, pp. 1263-1272. JMLR, org, 2017.
Jonathan Godwin, Michael Schaarschmidt, Alexander L Gaunt, Alvaro
Yulia Rubanova, Petar Veličković, James Kirkpatrick, and Peter Battaglia. Simple gnn regularisation for 3d molecular property prediction and beyond. In International Conference on Learning Representations, 2022.
Marco Gori, Gabriele Monfardini, and Franco Scarselli. A new model for learning in graph domains. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., volume 2, pp. 729-734. IEEE, 2005.
David Griffiths and Darrell Schroeter. Introduction to quantum mechanics. Cambridge University Press, 2018.
Yulan Guo, Hanyun Wang. Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learning for 3d point clouds: A survey. IEEE transactions on pattern analysis and machine intelligence, 2020.
Weihua Hu, Muhammed Shuaibi, Abhishek Das, Siddharth Goyal, Anuroop Sriram, Jure Leskovee, Devi Parikh, and C Lawrence Zitnick. Forcenet: A graph neural network for large-scale quantum calculations. arXiv preprint 1xXi::2103.01436 2021.
John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph- based protein design. In Advances in Neural Information Processing Systems, pp. 15794-15805, 2019.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980 69800 2014.
Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
Johannes Klicpera, Shankari Giri, Johannes T. Margraf, and Stephan Günnemann. Fast and uncertainty-aware directional message passing for non-equilibrium molecules. NewrIPS-W, 2020a.
Johannes Klicpera, Janek Groß, and Stephan Günnemann. Directional message passing for molecular graphs. In International Conference on Learning Representations (ICLR), 2020b.
Johannes Klicpera, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. arXiv preprint arXiv:2 102.20000000333 2021.
11

---

Published as a conference paper at ICLR 2022
Loic Landrieu and Mohamed Boussaha. Point cloud oversegmentation with graph struh-structurd deep metric learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7440-7449, 2019.
Loic Landrieu and Martin Simonovsky. Large-scale point cloud semantic segmentation with super- point graphs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1558-45611 2018.
Meng Liu, Youzhi Luo, Limei Wang. Yaochen Xie, Hao Yuan, Shurui Gui, Haiyang Yu, Zhao Xu, Jingtun Zhang, Yi Liu, et al. Dig: a turnkey library for diving into graph deep learning research. Journal of Machine Learning Research, 22(240):1 2021.
Shengchao Liu, Mehmet F Demirel, and Yingyu Liang. N-gram graph: Simple unsupervised repre- sentation for graphs, with applications to molecules. Advances in Neural Information Processing Systems, 32:8464 8476, 2019.
Yi Liu, Hao Yuan, Lei Cai, and Shuiwang Ji. Deep learning high-order interactions for protein interface prediction. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 679-687, 2020.
Chengqiang Lu, Qi Liu, Chao Wang, Zhenya Huang. Peize Lin, and Lixin He. Molecular property prediction: A multilevel quantum interactions modeling perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp- 1052-1060, 2019.
Haggai Maron, Heli Ben-Hamu, Hadar Serviansky, and Yaron Lipman. Provably powerful graph networks. Advances in Neural Information Processing Systems, 32:2153-2164, 2019.
Nathanaël Perraudin, Michaël Defferrard, Tomasz Kacprzak, and Raphael Sgier. Deepsphere: Effi- cient spherical convolutional neural network with healpix sampling for cosmological applications. Astronomy and Computing. 27:130-146. 2019.
Zhuoran Qiao, Matthew Welborn, Animashree Anandkumar, Frederick Manby, and Thomas Miller III. Orbnet: Deep learning for quantum chemistry using symmetry-adaptee atomic-orbital features. The Journal of Chemical Physics, 153(12):12411,, 2020.
Raghunathan Ramakrishnan, Pavlo O Dral, Mathhis Rupp, and Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1-7, 2014.
Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph networks. In International Conference Machine Learning, pp. 8459 8468. 2020.
Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Blaus-Robert Müller. Schnet: A continuous-filter convolutional neural network for modeling quantum interactions. In Advances in neural information processing systems, pp. 991-1001, 2017.
Kristof T Schlitt, Oliver T Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. arXiv preprint arXiv:2102.03150. 2021.
Yiru Shen, Chen Feng, Yaoqing Yang, and Dong Tian. Mining point cloud local structures by kernel correlation and graph pooling. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4548 4557, 2018
Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(9), 2011.
Gregor Simm, Robert Pinsler, and José Miguel Hernández-Lobbto. Reinforcement learning for molecular design guided by quantum mechanics. In International Conference on Machine Learning. pp. 8959-8969, 2020.
Martin Simonovsky and Nikos Komodakis. Dynamic edge- conditioned filters in convolutional neural networks on graphs. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3693-3702, 2017.
12

---

Published as a conference paper at ICLR 2022
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation- and consslation-quivaaiii neural networks for 3d point clouds. arXiv preprint wwx.vv:802.082222 2018.
Raphael Townshend, Rishi Bedi, Patricia Suriana, and Ron Dror. End-to-end learning on 3d protein Structure for interface prediction. Advances in Neural Information Processing Systems, 32: 15642- 15651, 2019.
Oliver T Unke and Markus Meuwly, Physnet: A neural network for predicting energies, forces, dipole moments, and partial charges. Jourmal of chemical theory and computation, 1566):377833633 2019.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In International Conference Learning Representations, 2018.
Clement Vignac, Andreas Loukas, and Pascal Frossard. Building powerful and equivariant graph neural networks with message-pant paaaagg....sing..... arXiv preprint arXiv:2006. 15107, 2020.
Lei Wang, Yuchun Huang. Yaolin Hou, Shenman Zhang, and Jie Shan. Graph attention convolution for point cloud semantic segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 10296-10305, 2019a.
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph enn for learning point clouds. Acm Transactions On Graphics (log), 3815):1-12, 2019b.
Zhengyang Wang, Meng Liu, Youzhi Luo, Zhao Xu, Yaochen Xie, Limei Wang, Lei Cai, and Shuiwang Ji. Advanced graph and sequence neural networks for molecular property prediction and drug discovery. arXiv preprint arXiv:2012.01981, 2020.
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learning. Chemical science, 9(2):513-55)) 2018.
Tian Xie and Jeffrey C Grossman. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Physical review letters, 120(14):15301) 2018.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? International Conference on Learning Representations, 2019.
Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. An end-to-end deep learning archi- tecture for graph classification. In Proceedings of the AAAI Conference Artificial Intelligence, volume 32, 2018
13

---

Published as a conference paper at ICLR 2022
Spherical Message Passing for 3D Molecular Graphs: Appendix
Messsagga E
C2
LB2
Inpul Block
Distance Vid)
LB2
Residial
Output
Dirtance Wild)
Vid.0.p 0,
Residual
o|LBI
Σ
LB2
Spherical Message Passing
Figure 4: Architecture of SphereNet. LB2 denotes linear block with two linear layers, o(LB) denotes a linear layer followed by an activation function, || denotes concatenation, and O denotes element-wise multiplication. Each L.2 aims at canceling bottlenecks by performing downprojection, followed by upprojection. Hence, is related to three hyperparameters: these are, input embedding size, intermediate size, and output embedding size. Each linear block LB is related hyperparameters including input embedding size and output embedding size. Description of each block is in Sec. B.
A UPDATE FUNCTIONS IN SMP
The function of is applied to each edge k and outputs the updated edge vector e/, The indices of the input geometries to 6" are illustrated in Fig. (a). Correspondingly, the inputs include the old edge vector сą. the receiver node vector Vra. the sender node vector V... the set of edges Es, that point to the node tion information for the nodes connected by the edge k and edges in Esg with the index set as Fk U Sy U. The function p² aggregates 3D information from these nodes update the edge k. The function d used for per-node update and generates the new node vector v for each node i. An illustration of the indices of the inputs 6² is provided in Fig. 5 (b). The inputs include the old node vector v,, the of edges E that point to the node i, and 3D information for all the related nodes (the index set is W,). The functions pa - is applied aggregate the input edge features for updating the node i. The function 6 is used to update the global graph feature, while the function - aggregates information from all the edge features.
E
and the 3D posi-
Nou
M
(a)
(b)
Figure 5: Illustrations of the functions df (a) and Φ" (b).
The three information update functions Φ² Φ" and Φ² can be implemented in different ways, such as using neural networks and mathematical operations. In SMP, the 3D information in is converted and incorporated to update each message e Hence, SMP uses - compute position representations for edges. Note that as absolute Cartesian coordinates stored P are not invariant to translation and rotation, they are not used immediate inputs machine learning models. The position aggregation function can be flexibly adapted to generate invariant representations. For example, p in Eq. (1) can be adapted spherical Bessel basis function.

---

Published as a conference paper at ICLR 2022
B INFORMATION UPDATE AND ARCHITECTURE OF SPHERENET
we assume the message e for the edge l the center message for update. The imput block generates the initial message for the edge k, and takes only the distance representation (dd as the input. Each interaction block updates the message for the edge k. The inputs include messages for all the neighboring edges, and all three representations, including (d,, 0, ()), (d, 0), and (cl) based on the edge k and its neighboring edges. The output block first takes both the distance representation and the current message for k as inputs. Then the feature vector of the receiver atom for the edge (atom 2%K in Fig. 2 and Eq. (1)) is obtained by aggregating all the messages pointing to it, where other messages have a similar update process as c².
Detailed architecture of SphereNet is provided in Fig. 4. Specifically. SphereNet is composed of an input block, followed by multiple interaction blocks and output blocks. For the purpose of simplicity, the architecture is explained by updating the receiver note Th of the message et. as described in Eq. (1) and Sec. main paper.
Input Block aims at constructing initial message eş for the edge k. Inputs include the distance representation Vi (d) for edge k, initial node embeddings VaL. Vr, for the sender node Sj, and the receiver node 7k- The distance information is encoded by using LB2 block.
Interaction Block updates the message e₁ with incorporating all the three physical representations. Input 3D information includes the distance embedding (d), the angle (dd, 0), and the torsion (d, 0, sp) The initial embedding sizes for them are I., N x L,, and N x L,, respectively. Other inputs are old message eş and the set of messages F.g, that point to the sender node sg- Similar to the input block, each type of 3D information is encoded by using a block LB2. Note that each @ indicates the element-wise multiplication between the corresponding 3D information represented as vector and each message in the set Exg- Thus, each neighboring message of C₁ is gated by the encoded 3D information. The Σ aggregates all the gated messages in Es, to a vector, which is added to the transformation of the old message eş as the updated message e/g- The transformation branch for old message e; is composed of several nonlinear layers and residual blocks, as shown Fig. 4. Output Block aggregates all the incoming messages to update the feature for node rk. Each incoming message has the same update process as e₃ by interaction blocks. For the purpose of clear illustration, use e, to represent each updated incoming message, which is further gated by the distance representation vector V (d).
C RELATIONS WITH PRIOR MESSAGE PASSING METHODS
Our SMP is specific architecture for 3D graphs and is formally defined in Eq. (1). Especially, the message passing schemes used in some existing models can be viewed as special cases of SMP as they only encode partial 3D information. In this section, we clearly give the used functions as well as their inputs for each existing method. Basically, we describe how each method realizes Eq. (1).
C.1 SCHNET, SCHÜTT ET AL. (2017)
In SchNet, the used aggregation function to encode 3D) position information pp-+e ((ri)errus,))
T ([]r,, Ts. rs,ll), Il), which converts the position information to an embedding of distance with radial basis functions. In addition to the p function, the &e function used is NN (NN (vrr) @ NN ( (Ilr>, Ts (]))), where NN denotes a neural network and @ denotes the element-wise multiplication. The -r function is [eirra,s e/_- The d² function is
v; + E(eh e-- The global feature u is updated based on the final node features yT and the
2

---

Published as a conference paper at ICLR 2022
function is 6² Ei=1:n NN (v?). Formally, the update process is expressed as
e = (Vres p (frn
=6e (Vras v (1|ro) -r)ll))
=NN (NN (Vr.) NN (v (1|rr2 - Tea 11))
V =6" (vi, (E()
=6"
Vi.
Σ
(charr JEE;
V
E/
(oL.TLIO))ER) =6 (p" (V*))
E NN (vT).
1=1m
C.2 PHYSNET, UNKE& & MEUWLY (2019)
PhysNet uses distance between atoms as an important feature and proposes more powerful neural net- works for chemical applications. The position aggregation function is p (fr, h=r)
V (lirr, Ts I)), where V is any radial basis function with a smooth cutoff. For the in- formation update functions,
of function is a (W1) (Von) & W2 (llr,, - -r, II, the ø function NN (W3 V, + NN (o (Wa) (v:) + (eh,ir..) ok and the 6" function is Ei=lo Ian NN (v"). Here NN denotes neural network, Wi, W2, W3, W4 are learnable weight matrices, a is an activate function, and @ denotes the element-wise multiplication. PhysNet is expressed
6 of WoEipPee ((ra)h=mwaa))
=o (Vexi V (1lTre - -rszll) Il))
(W1) (Voc) @ W2 (frrr - Frs 1I)
V -b" (vi, p (E()
=
Vi
Σ
ct
(ch.ru.na)ER
Σ e )
-NN WN
(v.)
u' =0" (p- (V'), u)
=u+ Σ NN (v/).
C.3 DIMENET, KLICPERA ET AL. (2020B)
DimeNet explicitly considers distances between atoms and directions of directed edges. The ag- gregation functions on the position information is pr (V (d) v (), where denotes concatenation, V (d) and Vi (d,0) O) are the same basis functions used in SphereNet as introduced in Sec. 4. Specifically, V (d) denotes the representation of the distance based spherical Bessel function, and V (d, 0) denotes the representation of distance and angle based on spherical Bessel function and spherical harmonics. For other functions, the df function used is e/, = (eh,1ll%%2) with
e NN (ex,1 + NN (oW (i,, + EE W2 (dj, 0zx) (W (dg) o.
3

---

Published as a conference paper at ICLR 2022
and 22,2 W. (dz) O 1/,1 where NN denotes neural network, W1, W2, W3. W4. W. different weight matrices, and is activation function. The pe-> function is Σ (or 1e. EE; 6/22
and the d² NN that pm pp : p= functions are not required in DimeNet. The whole model is expressed as
[ (Eee.ru. ek.2)
The pa is Ei=ion v and the 6²th is u+ Σx=1m V Note
o = (ex,11|ex.
0 (V (d) 1/ (d,
e6,1 = (ex, Bon po ((rdheEnvAuNN)))
W2V (Way (dz) (o)))o..e.))) )
-NN ei,1 NN oii
E
(o).)si8)EEBz
2,2 = V (dy) ① 1
v/ = (0 00 (E)))
=NN
Σ
6
(ch,irLoo)) EEE
u =Φ" (u, pu (V))
1=1:
(4)
D EXPERIMENTAL SETUP
For all the models used in three datasets, we set input embedding size = 256 and output embedding size 64 for both LB2 and LB blocks. For each separate model, we first perform warmup initial learning rate. Then two learning rate strategies, including ReduceLRRnnllataau and StepLR, are used for training. For StepLR, the learning rate decayed by the decay ratio every fixed epochs represented as step size. We do not use weight decay or dropout for all models. Some hyperparameters are fixed values, and some are tuned by grid search. Values/search space of hyperparameters for OC20, QM9, and MD17 are provided in Table 6, Table 7, and Table 8, respectively. Optimized hyperparameters are tuned on validation sets and applied to test sets for QM9 and MD17. For OC20, optimized hyperparameters are obtained on the Il) split within max epochs, and then applied to the other three splits. Pytorch used to implement all methods. For QM9 and MD17 datasets, all models are trained using one NVIDIA GeForce RTX 2080 Ti IIGB GPU, For the OC20 dataset, all models are trained using four NVIDIA RTX A6000 48GB GPUs.
E OC20 DATA DESCRIPTION
There exist three tasks including S2EF, IS2RS, and IS2RE. In this work, we focus on IS2RE that predicts structure's energy in the relaxed state. It the most common task in catalysis as relaxed energies usually influence the catalyst activity. The dataset for IS2RE is originally split into coaiiing/aaliattonns
sets. There are 460,318 structures in the training dataset total. The test label is not publicly available. Performance is evaluated on the validation set, which has four splits including In Domain (ID), Out Domain Adsorbates (OOD Ads), Out of Domain catalysts (OOD cat), and Out of Domain Adsorbates and catalysts (OOD Both), where numbers of structures are 24,943, 24,961, 24,963, 24.987, respectively. The average number of atoms per structure 77.75.
F EFFICIENCY STUDY OF SPHERENET
We study the efficiency of SphereNet by comparing with other models regarding number of parameters and time cost per epoch using the same computing infrastructure (Nvidia GeForce RTX 2080 TI
4

---

Published as a conference paper at ICLR 2022
Table 6: Values/search space for hyperparameters on OC20.
Hyperparameters
Values/search space
Interaction block distance LB2 intermediate size
Interaction block angle LB2 intermediate size Interaction block torsion LB2 intermediate size of interaction blocks # of RBFs N
3, 4
# of spherical harmonics L
3, 7
Cutoff distance
5, 6
Batch size
16, 32
Initial learning rate
le-4, 5e-4, 1e-3
Learning rate strategy
ReduceL.ROnPlateu, StepLR 0.4, 0.5, 0.6 4, 7, 10, 12, 2
Learning rate decay ratio (for StepLR) Learning rate milestones (for StepLR) Learning rate warmup epochs Learning rate warmup factor Max of Epochs
0.2
20
Table 7: Values/search space for hyperparameters on QM9.
Hyperparameters
Values/search space
Interaction block distance LB2 intermediate size
4, 8, 16
Interaction block angle LB2 intermediate size Interaction block torsion LB2 intermediate size # of interaction blocks # of RBFs N
4, 8, 16
4, 8, 16
3, 4,
# of spherical harmonics L
3,5,7 7
Cutoff distance
5,
Batch size
32, 64
Initial learning rate
lc-4, 5e-4, 1e-3
Learning rate strategy
StepLR 0.4, 0.5, 0.6
Learning rate decay ratio Learning rate step size
50, 100, 150
Max # of Epochs
500, 1000
1 1GBB). Experiments conducted the property Up of QM9 and results shown in Table 9. obvious that SphereNet uses similar computational resources as DimeNett+ and GemNet-T, and is much more efficient than DimeNet. The main reason could be we develop an efficient way to compute torsion, as introduced in Sec. 3 and Fig. 2 (b). Moreover, GemNer-Q cannot run on QM9 using the infrastructure as mentioned above.
5

---

Published as a conference paper at ICLR 2022
Table Values/search space for hyperparameters on MD17.
Hyperparameters
Values/search space
Interaction block distance LB2 intermediate size
4, 8, 16
Interaction block angle LB2 intermediate size
4, 8, 16
Interaction block torsion LB2 intermediate size # of interaction blocks # of RBFs N
4, 8, 16
2, 3, 4,
# of spherical harmonics L
3,5,7 7
Cutoff distance
5,
Batch size
1, 4, 16, 32
Initial learning rate
le-4, 5e-4, 1e-3
Learning rate strategy
StepLR 0.4, 0.5, 0.6
Learning rate decay ratio Learning rate step size
50, 100, 200
Max # of Epochs
500, 1000, 2000
Table 9: Efficiency comparisons between SphereNet and other models in terms of number of parameters and time cost per epoch using the same infrastructure.
SchNet
DimeNet
DimeNet++ GemNet-T
SphereNet
#Param. 185,153
2100,070
1887,110
2040,194
1898,566
Time (s)
100
840
240
290
340
G SPHERENET FILTER VISUALIZATION
We visualize SphereNet filters from learned SphereNet model. Specifically, we port learned weights for the block LB2 after the torsion embedding Vi 0,)p) in Fig. 4. For each location represented by tuple (d, 0, sp), the initial embedding size is N2 L. The computation for the above LB2
W, (W2 (ed, 0, +p)), which results in the new embedding size of 64 for each location (d, 0, :p). We then perform sampling on locations in 3D space for visualizing weights as SphereNet filters. The visualization results are provided in Fig. 6. We set sampling rate in the torsion direction to be π /4, thus, there are eight samples in the torsion direction. There are totally 64 elements for each location, and we randomly pick 6 elements. Apparently, among the distance, angle and torsion, considering any one when fixing the other two, the structural value of filters will be different when the one of interest changes. It essentially shows that all the distance, angle, and torsion information determine the structural semantics of filters. This further demonstrates that SMP enables the learning of different 3D information for improved representations.

---

Published as a conference paper at ICLR 2022
Figure 6: Visualization of six SphereNet filters. Each row corresponds to filter with torsion angles and 7 from left right.
π /4, 2/2, 3v/4, n. 5 3T/


Reviews:
[{"review_id": "PhveHZeImUs", "reviewer": "Reviewer_epnb", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The idea of incorporating torsion information when representing 3D molecules is novel and helpful. The proposed MP approach can better distinguish certain structures than some existing models. \n- The proposed model performs well on benchmark datasets, which validate the strong expressive power when representing 3D molecules.\n\nWeaknesses:\n- The proposed SMP scheme in Eq. (1) lacks novelty since it basically enriches the GN framework in [1] with geometry features.\n- I feel that the architecture of the proposed SphereNet is similar to DimeNet [2] except addressing torsion information. The importance and power to embed both distances and angles in 3D molecules have already been addressed by DimeNet. The ways of representing and embedding distances and angles are nearly the same in SphereNet and DimeNet. By comparing Figure 4 in this work and Figure 4 in DimeNet work [2], it's not hard to find the corresponding similarities in their architectures (each row below represents a correspondence):\n+---------------+----------------+\n\n|    DimeNet  | SphereNet    |\n\n+---------------+----------------+\n\n| Embedding | Input Block        |\n\n| Interaction | Interaction Block |\n\n| Residual    | Residual              |\n\n| Output      | Output Block         |\n\n+---------------+----------------+\n\n\n[1] Relational inductive biases, deep learning, and graph networks\n\n[2] Directional message passing for molecular graphs", "summary_of_the_paper": "This work aims at the representation learning of 3D molecular graphs and proposes a principled message passing framework to unify existing works. Besides, a novel message passing approach is presented by incorporating 3D geometric information including distance, angle, and torsion. In experiments, the performance of the proposed model is validated on three benchmark datasets.", "summary_of_the_review": "The main issue of the proposed model is the lack of novelty. Although introducing torsion is helpful when representing 3D molecules, the proposed model doesn't have novel architecture when embedding it as compared to existing models. The proposed framework also just follows the idea of existing work. Thus I think this work is below the acceptance threshold of ICLR.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "FUYV6hTEsSs", "reviewer": "Reviewer_xP3s", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nStrengths:\n\n(1) The SMP introduces an interesting method to alleviate the computation cost issue in SCS from O(nk^3) to O(nk^2). This method is important and can be generalized to more broad types of tasks.\n\n(2) The paper structure, i.e., from motivation to solution, is generally smooth. (Some potential points are listed below).\n\n(3) This is an empirical work, and the experimental results support the effectiveness of SMP.\n\n\n-----\nWeaknesses:\n\n(1) About motivation: Why SMP is better than Cartesian coordinate system (CCS) is not well explained. Authors give an example in Sec 2, but it is a little confusing for me.\n1. It says atomic connectivity is based on a cut-off distance and `considering all pairwise distances will lead to severe redundancies`, thus only local distance pairs are considered. But my understanding is the cut-off distance is merely a hyper-parameter, and it can include the H-H bond in the example in Fig 1.\n2. SMP alleviates the high computation cost issue when using SCS. However, its computational cost is still quite large, and is larger than the computational cost in CCS given the same cut-off threshold.\n3. From my point of view, SMP can better reflect the chirality property where models using pairwise distance cannot. This might be one point authors can try. \n\n(2) One question about the SMP model.\nSo SCS takes 3 atoms to define the reference plane. In SMP, it reduces it by introducing an interesting trick. However, I’m not quite sure how to pick the atom q_1 (in Figure b). Will the choice of q_1 have an effect on SMP? I didn’t find it in Sec 3.\n\n\n(3) Categories in related work.\nAbout the categories, I’m not sure why authors split the existing works into EGNN and relative information. From my understanding, models like SE(3)-Trans and EGNN are also using relative distance.\n\n\n\n-----\nMinor:\n\n(1) Typo in Intro: `... relative location of each atom in a 3D graph … `\n\n(2) In Intro, the authors say that `integrating the SMP and physical solutions to the Schrodinger equation`, can authors further explain what this means?\n", "summary_of_the_paper": "This paper provides a novel geometric GNN: SMP. SMP is based on the SCS, and can reduce the complexity by an order of magnitude. Besides, it is a unified framework that can cover some mainstream geometric GNN methods. The empirical results support the effectiveness of SMP.", "summary_of_the_review": "This paper is generally good, in terms of story-telling, method description, and technical novelty.\nThere are three main concerns: motivation/example, model description, and related work. This work can be further polished after fixing these points.\n\n-----\n\nAfter checking the replies from the authors and polished-up version of the paper, most concerns on the story have been addressed. I would like to raise the score to 8.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "ArCw5z1UhiJ", "reviewer": "Reviewer_6Y5X", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "As a premise, I am not an expert of molecular studies; in my review, I will focus on the technical and geometrical perspective, but I cannot assess the relevance of the method\\results for the biology community.\n\n==STRENGTHS==\n\nANALYSIS & REPLICABILITY\nThe experiments investigate different aspects of the method, providing insights on performance vs efficiency, ablation of used schema, and all the details of experimental setups and implementation. I think the paper can be replicable, and the claims are motivated.\n\nNOVELTY\nTo my best knowledge, the idea is novel. While message passing methods on graphs exploit only the connectivity, this work shows an interesting method to include the embedding information in the case of geometrical graphs. On this point, however, I would suggest a better contextualization. Geometrical graphs are well explained in this book [1], and other works propose methods to exploit such geometrical prior (e.g., [2]). I would consider this in the related work section.\n\n==WEAKNESSES==\n\nCOMPARISONS\nThe majority of the results are taken from other papers without re-running the experiments. I think this could be ok, but it is not completely clear if the split of training\\val\\test is preserved across the works (i.e., they are given by dataset) or if each method uses its own. I think it would be good to provide an average across multiple runs or reimplement at least the more recent competitors in this second case.\n\nAPPLICABILITY\nThe method explicitly addresses a specific domain, and I am not sure that it could be exported to other ones, e.g., meshes. How would it perform on geometrical graphs with thousands of nodes? The paper left such question for future works, but I think at least a computational timing on different domains would be useful to understand which kind of interest\\impact it could receive in ICLR audience. I am not from the biological field, and I do not get the dimensions of the considered graphs (but being molecules, I assume that the order of magnitude is tens of nodes, is it correct?)\n\n[1]: Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges; Bronstein et al., 2021\n[2]: Gauge Equivariant Mesh CNNs Anisotropic convolutions on geometric graphs; Haan et al., 2020\n", "summary_of_the_paper": "The paper proposes a message passing scheme using spherical coordinates. It is tested on three datasets of 3D moleclular graphs. The paper offer an in depth analysis of different aspects, with an extensive experimentation of the method.", "summary_of_the_review": "While I am not sure that the method can be applied in other contexts, I think the idea is valuable, and the experiments are convincing, and I am prone to accept the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper considers representation learning of 3D molecular graphs.
The authors propose a message passing scheme using spherical coordinates. It is
tested on three datasets of 3D moleclular graphs. The authors offer an in depth
analysis of different aspects, with an extensive experimentation of the method.

Strengths:

- The SMP introduces an interesting method to alleviate the computation cost issue in SCS from O(nk^3) to O(nk^2). This method is important and can be generalized to more broad types of tasks.
- This is an empirical work, and the experimental results support the effectiveness of SMP.
- The proposed MP approach can better distinguish certain structures than some existing models.
- Incorporating torsion information when representing 3D molecules is novel and helpful
- While message passing methods on graphs exploit only the connectivity, this work shows an interesting method to include the embedding information in the case of geometrical graphs.

Weaknesses:

- The proposed SMP scheme in Eq. (1) lacks novelty since it basically enriches the GN framework in [1] with geometry features
- the architecture of the proposed SphereNet is similar to DimeNet
- Why SMP is better than Cartesian coordinate system (CCS) is not well explained.

Overall, a majority of reviewers are in favor of acceptance and a third reviewer is happy with either acceptance or rejection and does not give strong reasons for rejecting the paper. My recommendation is, therefore, acceptance. I recommend the authors use the reviewers comments to improve the paper for its camera-ready version.
Decision: Accept (Poster)