Original: S PHERICAL M ESSAGE P ASSING FOR 3D M OLECULAR G RAPHS
Revised: S PHERICAL M ESSAGE P ASSING FOR 3D M OLECULAR G RAPHS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We consider representation learning of 3D molecular graphs in which each atom is associated with a spatial position in 3D. This is an under-explored area of research, and a principled message passing framework is currently lacking.
Revised: We consider representation learning of 3D molecular graphs in which each atom is associated with a spatial position in 3D. This is an under-explored area of research, and a principled message passing framework is currently lacking.

Original: In this work, we conduct analyses in the spherical coordinate system (SCS) for the complete identification of 3D graph structures.
Revised: In this work, we conduct analyses in the spherical coordinate system (SCS) for the complete identification of 3D graph structures.

Original: Based on such observations, we propose the spherical message passing (SMP) as a novel and powerful scheme for 3D molecular learning.
Revised: Based on such observations, we propose the spherical message passing (SMP) as a novel and powerful scheme for 3D molecular learning.

Original: SMP dramatically reduces training complexity, enabling it to perform efficiently on large-scale molecules.
Revised: SMP dramatically reduces training complexity, enabling it to perform efficiently on large-scale molecules.

Original: In addition, SMP is capable of distinguishing almost all molecular structures, and the uncovered cases may not exist in practice.
Revised: In addition, SMP is capable of distinguishing almost all molecular structures, and the uncovered cases may not exist in practice.

Original: Based on meaningful physically-based representations of 3D information, we further propose the SphereNet for 3D molecular learning.
Revised: Based on meaningful physically-based representations of 3D information, we further propose the SphereNet for 3D molecular learning.

Original: Experimental results demonstrate that the use of meaningful 3D information in
Revised: Experimental results demonstrate that the use of meaningful 3D information in

Original: SphereNet leads to significant performance improvements in prediction tasks.
Revised: SphereNet leads to significant performance improvements in prediction tasks.

Original: Our results also demonstrate the advantages of SphereNet in terms of capability, efficiency, and scalability.
Revised: Our results also demonstrate the advantages of SphereNet in terms of capability, efficiency, and scalability.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: In many real-world studies, structured objects such as molecules are naturally modeled as graphs (Gori et al., 2005; Wu et al., 2018; Shervashidze et al., 2011; Fout et al., 2017;
Revised: In many real-world studies, structured objects such as molecules are naturally modeled as graphs (Gori et al., 2005; Wu et al., 2018; Shervashidze et al., 2011; Fout et al., 2017;

Original: Liu et al., 2020; Wang et al., 2020).
Revised: Liu et al., 2020; Wang et al., 2020).

Original: With the advances of deep learning, graph neural networks (GNNs) have been developed for learning from graph data (Kipf & Welling, 2017; Defferrard et al., 2016; Veliˇckovi´c et al., 2018;
Revised: With the advances of deep learning, graph neural networks (GNNs) have been developed for learning from graph data (Kipf & Welling, 2017; Defferrard et al., 2016; Veliˇckovi´c et al., 2018;

Original: Zhang et al., 2018; Xu et al., 2019; Gao & Ji, 2019; Gao et al., 2018; 2020).
Revised: Zhang et al., 2018; Xu et al., 2019; Gao & Ji, 2019; Gao et al., 2018; 2020).

Original: Currently, the message passing scheme (Gilmer et al., 2017; Sanchez-Gonzalez et al., 2020; Vignac et al., 2020; Battaglia et al., 2018) is one of the commonly used architectures for realizing GNNs.
Revised: Currently, the message passing scheme (Gilmer et al., 2017; Sanchez-Gonzalez et al., 2020; Vignac et al., 2020; Battaglia et al., 2018) is one of the commonly used architectures for realizing GNNs.

Original: In this work, we aim at developing a novel message passing method for 3D graphs.
Revised: In this work, we aim at developing a novel message passing method for 3D graphs.

Original: Generally, a 3D molecular graph contains 3D coordinates for each atom given in the Cartesian system along with the graph structure (Liu et al., 2019; Townshend et al., 2019; Axelrod & Gomez-Bombarelli, 2020).
Revised: Generally, a 3D molecular graph contains 3D coordinates for each atom given in the Cartesian system along with the graph structure (Liu et al., 2019; Townshend et al., 2019; Axelrod & Gomez-Bombarelli, 2020).

Original: Different types of relative 3D information can be derived from 3D molecular graphs, and they can be important in molecular learning, such as bond lengths, angles between bonds (Schütt et al., 2017; Klicpera et al., 2020b).
Revised: Different types of relative 3D information can be derived from 3D molecular graphs, and they can be important in molecular learning, such as bond lengths, angles between bonds (Schütt et al., 2017; Klicpera et al., 2020b).

Original: We first investigate complete representations of 3D molecules.
Revised: We first investigate complete representations of 3D molecules.

Original: This requires the graph structure to be uniquely defined by relative 3D information.
Revised: This requires the graph structure to be uniquely defined by relative 3D information.

Original: To this end, we conduct formal analyses in the spherical coordinate system (SCS), and show that relative location of each atom in a 3D graph is uniquely determined by three geometries, including distance, angle, and torsion.
Revised: To this end, we conduct formal analyses in the spherical coordinate system (SCS), and show that relative location of each atom in a 3D graph is uniquely determined by three geometries, including distance, angle, and torsion.

Original: However, such completeness needs to involve edge-based 2-hop information, leading to excessively high computational complexity.
Revised: However, such completeness needs to involve edge-based 2-hop information, leading to excessively high computational complexity.

Original: To circumvent the computational cost, we propose a novel message passing scheme, known as the spherical message passing (SMP), for fast and accurate 3D molecular learning.
Revised: To circumvent the computational cost, we propose a novel message passing scheme, known as the spherical message passing (SMP), for fast and accurate 3D molecular learning.

Original: Our SMP is efficient and approximately complete in representing 3D molecules.
Revised: Our SMP is efficient and approximately complete in representing 3D molecules.

Original: First, we design a novel strategy to compute torsion, which only considers edge-based 1-hop information, thus substantially reducing training complexity.
Revised: First, we design a novel strategy to compute torsion, which only considers edge-based 1-hop information, thus substantially reducing training complexity.

Original: This enables the generalization of SMP to large-scale molecules.
Revised: This enables the generalization of SMP to large-scale molecules.

Original: In addition, we show that our SMP can distinguish almost all 3D graph structures.
Revised: In addition, we show that our SMP can distinguish almost all 3D graph structures.

Original: The uncovered cases are clearly stated and they seem rarely appear in practice.
Revised: The uncovered cases are clearly stated and they seem rarely appear in practice.

Original: By naturally using relative 3D information and a novel torsion, SMP yields predictions that are invariant to translation and rotation of input graphs.
Revised: By naturally using relative 3D information and a novel torsion, SMP yields predictions that are invariant to translation and rotation of input graphs.

Original: We apply the SMP to real-world molecular learning, where meaningful physical representations are needed.
Revised: We apply the SMP to real-world molecular learning, where meaningful physical representations are needed.

Original: Geometries ( d, θ, φ ) specified by SMP are then physically represented by Ψ( d, θ, φ ) , which can be a solution to the Schrödinger equation, as described in Sec. 4.
Revised: Geometries ( d, θ, φ ) specified by SMP are then physically represented by Ψ( d, θ, φ ) , which can be a solution to the Schrödinger equation, as described in Sec. 4.

Original: Based on this, we develop the spherical message passing neural networks, known as the SphereNet, for 3D molecular learning.
Revised: Based on this, we develop the spherical message passing neural networks, known as the SphereNet, for 3D molecular learning.

Original: We conduct experiments on various types of datasets including OC20, QM9, and MD17.
Revised: We conduct experiments on various types of datasets including OC20, QM9, and MD17.

Original: Results show that, compared with baseline methods, SphereNet achieves the best performance without increasing the computing budget.
Revised: Results show that, compared with baseline methods, SphereNet achieves the best performance without increasing the computing budget.

Original: Ablation study reveals contributions and necessity of different types of 3D information, including distance, angle, and torsion.
Revised: Ablation study reveals contributions and necessity of different types of 3D information, including distance, angle, and torsion.

Original: Particularly, we compare with a complete message passing scheme that can distinguish all 3D graph structures but involves edge-based 2-hop information.
Revised: Particularly, we compare with a complete message passing scheme that can distinguish all 3D graph structures but involves edge-based 2-hop information.

Original: Experimental results show that SphereNet achieves comparable performance but reduces running time by 4 times.
Revised: Experimental results show that SphereNet achieves comparable performance but reduces running time by 4 times.

Original: This suggests the use of SphereNet in practice rather than the complete message passing scheme, whose computational complexity prevents its use on large molecules.
Revised: This suggests the use of SphereNet in practice rather than the complete message passing scheme, whose computational complexity prevents its use on large molecules.

Original:  C OMPLETE R EPRESENTATIONS OF M OLECULES
Revised:  C OMPLETE R EPRESENTATIONS OF M OLECULES

Original: Equivariant graph neural networks (EGNNs) represent one research area for 3D molecular graphs, as introduced in Sec. 5.1.
Revised: Equivariant graph neural networks (EGNNs) represent one research area for 3D molecular graphs, as introduced in Sec. 5.1.

Original: These methods usually take coordinates in the Cartesian coordinate system (CCS) for all atoms as the raw input.
Revised: These methods usually take coordinates in the Cartesian coordinate system (CCS) for all atoms as the raw input.

Original: Hence, all the network layers need to be carefully designed to be equivariant.
Revised: Hence, all the network layers need to be carefully designed to be equivariant.

Original: The computing of some equivariant components is expensive, like spherical harmonics and Clebsh-Gordan coefficients (Thomas et al., 2018; Fuchs et al., 2020).
Revised: The computing of some equivariant components is expensive, like spherical harmonics and Clebsh-Gordan coefficients (Thomas et al., 2018; Fuchs et al., 2020).

Original: In addition, the complicated
Revised: In addition, the complicated

Original: SE(3) group representations may not be necessary for molecular learning where final representations are generally required to be invariant.
Revised: SE(3) group representations may not be necessary for molecular learning where final representations are generally required to be invariant.

Original: In this work, we focus on the other category of methods that take relative position information purely as input to graph learning models.
Revised: In this work, we focus on the other category of methods that take relative position information purely as input to graph learning models.

Original: Relative 3D information could be distance or angle, which is inherently invariant to translation and rotation of input molecules.
Revised: Relative 3D information could be distance or angle, which is inherently invariant to translation and rotation of input molecules.

Original: It is natural to consider such information in the spherical coordinate system (SCS).
Revised: It is natural to consider such information in the spherical coordinate system (SCS).

Original: We start by investigating the structure identification of 3D molecules in the SCS.
Revised: We start by investigating the structure identification of 3D molecules in the SCS.

Original: For any point in the SCS, its location is specified by a 3-tuple ( d, θ, φ ) , where d , θ , and φ denote the radial distance, polar angle, and the azimuthal angle, respectively.
Revised: For any point in the SCS, its location is specified by a 3-tuple ( d, θ, φ ) , where d , θ , and φ denote the radial distance, polar angle, and the azimuthal angle, respectively.

Original: When modeling 3D molecular graphs in the SCS, any atom i can be the origin of a local SCS, and d , θ , and φ naturally become the bond length, the angle between bonds, and the torsion angle, respectively.
Revised: When modeling 3D molecular graphs in the SCS, any atom i can be the origin of a local SCS, and d , θ , and φ naturally become the bond length, the angle between bonds, and the torsion angle, respectively.

Original: Thus, the relative location of each neighboring atom of atom i can be specified by the corresponding tuple ( d, θ, φ ) .
Revised: Thus, the relative location of each neighboring atom of atom i can be specified by the corresponding tuple ( d, θ, φ ) .

Original: Similarly, the relative location of each atom in the 3D molecular graph can be determined, leading to the identified structure, which is naturally invariant to translation and rotation of the input graph.
Revised: Similarly, the relative location of each atom in the 3D molecular graph can be determined, leading to the identified structure, which is naturally invariant to translation and rotation of the input graph.

Original: The SCS can be easily converted from the Cartesian coordinate system, thus, the tuple ( d, θ, φ ) can be easily obtained.
Revised: The SCS can be easily converted from the Cartesian coordinate system, thus, the tuple ( d, θ, φ ) can be easily obtained.

Original: O
Revised: O

Original: H H
Revised: H H

Original: O
Revised: O

Original: As in Fig.
Revised: As in Fig.

Original: 1, we use the chemical structure of the hydrogen peroxide (H 2 O 2 ) to show how d , θ , and φ are vital for the molecular structure identification.
Revised: 1, we use the chemical structure of the hydrogen peroxide (H 2 O 2 ) to show how d , θ , and φ are vital for the molecular structure identification.

Original: It is obvious that the structure is uniquely defined by the three bond lengths d 1 , d 2 , d 3 , the two bond angles θ 1 , θ 2 , and the torsion angle φ .
Revised: It is obvious that the structure is uniquely defined by the three bond lengths d 1 , d 2 , d 3 , the two bond angles θ 1 , θ 2 , and the torsion angle φ .

Original: Note that the input may not contain all pairwise distances (all possible bond lengths).
Revised: Note that the input may not contain all pairwise distances (all possible bond lengths).

Original: This is because the atomic connectivity is usually based on real chemical bonds and cut-off distances.
Revised: This is because the atomic connectivity is usually based on real chemical bonds and cut-off distances.

Original: The cut-off distance is usually set as a hyperparameter.
Revised: The cut-off distance is usually set as a hyperparameter.

Original: It is hard to guarantee that the cut-off is larger than any pairwise distance in a molecule.
Revised: It is hard to guarantee that the cut-off is larger than any pairwise distance in a molecule.

Original: Hence, in this example, H-H bond length may not be considered as input if the cut-off is small.
Revised: Hence, in this example, H-H bond length may not be considered as input if the cut-off is small.

Original: Setting a proper cut-off is even harder for other complicated and large molecules where a distance between two atoms could be large.
Revised: Setting a proper cut-off is even harder for other complicated and large molecules where a distance between two atoms could be large.

Original: In addition, considering all pairwise distances will cause severe redundancies, dramatically increasing the computational complexity.
Revised: In addition, considering all pairwise distances will cause severe redundancies, dramatically increasing the computational complexity.

Original: The model also easily gets confused by excessive noise, leading to unsatisfactory performance.
Revised: The model also easily gets confused by excessive noise, leading to unsatisfactory performance.

Original: From the perspective of completeness, using all pairwise distance is not capable of recognizing the chirality property.
Revised: From the perspective of completeness, using all pairwise distance is not capable of recognizing the chirality property.

Original: The chiral H 2 O 2 molecules cannot be distinguished even though the H-H bond is taken into account.
Revised: The chiral H 2 O 2 molecules cannot be distinguished even though the H-H bond is taken into account.

Original: To overcome the above challenges, we use a combination of distance, angle, and torsion for rigorous design and accurate learning.
Revised: To overcome the above challenges, we use a combination of distance, angle, and torsion for rigorous design and accurate learning.

Original: Apparently, the two O-H bonds can rotate around the O-O bond without changing any of the bond lengths and bond angles.
Revised: Apparently, the two O-H bonds can rotate around the O-O bond without changing any of the bond lengths and bond angles.

Original: In this situation, however, the torsion angle φ changes and the structure of the H 2 Ovaries accordingly.
Revised: In this situation, however, the torsion angle φ changes and the structure of the H 2 Ovaries accordingly.

Original: The importance of torsion angle has also been demonstrated in related research domains.
Revised: The importance of torsion angle has also been demonstrated in related research domains.

Original: Garg et al.
Revised: Garg et al.

Original: (2020) formally shows that the torsion along with the port numbering can improve the expressive power of GNNs in distinguishing geometric graph properties, such as girth and circumference, etc .
Revised: (2020) formally shows that the torsion along with the port numbering can improve the expressive power of GNNs in distinguishing geometric graph properties, such as girth and circumference, etc .

Original: Other studies (Ingraham et al., 2019; Simm et al., 2020) reveal that protein sequences and molecules can be accurately generated by considering the torsion in the given 3D
Revised: Other studies (Ingraham et al., 2019; Simm et al., 2020) reveal that protein sequences and molecules can be accurately generated by considering the torsion in the given 3D

Original: [Figure]
Revised: [Figure]

Original: structures.
Revised: structures.

Original: In this work, we propose SMP to systematically consider distance, angle, and torsion for approximately complete representation learning of 3D molecular graphs.
Revised: In this work, we propose SMP to systematically consider distance, angle, and torsion for approximately complete representation learning of 3D molecular graphs.

Original: Note that by using angle and torsion, SMP can easily recognize the chirality property.
Revised: Note that by using angle and torsion, SMP can easily recognize the chirality property.

Original:  S PHERICAL M ESSAGE P ASSING 3. M ESSAGE P ASSING S CHEME
Revised:  S PHERICAL M ESSAGE P ASSING 3. M ESSAGE P ASSING S CHEME

Original: Currently, the class of message passing neural networks (MPNNs)
Revised: Currently, the class of message passing neural networks (MPNNs)

Original: (Gilmer et al., 2017) are one of the most widely used architectures for GNNs.
Revised: (Gilmer et al., 2017) are one of the most widely used architectures for GNNs.

Original: Based upon the completeness analyses in Sec. 2, we propose to perform message passing in the spherical coordinate system (SCS), resulting in a novel and efficient scheme known as spherical message passing (SMP).
Revised: Based upon the completeness analyses in Sec. 2, we propose to perform message passing in the spherical coordinate system (SCS), resulting in a novel and efficient scheme known as spherical message passing (SMP).

Original: We show that message passing schemes used in existing methods, such as SchNet and DimeNet, are special cases of SMP.
Revised: We show that message passing schemes used in existing methods, such as SchNet and DimeNet, are special cases of SMP.

Original: We first formally define a 3D molecular graph, which is usually represented as a 4-tuple G = ( u , V, E, P ) .
Revised: We first formally define a 3D molecular graph, which is usually represented as a 4-tuple G = ( u , V, E, P ) .

Original: The u ∈ R d u is a global feature vector for the molecular graph G .
Revised: The u ∈ R d u is a global feature vector for the molecular graph G .

Original: V = { v i } i =1: n is the set of atom features, where each v i ∈
Revised: V = { v i } i =1: n is the set of atom features, where each v i ∈

Original: R
Revised: R

Original: d v is the feature vector for the atom i .
Revised: d v is the feature vector for the atom i .

Original: E
Revised: E

Original: = { ( e k , r k , s k ) } k
Revised: = { ( e k , r k , s k ) } k

Original: =1: m is the set of edges, where each e k ∈ R d e is the feature vector, r k is the index of the receiver atom, and s k is the index of the sender atom for the edge k .
Revised: =1: m is the set of edges, where each e k ∈ R d e is the feature vector, r k is the index of the receiver atom, and s k is the index of the sender atom for the edge k .

Original: P =
Revised: P =

Original: { r h } h =1: n is the set of 3D Cartesian coordinates that contains 3D spatial information for each atom.
Revised: { r h } h =1: n is the set of 3D Cartesian coordinates that contains 3D spatial information for each atom.

Original: In addition, we let E i =
Revised: In addition, we let E i =

Original: { ( e k , r k , s k ) } r k
Revised: { ( e k , r k , s k ) } r k

Original: = i,k =1: m denote the set of edges that point to the atom i , and N i denote the indices of incoming nodes of atom i .
Revised: = i,k =1: m denote the set of edges that point to the atom i , and N i denote the indices of incoming nodes of atom i .

Original: The outputs after a message passing process include the updated global feature vector u ′ ∈ R d u , the updated atom features V ′ =
Revised: The outputs after a message passing process include the updated global feature vector u ′ ∈ R d u , the updated atom features V ′ =

Original: { v ′ i } i =1: n , and the updated edges E
Revised: { v ′ i } i =1: n , and the updated edges E

Original: ′ =
Revised: ′ =

Original: { ( e ′ k , r k , s k ) } k
Revised: { ( e ′ k , r k , s k ) } k

Original: =1: m .
Revised: =1: m .

Original: An illustration of the message aggregation scheme for SMP is provided in Fig.
Revised: An illustration of the message aggregation scheme for SMP is provided in Fig.

Original: 2 (a).
Revised: 2 (a).

Original: Apparently, the embedding of the atom r k is obtained by aggregating each incoming message e k .
Revised: Apparently, the embedding of the atom r k is obtained by aggregating each incoming message e k .

Original: The message e k is updated based on E s k , the set of incoming messages pointing to the atom s k .
Revised: The message e k is updated based on E s k , the set of incoming messages pointing to the atom s k .

Original: Let q denote the sender atom of any message in E s k .
Revised: Let q denote the sender atom of any message in E s k .

Original: Hence, we can define a local SCS, where s k serves as the origin, and the direction of the message e k naturally serves as the z -axis.
Revised: Hence, we can define a local SCS, where s k serves as the origin, and the direction of the message e k naturally serves as the z -axis.

Original: We define a neighboring atom o of s k as the reference atom.
Revised: We define a neighboring atom o of s k as the reference atom.

Original: Thus, the reference plane is formed by three atoms s k , r k , and o .
Revised: Thus, the reference plane is formed by three atoms s k , r k , and o .

Original: For atom q , its location is uniquely defined by the tuple ( d, θ, φ ) , as shown in Fig.
Revised: For atom q , its location is uniquely defined by the tuple ( d, θ, φ ) , as shown in Fig.

Original: 4 (a).
Revised: 4 (a).

Original: Specifically, d determines its distance to the atom s k , θ specifies its direction to update the message e k .
Revised: Specifically, d determines its distance to the atom s k , θ specifies its direction to update the message e k .

Original: The torsion angle φ is formed by the defined reference plane and the plane spanned by s k , r k , and q .
Revised: The torsion angle φ is formed by the defined reference plane and the plane spanned by s k , r k , and q .

Original: Intuitively, as an advanced message passing architecture in spherical coordinates for 3D graphs, SMP specifies relative location for any neighboring atom q by considering all the distance, angle, and torsion information, leading to more comprehensive representations for 3D molecular graphs.
Revised: Intuitively, as an advanced message passing architecture in spherical coordinates for 3D graphs, SMP specifies relative location for any neighboring atom q by considering all the distance, angle, and torsion information, leading to more comprehensive representations for 3D molecular graphs.

Original: Generally, the atom s k may have several neighboring atoms, which we denote as q 1 , ..., q t .
Revised: Generally, the atom s k may have several neighboring atoms, which we denote as q 1 , ..., q t .

Original: It is easy to compute the corresponding bond lengths and bond angles for these t atoms.
Revised: It is easy to compute the corresponding bond lengths and bond angles for these t atoms.

Original: The SMP computes torsion angles by projecting all the t atoms to the plane that is perpendicular to e k and intersect with s k .
Revised: The SMP computes torsion angles by projecting all the t atoms to the plane that is perpendicular to e k and intersect with s k .

Original: Then on this plane, the torsion angles are formed in a predefined direction, such as the anticlockwise direction.
Revised: Then on this plane, the torsion angles are formed in a predefined direction, such as the anticlockwise direction.

Original: By doing this, any atom naturally becomes the reference atom for its next atom in the anticlockwise direction.
Revised: By doing this, any atom naturally becomes the reference atom for its next atom in the anticlockwise direction.

Original: Notably, the sum of these t torsion angles is 2 π .
Revised: Notably, the sum of these t torsion angles is 2 π .

Original: A simplified case is illustrated in Fig.
Revised: A simplified case is illustrated in Fig.

Original: 2 (b).
Revised: 2 (b).

Original: The atom s k has three neighboring atoms q 1 , q 2 , and q 3 ; q 3 is the reference atom for q 1 , and they form φ 1 ; q 1 is the reference atom for q 2 , and they form φ 2 ; similarly, q 2 is the reference atom for q 3 , and they form φ 3 .
Revised: The atom s k has three neighboring atoms q 1 , q 2 , and q 3 ; q 3 is the reference atom for q 1 , and they form φ 1 ; q 1 is the reference atom for q 2 , and they form φ 2 ; similarly, q 2 is the reference atom for q 3 , and they form φ 3 .

Original: It is obvious that the sum of φ 1 , φ 2 , and φ 3 is 2 π .
Revised: It is obvious that the sum of φ 1 , φ 2 , and φ 3 is 2 π .

Revised: As the torsion is defined relatively, q 1 can be picked arbitrarily, which will not affect the output of the message passing scheme, as we perform summation when aggregating information to s k from its neighbors q 1 , q 2 , and q 3 .

Original: Notably, by designing each atom to be the reference atom of the next one in the predefined direction, invariance is effectively achieved because reference atom is naturally relative.
Revised: Notably, by designing each atom to be the reference atom of the next one in the predefined direction, invariance is effectively achieved because reference atom is naturally relative.

Original: In addition, our method computes torsion within edge-based 1-hop neighborhood.
Revised: In addition, our method computes torsion within edge-based 1-hop neighborhood.

Original: Even though a torsion angle involves four atoms, our design avoids the number of torsion angles to be exponential, but makes it the same as the number of neighboring atoms.
Revised: Even though a torsion angle involves four atoms, our design avoids the number of torsion angles to be exponential, but makes it the same as the number of neighboring atoms.

Original: Hence, it is efficient and will not cause time or memory issues.
Revised: Hence, it is efficient and will not cause time or memory issues.

Original: Formally, the proposed SMP can be defined in the SCS as
Revised: Formally, the proposed SMP can be defined in the SCS as

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: where ϕ e , ϕ v , and ϕ u are three information update functions on edges, atoms, and the whole graph, respectively.
Revised: where ϕ e , ϕ v , and ϕ u are three information update functions on edges, atoms, and the whole graph, respectively.

Original: ρ e → v and ρ v → u aggregate information between different types of geometries.
Revised: ρ e → v and ρ v → u aggregate information between different types of geometries.

Original: Especially, in SMP, the 3D information in P is converted and incorporated to update each message e k .
Revised: Especially, in SMP, the 3D information in P is converted and incorporated to update each message e k .

Original: Hence,
Revised: Hence,

Original: SMP employs another position aggregation function ρ p → e for message update.
Revised: SMP employs another position aggregation function ρ p → e for message update.

Original: Notably, the main difference between our SMP scheme defined in Eq.
Revised: Notably, the main difference between our SMP scheme defined in Eq.

Original: 1 and the GN framework in Battaglia et al.
Revised: 1 and the GN framework in Battaglia et al.

Original: (2018) is the inclusion of 3D information P .
Revised: (2018) is the inclusion of 3D information P .

Original: In line with the research area described in Sec.
Revised: In line with the research area described in Sec.

Original: 5.1.2, we focus on such 3D information and develop a systematic solution to incorporate it completely and efficiently.
Revised: 5.1.2, we focus on such 3D information and develop a systematic solution to incorporate it completely and efficiently.

Original: Detailed description of these functions is given in Appendix A.
Revised: Detailed description of these functions is given in Appendix A

Revised: [Figure]

Original: 3. C OMPLETENESS V ERSUS E FFICIENCY
Revised: 3. C OMPLETENESS V ERSUS E FFICIENCY

Original: All the neighboring nodes of s k are projected to the plate perpendicular to the message of interest.
Revised: All the neighboring nodes of s k are projected to the plate perpendicular to the message of interest.

Original: We assume all the distances and angles are fixed (the molecules can be more easily distinguished otherwise).
Revised: We assume all the distances and angles are fixed (the molecules can be more easily distinguished otherwise).

Original: Hence, all the angle shown are torsion angles and they are formed in the anticlockwise direction.
Revised: Hence, all the angle shown are torsion angles and they are formed in the anticlockwise direction.

Original: (a) and (b) are chiral and SMP can distinguish them.
Revised: (a) and (b) are chiral and SMP can distinguish them.

Original: This is because in (a), q ′ 1 (90 ◦ ) , q ′ 2 (60 ◦ ) , q ′ 3 (120 ◦ ) , q ′ 4 (90 ◦ ) ; in (b), q ′ 1 (60 ◦ ) , q ′ 2 (120 ◦ ) , q ′ 3 (90 ◦ ) , q ′ 4 (90 ◦ ) .
Revised: This is because in (a), q ′ 1 (90 ◦ ) , q ′ 2 (60 ◦ ) , q ′ 3 (120 ◦ ) , q ′ 4 (90 ◦ ) ; in (b), q ′ 1 (60 ◦ ) , q ′ 2 (120 ◦ ) , q ′ 3 (90 ◦ ) , q ′ 4 (90 ◦ ) .

Original: SMP cannot distinguish (b) and (c) but this scenario may not exist in nature.
Revised: SMP cannot distinguish (b) and (c) but this scenario may not exist in nature.

Original: ∠ q ′ 1 s k q ′ 2 in (b) and ∠ q ′ 1 s k q ′ 3 in (c) usually are different as q ′ 2 and q ′ 3 are different atoms and the corresponding distances and angles are the same.
Revised: ∠ q ′ 1 s k q ′ 2 in (b) and ∠ q ′ 1 s k q ′ 3 in (c) usually are different as q ′ 2 and q ′ 3 are different atoms and the corresponding distances and angles are the same.

Original: The identification criteria described in Sec. 2 can fully determine the structure of a 3D molecule, but involves edge-based 2-hop information.
Revised: The identification criteria described in Sec. 2 can fully determine the structure of a 3D molecule, but involves edge-based 2-hop information.

Original: Hence, the computational complexity is as sizeable as O ( nk 3 ) , where n is the number of atoms, and k denotes the average number of neighboring atoms for each center atom.
Revised: Hence, the computational complexity is as sizeable as O ( nk 3 ) , where n is the number of atoms, and k denotes the average number of neighboring atoms for each center atom.

Original: Unfortunately, such design can hardly generalize to large molecular graphs.
Revised: Unfortunately, such design can hardly generalize to large molecular graphs.

Original: To this end, we propose SMP as an efficient and scalable scheme to realize message passing in SCS.
Revised: To this end, we propose SMP as an efficient and scalable scheme to realize message passing in SCS.

Original: Our SMP only involves edge-based 1hop information, thus the time complexity is reduced to O ( nk 2 ) .
Revised: Our SMP only involves edge-based 1hop information, thus the time complexity is reduced to O ( nk 2 ) .

Original: This enables the application of SMP to large molecules, like the newly released OC20 data (Chanussot et al., 2020).
Revised: This enables the application of SMP to large molecules, like the newly released OC20 data (Chanussot et al., 2020).

Original: We rigorously investigate the completeness of SMP and show that it can distinguish even complex geometric properties such as chirality, as indicated by Fig.
Revised: We rigorously investigate the completeness of SMP and show that it can distinguish even complex geometric properties such as chirality, as indicated by Fig.

Original: 3 (a) and Fig.
Revised: 3 (a) and Fig.

Original: 3 (b).
Revised: 3 (b).

Original: As SMP uses the last atom as the reference atom (like q 2 is the reference atom for q 3 in Fig.
Revised: As SMP uses the last atom as the reference atom (like q 2 is the reference atom for q 3 in Fig.

Original: 2 (b)) in a predefined direction, the relative order between adjacent atoms is considered while the absolute order is neglected.
Revised: 2 (b)) in a predefined direction, the relative order between adjacent atoms is considered while the absolute order is neglected.

Original: Hence, SMP can not distinguish the two molecules illustrated by Fig.
Revised: Hence, SMP can not distinguish the two molecules illustrated by Fig.

Original: 3 (b) and Fig.
Revised: 3 (b) and Fig.

Original: 3 (c).
Revised: 3 (c).

Original: However, this scenario may not exist in nature.
Revised: However, this scenario may not exist in nature.

Original: This is also demonstrated in experiments that our SMP achieves comparable performance with the complete representations, while the latter induces huge time complexity and severe memory issues.
Revised: This is also demonstrated in experiments that our SMP achieves comparable performance with the complete representations, while the latter induces huge time complexity and severe memory issues.

Original: 3. R ELATIONS WITH P RIOR M ESSAGE P ASSING M ETHODS
Revised: 3. R ELATIONS WITH P RIOR M ESSAGE P ASSING M ETHODS

Original: When developing message passing methods for 3D graphs, sphere message passing is an advanced scheme where the relative location of each atom is more specified.
Revised: When developing message passing methods for 3D graphs, sphere message passing is an advanced scheme where the relative location of each atom is more specified.

Original: The development for 3D graphs with relative information is still in early stage.
Revised: The development for 3D graphs with relative information is still in early stage.

Original: To our best knowledge, there exist several notable methods in literature, and they can be viewed as special cases of the SMP, as they capture partial 3D position information.
Revised: To our best knowledge, there exist several notable methods in literature, and they can be viewed as special cases of the SMP, as they capture partial 3D position information.

Original: For example, the SchNet and PhysNet consider distance, and the DimeNet encodes directional information.
Revised: For example, the SchNet and PhysNet consider distance, and the DimeNet encodes directional information.

Original: Formally, these methods can be perfectly fit the Spherical scheme defined in Eq.
Revised: Formally, these methods can be perfectly fit the Spherical scheme defined in Eq.

Original: We describe the details of these methods in Appendix C. Notably, compared with prior models, SMP provides a rigorous justification on its completeness with failure cases clearly described.
Revised: We describe the details of these methods in Appendix C. Notably, compared with prior models, SMP provides a rigorous justification on its completeness with failure cases clearly described.

Original: Importantly, SMP is developed based on the identification analyses of 3D molecular graphs.
Revised: Importantly, SMP is developed based on the identification analyses of 3D molecular graphs.

Original: Hence, it aims at learning complete data representations for 3D molecular graphs, rather than simply including extra 3D information (like angle or torsion).
Revised: Hence, it aims at learning complete data representations for 3D molecular graphs, rather than simply including extra 3D information (like angle or torsion).

Original:  S PHERE N ET
Revised:  S PHERE N ET

Original: The obtained 3-tuple ( d, θ, φ ) indicates the relative location of any atom in a 3D molecular graph.
Revised: The obtained 3-tuple ( d, θ, φ ) indicates the relative location of any atom in a 3D molecular graph.

Original: However, it cannot serve as the direct input to neural networks as it lacks meaningful representations.
Revised: However, it cannot serve as the direct input to neural networks as it lacks meaningful representations.

Original: Essentially, molecules are quantum systems thus the representation design needs to follow physics laws.
Revised: Essentially, molecules are quantum systems thus the representation design needs to follow physics laws.

Original: An important aspect is to choose appropriate basis functions that transform the 3-tuple ( d, θ, φ ) into physically-based representations.
Revised: An important aspect is to choose appropriate basis functions that transform the 3-tuple ( d, θ, φ ) into physically-based representations.

Original: Several basis functions have been explored in Hu et al.
Revised: Several basis functions have been explored in Hu et al.

Original: Klicpera et al.
Revised: Klicpera et al.

Original: (2020b), including MLP, Gaussian and sine functions, spherical Bessel basis, and spherical harmonics.
Revised: (2020b), including MLP, Gaussian and sine functions, spherical Bessel basis, and spherical harmonics.

Original: Especially, spherical Bessel is shown to be the best basis for encoding distance, and spherical harmonics is the most appropriate one for encoding angle (Hu et al., 2021; Klicpera et al., 2020b).
Revised: Especially, spherical Bessel is shown to be the best basis for encoding distance, and spherical harmonics is the most appropriate one for encoding angle (Hu et al., 2021; Klicpera et al., 2020b).

Original: We denote the final representation as Ψ( d, θ, φ ) .
Revised: We denote the final representation as Ψ( d, θ, φ ) .

Original: Referring to theories in Griffiths &
Revised: Referring to theories in Griffiths &

Original: Schroeter (2018); Cohen et al.
Revised: Schroeter (2018); Cohen et al.

Original: ; Klicpera et al.
Revised: ; Klicpera et al.

Original: (2020b), one form of the representation can be expressed as Ψ( d, θ, φ ) =
Revised: (2020b), one form of the representation can be expressed as Ψ( d, θ, φ ) =

Original: j ℓ (cid:16) β ℓn c d (cid:17) Y mℓ ( θ, φ ) , where j ℓ ( · ) is a spherical Bessel function of order ℓ , Y mℓ is a spherical harmonic function of degree m and order ℓ , c denotes the cutoff, β ℓn is the n -th root of the Bessel function of order ℓ .
Revised: j ℓ (cid:16) β ℓn c d (cid:17) Y mℓ ( θ, φ ) , where j ℓ ( · ) is a spherical Bessel function of order ℓ , Y mℓ is a spherical harmonic function of degree m and order ℓ , c denotes the cutoff, β ℓn is the n -th root of the Bessel function of order ℓ .

Original: We also have ℓ ∈ [0 , · · · , L − 1] , m ∈ [ − ℓ, · · · , ℓ ] and n ∈ [1 , · · · , N ] .
Revised: We also have ℓ ∈ [0 , · · · , L − 1] , m ∈ [ − ℓ, · · · , ℓ ] and n ∈ [1 , · · · , N ] .

Original: L and N denote the highest orders for the spherical harmonics and spherical Bessel functions, respectively.
Revised: L and N denote the highest orders for the spherical harmonics and spherical Bessel functions, respectively.

Original: They are hyperparameters in experimental settings.
Revised: They are hyperparameters in experimental settings.

Original: In addition, we can derive two simplified representations Ψ( d ) and Ψ( d, θ ) from Ψ( d, θ, φ ) .
Revised: In addition, we can derive two simplified representations Ψ( d ) and Ψ( d, θ ) from Ψ( d, θ, φ ) .

Original: Based upon the spherical message passing scheme described in Sec. 3 and physical representations, we build the SphereNet for molecular learning.
Revised: Based upon the spherical message passing scheme described in Sec. 3 and physical representations, we build the SphereNet for molecular learning.

Original: Apparently, SphereNet can produce data representations that are both accurate and physically meaningful.
Revised: Apparently, SphereNet can produce data representations that are both accurate and physically meaningful.

Original: By incorporating the position information in spherical coordinates, SphereNet also generates predictions invariant to translation and rotation of input molecules.
Revised: By incorporating the position information in spherical coordinates, SphereNet also generates predictions invariant to translation and rotation of input molecules.

Original: Following the architecture design in the research line stated in Sec.
Revised: Following the architecture design in the research line stated in Sec.

Original: 5.1.2, our network is composed of an input block, several interaction blocks, and an output block.
Revised: 5.1.2, our network is composed of an input block, several interaction blocks, and an output block.

Original: For clear description, we assume the message e k for the edge k in Fig.
Revised: For clear description, we assume the message e k for the edge k in Fig.

Original: 2 and Eq.
Revised: 2 and Eq.

Original: (1) is the message for update.
Revised: (1) is the message for update.

Original: The update process and detailed architecture for the SphereNet are provided in Appendix B.
Revised: The update process and detailed architecture for the SphereNet are provided in Appendix B.

Original:  R ELATED W ORK 5. M ETHODS FOR 3D M OLECULAR G RAPHS 5.1. E QUIVARIANT G RAPH N EURAL N ETWORKS
Revised:  R ELATED W ORK 5. M ETHODS FOR 3D M OLECULAR G RAPHS 5.1. E QUIVARIANT G RAPH N EURAL N ETWORKS

Original: One research line for 3D molecular graphs is equivariant graph neural networks (EGNNs) including tensor field networks (TFNs) (Thomas et al., 2018), SE(3)-transformers (Fuchs et al., 2020),
Revised: One research line for 3D molecular graphs is equivariant graph neural networks (EGNNs) including tensor field networks (TFNs) (Thomas et al., 2018), SE(3)-transformers (Fuchs et al., 2020),

Original: PaiNN (Schütt et al., 2021), NequIP (Batzner et al., 2021), etc.
Revised: PaiNN (Schütt et al., 2021), NequIP (Batzner et al., 2021), etc.

Original: The raw input of these methods usually contains the absolute information, such as coordinates in the Cartesian coordinate system.
Revised: The raw input of these methods usually contains the absolute information, such as coordinates in the Cartesian coordinate system.

Original: In intermediate layers, absolute information could be decomposed into partial absolute information and partial relative information as needed.
Revised: In intermediate layers, absolute information could be decomposed into partial absolute information and partial relative information as needed.

Original: A simple example is that a vector can be decomposed into its orientation (absolute) and length (relative) (Thomas et al., 2018; Schütt et al., 2021).
Revised: A simple example is that a vector can be decomposed into its orientation (absolute) and length (relative) (Thomas et al., 2018; Schütt et al., 2021).

Original: Apparently, network components of these methods should be carefully designed to be equivariant.
Revised: Apparently, network components of these methods should be carefully designed to be equivariant.

Original: The preliminary work like TFNs were developed for 3D point clouds.
Revised: The preliminary work like TFNs were developed for 3D point clouds.

Original: However, it is demonstrated that for molecules whose downstream tasks usually require the systems to be invariant, the complicated SE(3) group representations are not necessary but S 2 representations are enough (Klicpera et al., 2021).
Revised: However, it is demonstrated that for molecules whose downstream tasks usually require the systems to be invariant, the complicated SE(3) group representations are not necessary but S 2 representations are enough (Klicpera et al., 2021).

Original: Moreover, their performance on molecular tasks is not satisfactory.
Revised: Moreover, their performance on molecular tasks is not satisfactory.

Original: 5.1. I NVARIANT G RAPH N EURAL N ETWORKS
Revised: 5.1. I NVARIANT G RAPH N EURAL N ETWORKS

Original: Another category of methods purely take relative 3D information as input, such as distances between atoms, angles between bonds, angles between planes, etc.
Revised: Another category of methods purely take relative 3D information as input, such as distances between atoms, angles between bonds, angles between planes, etc.

Original: Hence, the network is naturally invariant.
Revised: Hence, the network is naturally invariant.

Original: The development of these methods is in early stage, and existing studies focus on leveraging different geometries.
Revised: The development of these methods is in early stage, and existing studies focus on leveraging different geometries.

Original: The SchNet (Schütt et al., 2017) incorporates the distance information during the information aggregation stage by using continuous-filter convolutional layers.
Revised: The SchNet (Schütt et al., 2017) incorporates the distance information during the information aggregation stage by using continuous-filter convolutional layers.

Original: The PhysNet (Unke &
Revised: The PhysNet (Unke &

Original: Meuwly, 2019) integrates both the atom features and distance information in the proposed interaction block.
Revised: Meuwly, 2019) integrates both the atom features and distance information in the proposed interaction block.

Original: The DimeNet (Klicpera et al., 2020b) is developed based on the PhysNet and moves a step forward by considering directional information in the interaction block.
Revised: The DimeNet (Klicpera et al., 2020b) is developed based on the PhysNet and moves a step forward by considering directional information in the interaction block.

Original: The GemNet (Klicpera et al., 2021) is proposed recently for universal molecular representations.
Revised: The GemNet (Klicpera et al., 2021) is proposed recently for universal molecular representations.

Original: The OrbNet (Qiao et al., 2020) combines distance information with the atomic orbital theory to design important SAAO features as inputs to GNNs.
Revised: The OrbNet (Qiao et al., 2020) combines distance information with the atomic orbital theory to design important SAAO features as inputs to GNNs.

Original: Generally, the use of 3D position information usually results in improved performance.
Revised: Generally, the use of 3D position information usually results in improved performance.

Original: However, existing methods simply include additional geometries such as distance and angle, and there lacks a rigorous justification on how different geometries contribute to the information aggregation process.
Revised: However, existing methods simply include additional geometries such as distance and angle, and there lacks a rigorous justification on how different geometries contribute to the information aggregation process.

Original: We conduct formal analysis and show that all the distance, angle, and torsion are necessary for 3D molecular identification, based on which we propose SphereNet to generate more powerful molecular representations.
Revised: We conduct formal analysis and show that all the distance, angle, and torsion are necessary for 3D molecular identification, based on which we propose SphereNet to generate more powerful molecular representations.

Original: 5. M ETHODS FOR O THER O BJECTS M ODELED AS G RAPHS
Revised: 5. M ETHODS FOR O THER O BJECTS M ODELED AS G RAPHS

Original: Besides molecules, many other data objects are also represented as graphs, such as 3D point clouds (Guo et al., 2020; Simonovsky & Komodakis, 2017;
Revised: Besides molecules, many other data objects are also represented as graphs, such as 3D point clouds (Guo et al., 2020; Simonovsky & Komodakis, 2017;

Original: Shen et al., 2018; Landrieu & Simonovsky, 2018) and meshes (Bronstein et al., 2021; De Haan et al., 2020; Perraudin et al., 2019).
Revised: Shen et al., 2018; Landrieu & Simonovsky, 2018) and meshes (Bronstein et al., 2021; De Haan et al., 2020; Perraudin et al., 2019).

Original: When modeling 3D point clouds as 3D graphs, points are represented as nodes and connections between points are directed edges.
Revised: When modeling 3D point clouds as 3D graphs, points are represented as nodes and connections between points are directed edges.

Original: Existing methods mainly capture distance information from local neighborhood in 3D space.
Revised: Existing methods mainly capture distance information from local neighborhood in 3D space.

Original: In DGCNN (Wang et al., 2019b), a novel layer namely EdgeConv is proposed to aggregate distance-based edges features for node learning.
Revised: In DGCNN (Wang et al., 2019b), a novel layer namely EdgeConv is proposed to aggregate distance-based edges features for node learning.

Original: In Landrieu & Boussaha (2019), neighborhood radius along with spatial orientation is incorporated in local point embedding.
Revised: In Landrieu & Boussaha (2019), neighborhood radius along with spatial orientation is incorporated in local point embedding.

Original: The work Wang et al. (2019a) proposes a graph attention convolution for 3D point clouds.
Revised: The work Wang et al. (2019a) proposes a graph attention convolution for 3D point clouds.

Original: Generally, these methods can be fit into our message passing framework defined in Eq.
Revised: Generally, these methods can be fit into our message passing scheme defined in Eq.

Original: The work De Haan et al.
Revised: The work De Haan et al.

Original: (2020) is an exemplary study that formulates meshes as graphs with considering geometrical information.
Revised: (2020) is an exemplary study that formulates meshes as graphs with considering geometrical information.

Original: The used convolutional kernel depends on the angle between a predefined reference edge and any other edge projected to the same plane for each vertex.
Revised: The used convolutional kernel depends on the angle between a predefined reference edge and any other edge projected to the tangent plane for each vertex.

Original: It focuses on the design of gauge equivariance rather than the learning of complete geometrical information.
Revised: It focuses on the design of gauge equivariance rather than the learning of complete geometry information.

Original: In this work, we study the complete learning of 3D molecules and leave extensive studies on other data types as future work.
Revised: In this work, we study the complete learning of 3D molecules and leave extensive studies on other data types as future work.

Original:  E XPERIMENTAL S TUDIES 6. E XPERIMENTAL S ETUP
Revised:  E XPERIMENTAL S TUDIES 6. E XPERIMENTAL S ETUP

Original: We apply our SphereNet to three benchmark datasets, including Open Catalyst 2020 (OC20) (Chanussot et al., 2020), QM9 (Ramakrishnan et al., 2014), and MD17 (Chmiela et al., 2017).
Revised: We apply our SphereNet to three benchmark datasets, including Open Catalyst 2020 (OC20) (Chanussot et al., 2020), QM9 (Ramakrishnan et al., 2014), and MD17 (Chmiela et al., 2017).

Original: Baseline methods include PPGN (Maron et al., 2019), SchNet (Schütt et al., 2017), PhysNet (Unke & Meuwly, 2019),
Revised: Baseline methods include PPGN (Maron et al., 2019), SchNet (Schütt et al., 2017), PhysNet (Unke & Meuwly, 2019),

Original: Cormorant (Anderson et al., 2019), PaiNN (Schütt et al., 2021), NequIP (Batzner et al., 2021),
Revised: Cormorant (Anderson et al., 2019), PaiNN (Schütt et al., 2021), NequIP (Batzner et al., 2021),

Original: MGCN (Lu et al., 2019), DimeNet (Klicpera et al., 2020b), DimeNet++ (Klicpera et al., 2020a),
Revised: MGCN (Lu et al., 2019), DimeNet (Klicpera et al., 2020b), DimeNet++ (Klicpera et al., 2020a),

Original: GemNet (Klicpera et al., 2021), CGCNN (Xie & Grossman, 2018), and sGDML (Chmiela et al., 2018).
Revised: GemNet (Klicpera et al., 2021), CGCNN (Xie & Grossman, 2018), and sGDML (Chmiela et al., 2018).

Original: Detailed configurations of all the models used in the following sections are provided in the supplementary material.
Revised: Detailed configurations of all the models used in the following sections are provided in the supplementary material.

Original: Unless otherwise specified, for all the baseline methods, we report the results taken from the referred papers or provided by the original authors.
Revised: Unless otherwise specified, for all the baseline methods, we report the results taken from the referred papers or provided by the original authors.

Original: For the SphereNet, all models are trained using the Adam optimizer (Kingma & Ba, 2014).
Revised: For the SphereNet, all models are trained using the Adam optimizer (Kingma & Ba, 2014).

Original: The optimal hyperparameters are obtained by grid search.
Revised: The optimal hyperparameters are obtained by grid search.

Original: Network configurations and search space for all models are provided in Appendix D.
Revised: Network configurations and search space for all models are provided in Appendix D.

Original: Code is contained in supplementary material.
Revised: Code is contained in supplementary material.

Original: 6. OC 
Revised: 6. OC

Original: The Open Catalyst 2020 (OC20) dataset is a newly released large-scale dataset for catalyst discovery and optimization (Chanussot et al., 2020).
Revised: The Open Catalyst 2020 (OC20) dataset is a newly released large-scale dataset for catalyst discovery and optimization (Chanussot et al., 2020).

Original: It comprises millions of DFT relaxations across huge
Revised: It comprises millions of DFT relaxations across huge

Original: [Table] [Table]
Revised: [Table] [Table]

Original: chemical structure space such that machine learning models can be fully trained.
Revised: chemical structure space such that machine learning models can be fully trained.

Original: We focus on the
Revised: We focus on the

Original: IS2RE task in this work, and description of the data is provided in Appendix E. Results for CGCNN,
Revised: IS2RE task in this work, and description of the data is provided in Appendix E. Results for CGCNN,

Original: SchNet, and DimeNet++ are provided in Chanussot et al.
Revised: SchNet, and DimeNet++ are provided in Chanussot et al.

Original: The original GemNet paper does not contain results on the OC20 dataset, We use the publicly available code from the OC Project website 1 to produce results for GemNet-T.
Revised: The original GemNet paper does not contain results on the OC20 dataset, We use the publicly available code from the OC Project website 1 to produce results for GemNet-T.

Original: We report evaluation results of fixed epochs for SphereNet.
Revised: We report evaluation results of fixed epochs for SphereNet.

Original: Following a setting in Chanussot et al.
Revised: Following a setting in Chanussot et al.

Original: (2020), we use the direct approach and all the training data for training models.
Revised: (2020), we use the direct approach and all the training data for training models.

Original: The used metrics are the energy MAE and the percentage of Energies within a Threshold (EwT) of the ground truth energy.
Revised: The used metrics are the energy MAE and the percentage of Energies within a Threshold (EwT) of the ground truth energy.

Original: Table 1 shows that the SphereNet achieves the best performance on 3 out of 4 splits and the average in terms of energy MAE.
Revised: Table 1 shows that the SphereNet achieves the best performance on 3 out of 4 splits and the average in terms of energy MAE.

Original: For EwT, SphereNet is the best on all the 4 splits.
Revised: For EwT, SphereNet is the best on all the 4 splits.

Original: Specifically, it reduces the average energy MAE by 0.019, which is 3.10% of the second best model.
Revised: Specifically, it reduces the average energy MAE by 0.019, which is 3.10% of the second best model.

Original: In addition, it improves the average EwT from 3.42% to 3.64%, which is a large margin considering the inherently low EwT values.
Revised: In addition, it improves the average EwT from 3.42% to 3.64%, which is a large margin considering the inherently low EwT values.

Original: Notably, ForceNet (Hu et al., 2021) and GemNet (Klicpera et al., 2021) are recently proposed for quantum system learning.
Revised: Notably, ForceNet (Hu et al., 2021) and GemNet (Klicpera et al., 2021) are recently proposed for quantum system learning.

Original: A prominent advantage for ForceNet is its high efficiency and scalability to large molecules.
Revised: A prominent advantage for ForceNet is its high efficiency and scalability to large molecules.

Original: ForceNet focuses on S2EF thus there are no original results for the IS2RE task.
Revised: ForceNet focuses on S2EF thus there are no original results for the IS2RE task.

Original: However, DimeNet++ is slightly better than ForceNet in terms of performance, and our SphereNet outperforms DimeNet++ significantly.
Revised: However, DimeNet++ is slightly better than ForceNet in terms of performance, and our SphereNet outperforms DimeNet++ significantly.

Original: GemNet has two variants GemNet-T and GemNet-Q. GemNetT considers distance and angle information as input, and contains an effective architecture with novel network components, such as bilinear layers and scaling factors.
Revised: GemNet has two variants GemNet-T and GemNet-Q. GemNetT considers distance and angle information as input, and contains an effective architecture with novel network components, such as bilinear layers and scaling factors.

Original: We can see GemNet-T is similar as DimeNet++ in terms of performance.
Revised: We can see GemNet-T is similar as DimeNet++ in terms of performance.

Original: GemNet-Q is claimed to be able to capture universal representations of molecules.
Revised: GemNet-Q is claimed to be able to capture universal representations of molecules.

Original: However, it considers edge-base 2-hop information and the time complexity is extremely high.
Revised: However, it considers edge-base 2-hop information and the time complexity is extremely high.

Original: It cannot be configured properly to run experiments on the large catalyst molecules.
Revised: It cannot be configured properly to run experiments on the large catalyst molecules.

Original: [Table]
Revised: [Table]

Original: 6. QM 
Revised: 6. QM

Original: We apply the SphereNet to the QM9 dataset, which is widely used for predicting various properties of molecules.
Revised: We apply the SphereNet to the QM9 dataset, which is widely used for predicting various properties of molecules.

Original: It consists organic molecules composed of up to 9 heavy atoms.
Revised: It consists organic molecules composed of up to 9 heavy atoms.

Original: Thus, this test can examine the power of the SphereNet for similar quantum chemistry systems.
Revised: Thus, this test can examine the power of the SphereNet for similar quantum chemistry systems.

Original: The dataset is original split into three sets, where the training set contains 110,000, the validation set contains 10,000, and the test set contains 10,831 molecules.
Revised: The dataset is original split into three sets, where the training set contains 110,000, the validation set contains 10,000, and the test set contains 10,831 molecules.

Original: For energy-related properties, the training processes use the unit eV. All hyperparameters are tuned on the validation set and applied to the test set.
Revised: For energy-related properties, the training processes use the unit eV. All hyperparameters are tuned on the validation set and applied to the test set.

Original: We compare our SphereNet with baselines using mean absolute error (MAE) for each property and the overall mean standarized MAE (std. MAE) for all the 12 properties.
Revised: We compare our SphereNet with baselines using mean absolute error (MAE) for each property and the overall mean standarized MAE (std. MAE) for all the 12 properties.

Original: The comparison results are summarized in Table 2.
Revised: The comparison results are summarized in Table 2.

Original: SphereNets achieves best performance on 5 properties and the second best performance on 3 properties.
Revised: SphereNets achieves best performance on 5 properties and the second best performance on 3 properties.

Original: It also improves the overall mean std.
Revised: It also improves the overall mean std.

Original: MAE of the QM9 dataset from 0.98 to 0.
Revised: MAE of the QM9 dataset from 0.98 to 0.

Original: and sets the new state of the art.
Revised: and sets the new state of the art.

Original: Notably, the most recent method PaiNN uses the same data splits as SphereNet in terms of sample numbers.
Revised: Notably, the most recent method PaiNN uses the same data splits as SphereNet in terms of sample numbers.

Original: Its final performance is the average of three different runs on three random splits.
Revised: Its final performance is the average of three different runs on three random splits.

Original: We follow such settings and run SphereNet on four properties including ϵ HOMO , ϵ LUMO , U 0 , and µ .
Revised: We follow such settings and run SphereNet on four properties including ϵ HOMO , ϵ LUMO , U 0 , and µ .

Original: The corresponding results are 22 .
Revised: The corresponding results are 22 .

Original: 9 ± 0 .
Revised: 9 ± 0 .

Original: 2 , 18 . 8 ± 0 .
Revised: 2 , 18 . 8 ± 0 .

Original: 28 ± 0 .
Revised: 28 ± 0 .

Original: 05 , and 0 .
Revised: 05 , and 0 .

Original: 0243 ± 0 . 00 , respectively.
Revised: 0243 ± 0 . 00 , respectively.

Original: It is obvious that these results are highly close to those in Table 2, thus, we can draw consistent conclusions.
Revised: It is obvious that these results are highly close to those in Table 2, thus, we can draw consistent conclusions.

Original: 6. MD 
Revised: 6. MD

Original: The MD17 dataset is used to examine the expressive power of SphereNet for molecular dynamics simulations.
Revised: The MD17 dataset is used to examine the expressive power of SphereNet for molecular dynamics simulations.

Original: Following the settings in Schütt et al.
Revised: Following the settings in Schütt et al.

Original: ; Klicpera et al.
Revised: ; Klicpera et al.

Original: (2020b), we train a separate model for each molecule to predict atomic forces.
Revised: (2020b), we train a separate model for each molecule to predict atomic forces.

Original: We use 1000 samples for training, and each of the eight molecules has both the validation and test sets.
Revised: We use 1000 samples for training, and each of the eight molecules has both the validation and test sets.

Original: Note that all the baseline models employ a joint loss of forces and conserved energy during training.
Revised: Note that all the baseline models employ a joint loss of forces and conserved energy during training.

Original: In the original SchNet (Schütt et al., 2017) and DimeNet (Klicpera et al., 2020b) papers, the authors set the weight of force over energy (WoFE) to 100, while the NequIP (Batzner et al., 2021) and GemNet (Klicpera et al., 2021) papers use a weight of 1000.
Revised: In the original SchNet (Schütt et al., 2017) and DimeNet (Klicpera et al., 2020b) papers, the authors set the weight of force over energy (WoFE) to 100, while the NequIP (Batzner et al., 2021) and GemNet (Klicpera et al., 2021) papers use a weight of 1000.

Original: As the WoFE tends to affect the force prediction significantly, we perform SphereNet with both WoFE values for fair comparisons.
Revised: As the WoFE tends to affect the force prediction significantly, we perform SphereNet with both WoFE values for fair comparisons.

Original: PaiNN (Schütt et al., 2021) uses neither 100 noras WoFE, so we do not compare with it on MD17.
Revised: PaiNN (Schütt et al., 2021) uses neither 100 noras WoFE, so we do not compare with it on MD17.

Original: The results for forces are reported in Table 3.
Revised: The results for forces are reported in Table 3.

Original: Note that for Benzene, all models are evaluated on Benzene17, thus, the result for sGDML is 0.
Revised: Note that for Benzene, all models are evaluated on Benzene17, thus, the result for sGDML is 0.

Original: rather than 0.06 (Benzene18).
Revised: rather than 0.06 (Benzene18).

Original: We can observe from the table that when WoFE is 100 for all models,
Revised: We can observe from the table that when WoFE is 100 for all models,

Original: SphereNet consistently outperforms SchNet and DimeNet by largin margins.
Revised: SphereNet consistently outperforms SchNet and DimeNet by largin margins.

Original: Notably, sGDML is one of the original work that created the MD17 dataset with carefully-designed features.
Revised: Notably, sGDML is one of the original work that created the MD17 dataset with carefully-designed features.

Original: Compared with sGDML, SphereNet performs better on four and worse on the other four molecules, which is similar to DimeNet.
Revised: Compared with sGDML, SphereNet performs better on four and worse on the other four molecules, which is similar to DimeNet.

Original: One reason is sGDML incorporates molecular symmetries to boost precision, and different molecules have different symmetries.
Revised: One reason is sGDML incorporates molecular symmetries to boost precision, and different molecules have different symmetries.

Original: However, sGDML has poorer generalization power to larger datasets without hand-engineered features.
Revised: However, sGDML has poorer generalization power to larger datasets without hand-engineered features.

Original: In addition, SphereNet achieves much better overall std.
Revised: In addition, SphereNet achieves much better overall std.

Original: MAE than sGDML.
Revised: MAE than sGDML.

Original: When using the same WoFE that is 1000, SphereNet achieves similar results with GemNet in spite of that GemNet-T is of high complexity and contains carefully designed network components for performance boost.
Revised: When using the same WoFE that is 1000, SphereNet achieves similar results with GemNet in spite of that GemNet-T is of high complexity and contains carefully designed network components for performance boost.

Original: 6. C OMPLETENESS VERSUS E FFICIENCY
Revised: 6. C OMPLETENESS VERSUS E FFICIENCY

Original: [Table]
Revised: [Table]

Original: The message passing scheme Q-MP in GemNet represents the edge-based 2-hop geometric message passing and can generate complete representations of 3D molecular graphs.
Revised: The message passing scheme Q-MP in GemNet represents the edge-based 2-hop geometric message passing and can generate complete representations of 3D molecular graphs.

Original: We study the capability and efficiency of the proposed SMP by comparing with Q-MP.
Revised: We study the capability and efficiency of the proposed SMP by comparing with Q-MP.

Original: Specifically, We use the same backbone network for these two MP methods for fair comparisons.
Revised: Specifically, We use the same backbone network for these two MP methods for fair comparisons.

Original: We extensively use two backbones, which are the SphereNet backbone introduced in
Revised: We extensively use two backbones, which are the SphereNet backbone introduced in

Original: Sec. 4 and the GemNet backbone proposed in Klicpera et al.
Revised: Sec. 4 and the GemNet backbone proposed in Klicpera et al.

Original: We conduct experiments on MD17, reporting performance and average running time for all the 8 molecules per epoch using the same computing infrastructure (Nvidia GeForce RTX 2080 TI 11GB).
Revised: We conduct experiments on MD17, reporting performance and average running time for all the 8 molecules per epoch using the same computing infrastructure (Nvidia GeForce RTX 2080 TI 11GB).

Original: Results are shown in Table 4, from which we can observe that on either backbone network, SMP achieves very similar results with Q-MP.
Revised: Results are shown in Table 4, from which we can observe that on either backbone network, SMP achieves very similar results with Q-MP.

Original: However, the time cost is much less than SMP, which indicates it is much more efficient than Q-MP.
Revised: However, the time cost is much less than SMP, which indicates it is much more efficient than Q-MP.

Original: Based on analyses in Sec. 3.2, SMP can distinguish almost all molecular structures, and the failure cases may not exist in nature.
Revised: Based on analyses in Sec. 3.2, SMP can distinguish almost all molecular structures, and the failure cases may not exist in nature.

Original: Hence, SMP performs similarly with Q-MP even though the latter is complete theoretically but not scalable in practice.
Revised: Hence, SMP performs similarly with Q-MP even though the latter is complete theoretically but not scalable in practice.

Original: We further compare the efficiency between
Revised: We further compare the efficiency between

Original: SphereNet and other models in terms of parameters and time cost in Appendix F. SphereNet uses similar computing budget with others but achieves the best performance.
Revised: SphereNet and other models in terms of parameters and time cost in Appendix F. SphereNet uses similar computing budget with others but achieves the best performance.

Original: 6. A BLATION S TUDY
Revised: 6. A BLATION S TUDY

Original: [Table]
Revised: [Table]

Original: The proposed SMP considers all the distance, angle, and torsion, leading to more powerful data representations.
Revised: The proposed SMP considers all the distance, angle, and torsion, leading to more powerful data representations.

Original: We investigate contributions of different geometries to demonstrate the advances of our SMP.
Revised: We investigate contributions of different geometries to demonstrate the advances of our SMP.

Original: We remove torsion information from SMP which we denote as “SMP w/o φ ”; we further remove angle information which we denote as “SMP w/o ( θ , φ )”.
Revised: We remove torsion information from SMP which we denote as “SMP w/o φ ”; we further remove angle information which we denote as “SMP w/o ( θ , φ )”.

Original: The three message passing strategies are integrated to the same architecture with other network parts remaining the same.
Revised: The three message passing strategies are integrated to the same architecture with other network parts remaining the same.

Original: We evaluate these models on four molecules of
Revised: We evaluate these models on four molecules of

Original: MD17.
Revised: MD17.

Original: Table 5 shows that SMP outperforms SMP w/o φ , and SMP w/o φ outperforms “SMP w/o ( θ , φ )”.
Revised: Table 5 shows that SMP outperforms SMP w/o φ , and SMP w/o φ outperforms “SMP w/o ( θ , φ )”.

Original: These results demonstrate the effectiveness of angle and torsion information used in the SMP.
Revised: These results demonstrate the effectiveness of angle and torsion information used in the SMP.

Original: The best performance of SMP further reveals that SMP represents an accurate scheme for 3D graphs.
Revised: The best performance of SMP further reveals that SMP represents an accurate scheme for 3D graphs.

Original: In addition, we provide visualization results for SphereNet filters in Appendix G to further show that all the distance, angle, and torsion information determine the structural semantics of filters.
Revised: In addition, we provide visualization results for SphereNet filters in Appendix G to further show that all the distance, angle, and torsion information determine the structural semantics of filters.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: 3D information is important for molecules but there lacks a principled message passing framework to consider it.
Revised: 3D information is important for molecules but there lacks a principled message passing framework to consider it.

Original: We first propose the spherical message passing as a unifying and efficient scheme that can achieve approximately complete representations of molecules without increasing computing budget.
Revised: We first propose the spherical message passing as a unifying and efficient scheme that can achieve approximately complete representations of molecules without increasing computing budget.

Original: Based on SMP and meaningful physical representations, SphereNet is presented, and experiments on various types of datasets demonstrates its capability, efficiency, and scalibility.
Revised: Based on SMP and meaningful physical representations, SphereNet is presented, and experiments on various types of datasets demonstrates its capability, efficiency, and scalibility.

Original: REPRODUCIBILITY STATEMENT
Revised: REPRODUCIBILITY STATEMENT

Original: Detailed experimental setup is provided in Appendix D. Implementation hyper-parameters of
Revised: Detailed experimental setup is provided in Appendix D. Implementation hyper-parameters of

Original: SphereNet on all the three datasets OC20, QM9, and MD17 are given in Table 6, Table and
Revised: SphereNet on all the three datasets OC20, QM9, and MD17 are given in Table 6, Table and

Original: Table 8, respectively.
Revised: Table 8, respectively.

Original: Code is attached in supplementary material.
Revised: Code is attached in supplementary material.

Original: A U PDATE F UNCTIONS IN SMP
Revised: A U PDATE F UNCTIONS IN SMP

Original: [Figure]
Revised: [Figure]

Original: The function ϕ e is applied to each edge k and outputs the updated edge vector e ′ k .
Revised: The function ϕ e is applied to each edge k and outputs the updated edge vector e ′ k .

Original: The indices of the input geometries to ϕ e are illustrated in
Revised: The indices of the input geometries to ϕ e are illustrated in

Original: Fig.
Revised: Fig.

Original: 5 (a).
Revised: 5 (a).

Original: Correspondingly, the inputs include the old edge vector e k , the receiver node vector v r k , the sender node vector v s k , the set of edges
Revised: Correspondingly, the inputs include the old edge vector e k , the receiver node vector v r k , the sender node vector v s k , the set of edges

Original: E s k that point to the node s k , and the 3D position information for all the nodes connected by the edge k and edges in E s k with the index set as r k ∪ s k
Revised: E s k that point to the node s k , and the 3D position information for all the nodes connected by the edge k and edges in E s k with the index set as r k ∪ s k

Original: ∪N s k .
Revised: ∪N s k .

Original: The function ρ p → e aggregates 3D information from these nodes to update the edge k .
Revised: The function ρ p → e aggregates 3D information from these nodes to update the edge k .

Original: The function ϕ v is used for per-node update and generates the new node vector v ′
Revised: The function ϕ v is used for per-node update and generates the new node vector v ′

Original: i for each node i .
Revised: i for each node i .

Original: An illustration of the indices of the inputs to ϕ v is provided in Fig.
Revised: An illustration of the indices of the inputs to ϕ v is provided in Fig.

Original: 5 (b).
Revised: 5 (b).

Original: The inputs include the old node vector v i , the set of edges E
Revised: The inputs include the old node vector v i , the set of edges E

Original: ′ i
Revised: ′ i

Original: that point to the node i , and 3D information for all the related nodes (the index set is i ∪ N i ).
Revised: that point to the node i , and 3D information for all the related nodes (the index set is i ∪ N i ).

Original: The functions ρ e → v is applied to aggregate the input edge features for updating the node i .
Revised: The functions ρ e → v is applied to aggregate the input edge features for updating the node i .

Original: The function ϕ u is used to update the global graph feature, while the function ρ v → u aggregates information from all the edge features.
Revised: The function ϕ u is used to update the global graph feature, while the function ρ v → u aggregates information from all the edge features.

Original: The three information update functions ϕ e , ϕ v , and ϕ u can be implemented in different ways, such as using neural networks and mathematical operations.
Revised: The three information update functions ϕ e , ϕ v , and ϕ u can be implemented in different ways, such as using neural networks and mathematical operations.

Original: In SMP, the 3D information in P is converted and incorporated to update each message e k .
Revised: In SMP, the 3D information in P is converted and incorporated to update each message e k .

Original: Hence, SMP uses ρ p → e compute position representations for edges.
Revised: Hence, SMP uses ρ p → e compute position representations for edges.

Original: Note that as absolute Cartesian coordinates stored in P are not invariant to translation and rotation, they are not used as immediate inputs to machine learning models.
Revised: Note that as absolute Cartesian coordinates stored in P are not invariant to translation and rotation, they are not used as immediate inputs to machine learning models.

Original: The position aggregation function can be flexibly adapted to generate invariant representations.
Revised: The position aggregation function can be flexibly adapted to generate invariant representations.

Original: For example, ρ p → e in Eq.
Revised: For example, ρ p → e in Eq.

Original: (1) can be adapted to a spherical Bessel basis function.
Revised: (1) can be adapted to a spherical Bessel basis function.

Original: B I NFORMATION U PDATE AND A RCHITECTURE OF S PHERE N ET
Revised: B I NFORMATION U PDATE AND A RCHITECTURE OF S PHERE N ET

Original: we assume the message e k for the edge k is the center message for update.
Revised: we assume the message e k for the edge k is the center message for update.

Original: The input block generates the initial message for the edge k , and takes only the distance representation Ψ( d ) as the input.
Revised: The input block generates the initial message for the edge k , and takes only the distance representation Ψ( d ) as the input.

Original: Each interaction block updates the message for the edge k .
Revised: Each interaction block updates the message for the edge k .

Original: The inputs include messages for all the neighboring edges, and all three representations, including Ψ( d, θ, φ ) , Ψ( d, θ ) , and Ψ( d ) based on the edge k and its neighboring edges.
Revised: The inputs include messages for all the neighboring edges, and all three representations, including Ψ( d, θ, φ ) , Ψ( d, θ ) , and Ψ( d ) based on the edge k and its neighboring edges.

Original: The output block first takes both the distance representation and the current message for k as inputs.
Revised: The output block first takes both the distance representation and the current message for k as inputs.

Original: Then the feature vector of the receiver atom for the edge k (atom r k in Fig.
Revised: Then the feature vector of the receiver atom for the edge k (atom r k in Fig.

Original: 2 and Eq.
Revised: 2 and Eq.

Original: (1)) is obtained by aggregating all the messages pointing to it, where other messages have a similar update process as e k .
Revised: (1)) is obtained by aggregating all the messages pointing to it, where other messages have a similar update process as e k .

Original: Detailed architecture of SphereNet is provided in Fig.
Revised: Detailed architecture of SphereNet is provided in Fig.

Original: Specifically, SphereNet is composed of an input block, followed by multiple interaction blocks and output blocks.
Revised: Specifically, SphereNet is composed of an input block, followed by multiple interaction blocks and output blocks.

Original: For the purpose of simplicity, the architecture is explained by updating the receiver note r k of the message e k , as described in
Revised: For the purpose of simplicity, the architecture is explained by updating the receiver note r k of the message e k , as described in

Original: Eq.
Revised: Eq.

Original: (1) and Sec. 4 in main paper.
Revised: (1) and Sec. 4 in main paper.

Original: Input Block aims at constructing initial message e k for the edge k .
Revised: Input Block aims at constructing initial message e k for the edge k .

Original: Inputs include the distance representation Ψ( d ) for edge k , initial node embeddings v s k , v r k for the sender node s k , and the receiver node r k .
Revised: Inputs include the distance representation Ψ( d ) for edge k , initial node embeddings v s k , v r k for the sender node s k , and the receiver node r k .

Original: The distance information is encoded by using a LB2 block.
Revised: The distance information is encoded by using a LB2 block.

Original: Interaction Block updates the message e k with incorporating all the three physical representations.
Revised: Interaction Block updates the message e k with incorporating all the three physical representations.

Original: Input 3D information includes the distance embedding Ψ( d ) , the angle Ψ( d, θ ) , and the torsion Ψ( d, θ, φ ) .
Revised: Input 3D information includes the distance embedding Ψ( d ) , the angle Ψ( d, θ ) , and the torsion Ψ( d, θ, φ ) .

Original: The initial embedding sizes for them are L , N × L , and N 2 × L , respectively.
Revised: The initial embedding sizes for them are L , N × L , and N 2 × L , respectively.

Original: Other inputs are old message e k and the set of messages
Revised: Other inputs are old message e k and the set of messages

Original: E s k that point to the sender node s k .
Revised: E s k that point to the sender node s k .

Original: Similar to the input block, each type of 3D information is encoded by using a block LB2.
Revised: Similar to the input block, each type of 3D information is encoded by using a block LB2.

Original: Note that each ⊙ indicates the element-wise multiplication between the corresponding 3D information represented as a vector and each message in the set E s k .
Revised: Note that each ⊙ indicates the element-wise multiplication between the corresponding 3D information represented as a vector and each message in the set E s k .

Original: Thus, each neighboring message of e k is gated by the encoded 3D information.
Revised: Thus, each neighboring message of e k is gated by the encoded 3D information.

Original: The (cid:80) aggregates all the gated messages in E s k to a vector, which is added to the transformation of the old message e k as the updated message e ′ k .
Revised: The (cid:80) aggregates all the gated messages in E s k to a vector, which is added to the transformation of the old message e k as the updated message e ′ k .

Original: The transformation branch for old message e k is composed of several nonlinear layers and residual blocks, as shown in Fig.
Revised: The transformation branch for old message e k is composed of several nonlinear layers and residual blocks, as shown in Fig.

Original: Output Block aggregates all the incoming messages to update the feature for node r k .
Revised: Output Block aggregates all the incoming messages to update the feature for node r k .

Original: Each incoming message has the same update process as e k by interaction blocks.
Revised: Each incoming message has the same update process as e k by interaction blocks.

Original: For the purpose of clear illustration, we use e ′ k to represent each updated incoming message, which is further gated by the distance representation vector Ψ( d ) .
Revised: For the purpose of clear illustration, we use e ′ k to represent each updated incoming message, which is further gated by the distance representation vector Ψ( d ) .

Original: C R ELATIONS WITH P RIOR M ESSAGE P ASSING M ETHODS
Revised: C R ELATIONS WITH P RIOR M ESSAGE P ASSING M ETHODS

Original: Our SMP is a specific architecture for 3D graphs and is formally defined in Eq.
Revised: Our SMP is a specific architecture for 3D graphs and is formally defined in Eq.

Original: Especially, the message passing schemes used in some existing models can be viewed as special cases of SMP as they only encode partial 3D information.
Revised: Especially, the message passing schemes used in some existing models can be viewed as special cases of SMP as they only encode partial 3D information.

Original: In this section, we clearly give the used functions as well as their inputs for each existing method.
Revised: In this section, we clearly give the used functions as well as their inputs for each existing method.

Original: Basically, we describe how each method realizes Eq.
Revised: Basically, we describe how each method realizes Eq.

Original: C. S CH N ET , S CHÜTT ET AL . (2017)
Revised: C. S CH N ET , S CHÜTT ET AL . (2017)

Original: In SchNet, the used aggregation function to encode 3D position information is ρ p → e ( { r h } h = r k ∪ s k )
Revised: In SchNet, the used aggregation function to encode 3D position information is ρ p → e ( { r h } h = r k ∪ s k )

Original: =
Revised: =

Original: Ψ ( ∥ r r k −
Revised: Ψ ( ∥ r r k −

Original: r
Revised: r

Original: s k ∥ ) , which converts the position information to an embedding of distance with radial basis functions.
Revised: s k ∥ ) , which converts the position information to an embedding of distance with radial basis functions.

Original: In addition to the ρ p → e function, the ϕ e function used is
Revised: In addition to the ρ p → e function, the ϕ e function used is

Original: NN ( NN ( v r k ) ⊙ NN (Ψ ( ∥
Revised: NN ( NN ( v r k ) ⊙ NN (Ψ ( ∥

Original: r r k − r s k ∥ ))) , where NN denotes a neural network and ⊙ denotes the element-wise multiplication.
Revised: r r k − r s k ∥ ))) , where NN denotes a neural network and ⊙ denotes the element-wise multiplication.

Original: The ρ e → v function is (cid:80) ( e ′ k ,r k
Revised: The ρ e → v function is (cid:80) ( e ′ k ,r k

Original: ,s k ) ∈ E ′
Revised: ,s k ) ∈ E ′

Original: i e ′ k .
Revised: i e ′ k .

Original: The ϕ v function is v
Revised: The ϕ v function is v

Original: i + (cid:80) ( e ′ k ,r k ,s k ) ∈
Revised: i + (cid:80) ( e ′ k ,r k ,s k ) ∈

Original: E
Revised: E

Original: ′ i
Revised: ′ i

Original: e ′ k .
Revised: e ′ k .

Original: The global feature u is updated based on the final node features V T and the function is ϕ u
Revised: The global feature u is updated based on the final node features V T and the function is ϕ u

Original: = (cid:80) i
Revised: = (cid:80) i

Original: =1: n NN (cid:0) v Ti (cid:1) .
Revised: =1: n NN (cid:0) v Ti (cid:1) .

Original: Formally, the update process is expressed as
Revised: Formally, the update process is expressed as

Original: [Equation]
Revised: [Equation]

Original: C. P HYS N ET , U NKE & M EUWLY (2019)
Revised: C. P HYS N ET , U NKE & M EUWLY (2019)

Original: PhysNet uses distance between atoms as an important feature and proposes more powerful neural networks for chemical applications.
Revised: PhysNet uses distance between atoms as an important feature and proposes more powerful neural networks for chemical applications.

Original: The position aggregation function is ρ p → e ( { r h } h = r k ∪ s k )
Revised: The position aggregation function is ρ p → e ( { r h } h = r k ∪ s k )

Original: =
Revised: =

Original: Ψ ( ∥ r r k −
Revised: Ψ ( ∥ r r k −

Original: r
Revised: r

Original: s k ∥ ) , where Ψ is any radial basis function with a smooth cutoff.
Revised: s k ∥ ) , where Ψ is any radial basis function with a smooth cutoff.

Original: For the information update functions, the ϕ e function is σ ( W 1 ) σ ( v s k ) ⊙ W 2 Ψ (
Revised: For the information update functions, the ϕ e function is σ ( W 1 ) σ ( v s k ) ⊙ W 2 Ψ (

Original: ∥ r r k − r s k ∥ ) , the ϕ v function is NN (cid:16) W 3 ⊙ v i + NN (cid:16) σ ( W 4 ) σ ( v i )
Revised: ∥ r r k − r s k ∥ ) , the ϕ v function is NN (cid:16) W 3 ⊙ v i + NN (cid:16) σ ( W 4 ) σ ( v i )

Original: + (cid:80) ( e ′ k ,r k
Revised: + (cid:80) ( e ′ k ,r k

Original: ,s k ) ∈ E ′
Revised: ,s k ) ∈ E ′

Original: i e ′ k
Revised: i e ′ k

Original: (cid:17)(cid:17) and the ϕ u function is u + (cid:80) i
Revised: (cid:17)(cid:17) and the ϕ u function is u + (cid:80) i

Original: =1: n NN ( v ′ i ) .
Revised: =1: n NN ( v ′ i ) .

Original: Here NN denotes a neural network, W 1 , W 2 , W 3 , W 4 are learnable weight matrices, σ is an activate function, and ⊙ denotes the element-wise multiplication.
Revised: Here NN denotes a neural network, W 1 , W 2 , W 3 , W 4 are learnable weight matrices, σ is an activate function, and ⊙ denotes the element-wise multiplication.

Original: PhysNet is expressed as
Revised: PhysNet is expressed as

Original: [Equation]
Revised: [Equation]

Original: C. D IME N ET , K LICPERA ET AL . (2020 B )
Revised: C. D IME N ET , K LICPERA ET AL . (2020 B )

Original: DimeNet explicitly considers distances between atoms and directions of directed edges.
Revised: DimeNet explicitly considers distances between atoms and directions of directed edges.

Original: The aggregation functions on the position information is ρ p → e = (Ψ ( d ) ∥
Revised: The aggregation functions on the position information is ρ p → e = (Ψ ( d ) ∥

Original: Ψ ( d, θ )) , where ∥ denotes concatenation, Ψ ( d ) and Ψ ( d, θ ) are the same basis functions used in SphereNet as introduced in Sec. 4.
Revised: Ψ ( d, θ )) , where ∥ denotes concatenation, Ψ ( d ) and Ψ ( d, θ ) are the same basis functions used in SphereNet as introduced in Sec. 4.

Original: Specifically, Ψ ( d ) denotes the representation of the distance based on spherical Bessel function, and Ψ ( d, θ ) denotes the representation of distance and angle based on spherical Bessel function and spherical harmonics.
Revised: Specifically, Ψ ( d ) denotes the representation of the distance based on spherical Bessel function, and Ψ ( d, θ ) denotes the representation of distance and angle based on spherical Bessel function and spherical harmonics.

Original: For other functions, the ϕ e function used is e ′ k
Revised: For other functions, the ϕ e function used is e ′ k

Original: = (cid:16) e ′ k, 1 ∥
Revised: = (cid:16) e ′ k, 1 ∥

Original: e ′ k, 2 (cid:17) with e ′ k,
Revised: e ′ k, 2 (cid:17) with e ′ k,

Original: [Equation]
Revised: [Equation]

Original: and e ′ k, 2 = W 5 Ψ ( d j ) ⊙ e ′ k, 1 , where NN denotes a neural network, W 1 , W 2 , W 3 , W 4 , W 5 are different weight matrices, and σ is an activation function.
Revised: and e ′ k, 2 = W 5 Ψ ( d j ) ⊙ e ′ k, 1 , where NN denotes a neural network, W 1 , W 2 , W 3 , W 4 , W 5 are different weight matrices, and σ is an activation function.

Original: The ρ e → v function is (cid:80) ( e ′ k ,r k
Revised: The ρ e → v function is (cid:80) ( e ′ k ,r k

Original: ,s k ) ∈ E ′
Revised: ,s k ) ∈ E ′

Original: i e ′ k,and the ϕ v is NN (cid:16)(cid:80) ( e ′ k ,r k
Revised: i e ′ k,and the ϕ v is NN (cid:16)(cid:80) ( e ′ k ,r k

Original: ,s k ) ∈ E ′
Revised: ,s k ) ∈ E ′

Original: i e ′ k, 2 (cid:17) .
Revised: i e ′ k, 2 (cid:17) .

Original: The ρ v → u is (cid:80) i =1: n v ′ i and the ϕ u is u + (cid:80) i
Revised: The ρ v → u is (cid:80) i =1: n v ′ i and the ϕ u is u + (cid:80) i

Original: =1: n v ′ i .
Revised: =1: n v ′ i .

Original: Note that ρ p → v , ρ p → u , ρ e → u functions are not required in DimeNet.
Revised: Note that ρ p → v , ρ p → u , ρ e → u functions are not required in DimeNet.

Original: The whole model is expressed as e
Revised: The whole model is expressed as e

Original: [Equation]
Revised: [Equation]

Original: = ( e k,
Revised: = ( e k,

Original: [Equation]
Revised: [Equation]

Original: e k, 2 ) ,
Revised: e k, 2 ) ,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: θ )) ,
Revised: θ )) ,

Original: [Equation]
Revised: [Equation]

Original: (cid:17)(cid:17)
Revised: (cid:17)(cid:17)

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: i =1: n
Revised: i =1: n

Original: [Equation]
Revised: [Equation]

Original: D E XPERIMENTAL S ETUP
Revised: D E XPERIMENTAL S ETUP

Original: For all the models used in three datasets, we set input embedding size = 256 and output embedding size = 64 for both LB2 and LB blocks.
Revised: For all the models used in three datasets, we set input embedding size = 256 and output embedding size = 64 for both LB2 and LB blocks.

Original: For each separate model, we first perform warmup on initial learning rate.
Revised: For each separate model, we first perform warmup on initial learning rate.

Original: Then two learning rate strategies, including ReduceLROnPlateau and StepLR, are used for training.
Revised: Then two learning rate strategies, including ReduceLROnPlateau and StepLR, are used for training.

Original: For StepLR, the learning rate is decayed by the decay ratio every fixed epochs represented as step size.
Revised: For StepLR, the learning rate is decayed by the decay ratio every fixed epochs represented as step size.

Original: We do not use weight decay or dropout for all models.
Revised: We do not use weight decay or dropout for all models.

Original: Some hyperparameters are fixed values, and some are tuned by grid search.
Revised: Some hyperparameters are fixed values, and some are tuned by grid search.

Original: Values/search space of hyperparameters for OC20, QM9, and MD17 are provided in Table 6, Table 7, and Table 8, respectively.
Revised: Values/search space of hyperparameters for OC20, QM9, and MD17 are provided in Table 6, Table 7, and Table 8, respectively.

Original: Optimized hyperparameters are tuned on validation sets and applied to test sets for QM9 and MD17.
Revised: Optimized hyperparameters are tuned on validation sets and applied to test sets for QM9 and MD17.

Original: For OC20, optimized hyperparameters are obtained on the ID split within max epochs, and then applied to the other three splits.
Revised: For OC20, optimized hyperparameters are obtained on the ID split within max epochs, and then applied to the other three splits.

Original: Pytorch is used to implement all methods.
Revised: Pytorch is used to implement all methods.

Original: For QM9 and MD17 datasets, all models are trained using one NVIDIA GeForce RTX 2080 Ti 11GB GPU.
Revised: For QM9 and MD17 datasets, all models are trained using one NVIDIA GeForce RTX 2080 Ti 11GB GPU.

Original: For the OC20 dataset, all models are trained using four NVIDIA RTX A6000 48GB GPUs.
Revised: For the OC20 dataset, all models are trained using four NVIDIA RTX A6000 48GB GPUs.

Original: E OC20 D ATA D ESCRIPTION
Revised: E OC20 D ATA D ESCRIPTION

Original: There exist three tasks including S2EF, IS2RS, and IS2RE.
Revised: There exist three tasks including S2EF, IS2RS, and IS2RE.

Original: In this work, we focus on IS2RE that predicts structure’s energy in the relaxed state.
Revised: In this work, we focus on IS2RE that predicts structure’s energy in the relaxed state.

Original: It is the most common task in catalysis as relaxed energies usually influence the catalyst activity.
Revised: It is the most common task in catalysis as relaxed energies usually influence the catalyst activity.

Original: The dataset for IS2RE is originally split into training/validation/test sets.
Revised: The dataset for IS2RE is originally split into training/validation/test sets.

Original: There are 460,318 structures in the training dataset in total.
Revised: There are 460,318 structures in the training dataset in total.

Original: The test label is not publicly available.
Revised: The test label is not publicly available.

Original: Performance is evaluated on the validation set, which has four splits including In Domain (ID), Out of Domain Adsorbates (OOD Ads), Out of Domain catalysts (OOD cat), and Out of Domain Adsorbates and catalysts (OOD Both), where numbers of structures are 24,943, 24,961, 24,963, 24,987, respectively.
Revised: Performance is evaluated on the validation set, which has four splits including In Domain (ID), Out of Domain Adsorbates (OOD Ads), Out of Domain catalysts (OOD cat), and Out of Domain Adsorbates and catalysts (OOD Both), where numbers of structures are 24,943, 24,961, 24,963, 24,987, respectively.

Original: The average number of atoms per structure is 77.75.
Revised: The average number of atoms per structure is 77.75.

Original: F E FFICIENCY S TUDY OF S PHERE N ET
Revised: F E FFICIENCY S TUDY OF S PHERE N ET

Original: We study the efficiency of SphereNet by comparing with other models regarding number of parameters and time cost per epoch using the same computing infrastructure (Nvidia GeForce RTX 2080 TI
Revised: We study the efficiency of SphereNet by comparing with other models regarding number of parameters and time cost per epoch using the same computing infrastructure (Nvidia GeForce RTX 2080 TI

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 11GB).
Revised: 11GB).

Original: Experiments are conducted on the property U 0 of QM9 and results are shown in Table
Revised: Experiments are conducted on the property U 0 of QM9 and results are shown in Table

Original: [Table]
Revised: [Table]

Original: It is obvious that SphereNet uses similar computational resources as DimeNet++ and GemNet-T, and is much more efficient than DimeNet.
Revised: It is obvious that SphereNet uses similar computational resources as DimeNet++ and GemNet-T, and is much more efficient than DimeNet.

Original: The main reason could be we develop an efficient way to compute torsion, as introduced in Sec. 3 and Fig.
Revised: The main reason could be we develop an efficient way to compute torsion, as introduced in Sec. 3 and Fig.

Original: 2 (b).
Revised: 2 (b).

Original: Moreover, GemNet-Q cannot run on QM using the infrastructure as mentioned above.
Revised: Moreover, GemNet-Q cannot run on QM using the infrastructure as mentioned above.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: G S PHERE N ET F ILTER V ISUALIZATION
Revised: G S PHERE N ET F ILTER V ISUALIZATION

Original: We visualize SphereNet filters from a learned SphereNet model.
Revised: We visualize SphereNet filters from a learned SphereNet model.

Original: Specifically, we port learned weights for the block LB2 after the torsion embedding Ψ( d, θ, φ ) in Fig.
Revised: Specifically, we port learned weights for the block LB2 after the torsion embedding Ψ( d, θ, φ ) in Fig.

Original: For each location represented by a tuple ( d, θ, φ ) , the initial embedding size is N 2 × L .
Revised: For each location represented by a tuple ( d, θ, φ ) , the initial embedding size is N 2 × L .

Original: The computation for the above LB2 is
Revised: The computation for the above LB2 is

Original: W 1 ( W 2 Ψ( d, θ, φ )) , which results in the new embedding size of 64 for each location ( d, θ, φ ) .
Revised: W 1 ( W 2 Ψ( d, θ, φ )) , which results in the new embedding size of 64 for each location ( d, θ, φ ) .

Original: We then perform sampling on locations in 3D space for visualizing weights as SphereNet filters.
Revised: We then perform sampling on locations in 3D space for visualizing weights as SphereNet filters.

Original: The visualization results are provided in Fig.
Revised: The visualization results are provided in Fig.

Original: We set sampling rate in the torsion direction to be π/ 4 , thus, there are eight samples in the torsion direction.
Revised: We set sampling rate in the torsion direction to be π/ 4 , thus, there are eight samples in the torsion direction.

Original: There are totally 64 elements for each location, and we randomly pick 6 elements.
Revised: There are totally 64 elements for each location, and we randomly pick 6 elements.

Original: Apparently, among the distance, angle and torsion, considering any one when fixing the other two, the structural value of filters will be different when the one of interest changes.
Revised: Apparently, among the distance, angle and torsion, considering any one when fixing the other two, the structural value of filters will be different when the one of interest changes.

Original: It essentially shows that all the distance, angle, and torsion information determine the structural semantics of filters.
Revised: It essentially shows that all the distance, angle, and torsion information determine the structural semantics of filters.

Original: This further demonstrates that SMP enables the learning of different 3D information for improved representations.
Revised: This further demonstrates that SMP enables the learning of different 3D information for improved representations.