Original: Spherical Channels for Modeling Atomic Interactions
Revised: Spherical Channels for Modeling Atomic Interactions

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Modeling the energy and forces of atomic systems is a fundamental problem incomputational chemistry with the potential to help address many of the world’s mostpressing problems, including those related to energy scarcity and climate change.
Revised: Modeling the energy and forces of atomic systems is a fundamental problem incomputational chemistry with the potential to help address many of the world’s mostpressing problems, including those related to energy scarcity and climate change.

Original: These calculations are traditionally performed using Density Functional Theory,which is computationally very expensive.
Revised: These calculations are traditionally performed using Density Functional Theory,which is computationally very expensive.

Original: Machine learning has the potential todramatically improve the efficiency of these calculations from days or hours toseconds.
Revised: Machine learning has the potential todramatically improve the efficiency of these calculations from days or hours toseconds.

Original: We propose the Spherical Channel Network (SCN) to model atomic energiesand forces.
Revised: We propose the Spherical Channel Network (SCN) to model atomic energiesand forces.

Original: The SCN is a graph neural network where nodes represent atomsand edges their neighboring atoms.
Revised: The SCN is a graph neural network where nodes represent atomsand edges their neighboring atoms.

Original: The atom embeddings are a set of sphericalfunctions, called spherical channels, represented using spherical harmonics.
Revised: The atom embeddings are a set of sphericalfunctions, called spherical channels, represented using spherical harmonics.

Original: Wedemonstrate, that by rotating the embeddings based on the 3D edge orientation,more information may be utilized while maintaining the rotational equivariance ofthe messages.
Revised: Wedemonstrate, that by rotating the embeddings based on the 3D edge orientation,more information may be utilized while maintaining the rotational equivariance ofthe messages.

Original: While equivariance is a desirable property, we find that by relaxingthis constraint in both message passing and aggregation, improved accuracy may beachieved.
Revised: While equivariance is a desirable property, we find that by relaxingthis constraint in both message passing and aggregation, improved accuracy may beachieved.

Original: We demonstrate state-of-the-art results on the large-scale Open Catalyst2020 dataset in both energy and force prediction for numerous tasks and metrics.
Revised: We demonstrate state-of-the-art results on the large-scale Open Catalystdataset in both energy and force prediction for numerous tasks and metrics.

Original:  Introduction
Revised:  Introduction

Original: Modeling the properties of atomic systems is a foundational challenge in computational chemistryand critical to advancing technologies across numerous application domains.
Revised: Modeling the properties of atomic systems is a foundational challenge in computational chemistryand critical to advancing technologies across numerous application domains.

Original: Notable applicationsinclude drug discovery [29, 36] and the design of new catalysts for renewable energy storage tohelp in addressing climate change [44, 30].
Revised: Notable applicationsinclude drug discovery [28, 35] and the design of new catalysts for renewable energy storage tohelp in addressing climate change [44, 29].

Original: For catalyst discovery, new materials are currentlyevaluated using Density Functional Theory (DFT) that can estimate atomic energies and forces but iscomputationally very expensive; taking hours or days to evaluate a single material.
Revised: For catalyst discovery, new materials are currentlyevaluated using Density Functional Theory (DFT) that can estimate atomic energies and forces but iscomputationally very expensive; taking hours or days to evaluate a single material.

Original: Machine Learning(ML) has the potential to approximate DFT and dramatically speed up these calculations, allowingfor high throughput screening of new materials to help address some of the world’s most pressingchallenges.
Revised: Machine Learning(ML) has the potential to approximate DFT and dramatically speed up these calculations, allowingfor high throughput screening of new materials to help address some of the world’s most pressingchallenges.

Original: Our goal is to approximate DFT calculations using ML.
Revised: Our goal is to approximate DFT calculations using ML.

Original: An ML model takes as input a set of atompositions and their atomic numbers.
Revised: An ML model takes as input a set of atompositions and their atomic numbers.

Original: As outputs, a model calculates the structure’s energy (or otherproperties) and the per-atom forces, i.e., the forces exerted on each atom by the other atoms.
Revised: As outputs, a model calculates the structure’s energy (or otherproperties) and the per-atom forces, i.e., the forces exerted on each atom by the other atoms.

Original: Acommon approach to this problem is to use Graph Neural Networks (GNNs) [15, 43] where eachnode represents an atom and the set of nearby atoms as edges [34, 14, 19, 32, 35, 42, 28, 21].
Revised: Acommon approach to this problem is to use Graph Neural Networks (GNNs) [15, 43] where eachnode represents an atom and the set of nearby atoms as edges [33, 13, 19, 31, 34, 41, 27, 21].

Original: A key challenge in network design is balancing the use of model constraints.
Revised: A key challenge in network design is balancing the use of model constraints.

Original: When modeling atomicsystems, a common constraint is SO(3) rotation equivariance [40, 3, 1, 39, 31, 33], i.e ., if the atomic
Revised: When modeling atomicsystems, a common constraint is SO(3) rotation equivariance [39, 2, 1, 38, 30, 32], i.e ., if the atomic

Original: [Figure]
Revised: [Figure]

Original: system is rotated, the energies should remain constant and the atomic forces should similarly rotate.
Revised: system is rotated, the energies should remain constant and the atomic forces should similarly rotate.

Original: While this provides strong priors on the model to help in generalization, especially for smallerdatasets[7, 29], it can result in limiting the expressiveness of the network due to restrictions onnon-linear transformations for equivariant models [3, 39, 31, 33], or limiting interactions to pairs[32, 35, 42], triplets [21] or quadruplets of atoms [25, 22, 12] for invariant models.
Revised: While this provides strong priors on the model to help in generalization, especially for smallerdatasets[6, 28], it can result in limiting the expressiveness of the network due to restrictions onnon-linear transformations [2, 38, 30, 32], or limiting interactions to pairs [31, 34, 41], triplets [21]or quadruplets of atoms [25, 22, 11].

Original: Alternatively, anon-equivariant model [18] can provide more freedom to the model, but lead to the model needingto learn approximate equivariance through methods such as data augmentation [37].
Revised: Alternatively, a non-equivariant model [18] can provide morefreedom to the model, but lead to the model needing to learn approximate equivariance throughmethods such as data augmentation [36].

Original: To draw ananalogy with image detection, the use of a CNN [24] provides translation equivariance and removesthe need for the network to learn how to detect the same object at different locations.
Revised: To draw an analogy with image detection, the use of a CNN [24] provides translation equivariance and removes the need for the network to learn how todetect the same object at different locations.

Original: However, most CNNs are not equivariant to scale or rotation [41, 40], but are still effective in learning approximateequivariance through rotation and scale diversity in the training data.
Revised: However, most CNNs are not equivariant to scale orrotation [40, 39], but are still effective in learning approximate equivariance through rotation andscale diversity in the training data.

Original: What is the analogous balanceof constraints for modeling atomic systems?
Revised: What is the analogous balance of constraints for modeling atomicsystems?

Original: In this paper, we propose a GNN [15, 43] that balances the use of model constraints to aid ingeneralization while providing the network with the flexibility to learn accurate representations.
Revised: In this paper, we propose a GNN [15, 43] that balances the use of model constraints to aid ingeneralization while providing the network with the flexibility to learn accurate representations.

Original: We introduce the Spherical Channel Network (SCN) that explicitly models relative orientations ofall neighboring atoms; information that is critical to accurately predicting atomic properties.
Revised: We introduce the Spherical Channel Network (SCN) that explicitly models relative orientations ofall neighboring atoms; information that is critical to accurately predicting atomic properties.

Original: Eachnode’s embedding is a set of functions defined on the surface of a sphere ( S 2 → R ) .
Revised: Eachnode’s embedding is a set of functions defined on the surface of a sphere ( S 2 → R ) .

Original: The functionsare represented using spherical harmonics, similar to approaches that build strictly equivariantmodels [39, 1, 3, 10].
Revised: The functionsare represented using spherical harmonics, similar to approaches that build strictly equivariantmodels [38, 1, 2, 9].

Original: During message passing, angular information between atoms is conveyed byrotating or steering [5] the embeddings based on each edge’s orientation [38, 25].
Revised: During message passing, angular information between atoms is conveyed byrotating or steering [4] the embeddings based on each edge’s orientation [37, 25].

Original: We identify anexpanded set of spherical harmonic coefficients that are invariant to rotation, which can provide richinformation while maintaining a message’s rotation equivariance.
Revised: We identify anexpanded set of spherical harmonic coefficients that are invariant to rotation, which can provide richinformation while maintaining a message’s rotation equivariance.

Original: In addition, we demonstrate that ifthe equivariance constraint is relaxed, improved performance can be achieved by using additionalcoefficients.
Revised: In addition, we demonstrate that ifthe equivariance constraint is relaxed, improved performance can be achieved by using additionalcoefficients.

Original: We further improve the expressivity of the network by performing a pointwise non-linearfunction, which is only approximately equivariant, on the embeddings during message aggregation.
Revised: We further improve the expressivity of the network by performing a pointwise non-linearfunction on the embeddings during message aggregation.

Original: We demonstrate our Spherical Channel Network on the large-scale Open Catalyst 2020 (OC20) dataset[6], which contains atomic structures useful for numerous applications important to addressing climatechange.
Revised: We demonstrate our Spherical Channel Network on the large-scale Open Catalyst 2020 (OC20) dataset[5], which contains atomic structures useful for numerous applications important to addressing climatechange.

Original: State-of-the-art results are achieved for atomic force and initial structure to relaxed energyprediction with improvements of 8% − 11% .
Revised: State-of-the-art results are achieved for atomic force and initial structure to relaxed energyprediction with improvements of 8% − 11% .

Original: In addition, we demonstrate our model is more sampleefficient as compared to other state-of-the-art models.
Revised: In addition, we demonstrate our model is more sampleefficient as compared to other state-of-the-art models.

Original:  Approach
Revised:  Approach

Original: Given an atomic structure with n atoms, our goal is to predict the structure’s energy E and theper-atom forces f i for each atom i ∈ n .
Revised: Given an atomic structure with n atoms, our goal is to predict the structure’s energy E and theper-atom forces f i for each atom i ∈ n .

Original: These values are estimated using a Graph Neural Network(GNN) [15, 43] where each node represents an atom and edges represent nearby atoms. As input, the
Revised: These values are estimated using a Graph Neural Network

Original: [Figure]
Revised: [Figure]

Revised: (GNN) [15, 43] where each node represents an atom and edges represent nearby atoms.

Original: network is given the distance d ij between atoms i and j , and each atom’s atomic number a i .
Revised: As input, thenetwork is given the distance d ij between atoms i and j , and each atom’s atomic number a i .

Original: Theneighbors N i for an atom i are determined using a fixed distance threshold, or by picking a fixednumber of closest atoms.
Revised: Theneighbors N i for an atom i are determined using a fixed distance threshold, or by picking a fixednumber of closest atoms.

Original: 2.1 Node Embeddings
Revised: 2.1 Node Embeddings

Original: The angular or relative orientation information between atoms is critical to accurately modelingatomic properties such as energies and forces [21, 22].
Revised: The angular or relative orientation information between atoms is critical to accurately modelingatomic properties such as energies and forces [21, 22].

Original: Inspired by this, our node embeddingmodels the angular information from all neighboring atoms using spherical functions.
Revised: Inspired by this, our node embedding explicitlymodels the angular information from all neighboring atoms.

Original: Each nodei ’s embedding s
Revised: Each node i ’s embedding s

Original: i is a set of C functions or channels represented on a sphere ( S 2 → R ) , whoseargument is a 3D unit vector indicating the orientation.
Revised: i is a setof C functions or channels represented on a sphere ( S 2 → R ) , whose argument is a 3D unit vectorindicating the orientation.

Original: That is, s ic (ˆ r ) is the value of channel c fornode i for some orientation ˆ r ∈ R 3 .
Revised: That is, s ic (ˆ r ) is the value of channel c for node i for some orientationˆ r ∈ R 3 .

Original: Since the spherical channels contain orientation informationover the entire sphere, the network may reason about directional information for all neighboringatoms and not just atom pairs, triplets, etc.
Revised: Since the spherical channels contain orientation information over the entire sphere, thenetwork may reason about directional information for all neighboring atoms and not just atom pairs,triplets, etc.

Original: Each spherical channel c may be represented using severaldifferent approaches, such as discrete 2D grids sampled over the sphere, or spherical harmonics.
Revised: Each spherical channel c may be represented using several different approaches, suchas discrete 2D grids sampled over the sphere, or spherical harmonics.

Original: Aswe describe later, we use spherical harmonics due to their property of being SO(3)-equivariant (3Drotation equivariant).
Revised: As we describe later, we usespherical harmonics due to their property of being SO(3)-equivariant (3D rotation equivariant).

Original: Using spherical harmonics, a function s c (ˆ r ) defined on the sphere is represented by a set of weightedspherical harmonics Y ml with s c (ˆ r )
Revised: Using spherical harmonics, a function s c (ˆ r ) defined on the sphere is represented by a set of weightedspherical harmonics Y ml with s c (ˆ r )

Original: = (cid:80) l,m x
Revised: = (cid:80) l,m x

Original: lmc Y ml (ˆ r ) where l and m are the degree and orderof the basis functions with m ∈ [ − l, l ] and l ∈ L .
Revised: lmc Y ml (ˆ r ) where l and m are the degree and orderof the basis functions with m ∈ [ − l, l ] and l ∈ L .

Original: x lmc are the spherical harmonic coefficients,which we refer to as the coefficients of the spherical channel.
Revised: x lmc are the spherical harmonic coefficients,which we refer to as the coefficients of the spherical channel.

Original: For every degree l there exists 2 l + 1spherical harmonics (see Figure 1(a) for an illustration of functions for l ≤ 4 ), which results in a totalof ( L + 1) 2 basis functions and coefficients up to degree L .
Revised: For every degree l there exists 2 l + 1spherical harmonics (see Figure 1(a) for an illustration of functions for l ≤ 4 ), which results in a totalof ( L + 1) 2 basis functions and coefficients up to degree L .

Original: Therefore for a maximum degree of Lwith C channels, x has size ( L + 1) 2 × C .
Revised: Therefore for a maximum degree of Lwith C channels, x has size ( L + 1) 2 × C .

Original: The spherical channels are updated by the GNN through message passing for K layers to obtainthe final node embeddings S ( K ) .
Revised: The spherical channels are updated by the GNN through message passing for K layers to obtainthe final node embeddings S ( K ) .

Original: S (0) is initialized from an embedding based on the atom’s atomicnumber a i for l = 0 coefficients and the l ̸ = 0 coefficients are set to zero.
Revised: S (0) is initialized from an embedding based on the atom’s atomicnumber a i for l = 0 coefficients and the l ̸ = 0 coefficients are set to zero.

Original: The nodes’ embeddingsare updated by first calculating a set of messages m ij for each edge, which are then aggregated ateach node.
Revised: The nodes’ embeddingsare updated by first calculating a set of messages m ij for each edge, which are then aggregated ateach node.

Original: Finally, the energy and forces are estimated from S ( K ) .
Revised: Finally, the energy and forces are estimated from S ( K ) .

Original: We describe each of these stepsin turn.
Revised: We describe each of these stepsin turn.

Original: 2.2 Message Passing
Revised: 2.2 Message Passing

Original: Given a target node t and its neighbors s ∈ N t we want to update the embeddings x ( k ) t at iterationk ∈ K .
Revised: Given a target node t and its neighbors s ∈ N t we want to update the embeddings x ( k ) t at iterationk ∈ K .

Original: The embeddings x ( k ) t are a set of spherical harmonic coefficients indexed by their degreel , order m and channel c .
Revised: The embeddings x ( k ) t are a set of spherical harmonic coefficients indexed by their degreel , order m and channel c .

Original: An important and useful property of spherical harmonics is the function represented by the coefficients is steerable [11, 5], i.e ., it can be rotated in 3D space using a lineartransformation of the coefficients.
Revised: An important and useful property of spherical harmonics is the functionrepresented by the coefficients is steerable [10, 4], i.e ., it can be rotated in 3D space using a linear transformation of the coefficients.

Original: Specifically, for a 3D rotation matrix R there exists a matrix calleda Wigner D-matrix D l of size (2 l + 1 × 2 l + 1) that rotates the coefficients of degree l by the rotation
Revised: Specifically, for a 3D rotation matrix R there exists a matrix calleda Wigner D-matrix A l of size (2 l + 1 × 2 l + 1) that rotates the coefficients of degree l by the rotation

Original: R .
Revised: R .

Original: If x ilc are all coefficients of degree l for node i and channel c , then for any orientation ˆ r androtation R there exists a D l such that:
Revised: If x ilc are all coefficients of degree l for node i and channel c , then for any orientation ˆ r :s ic

Original: [Equation]
Revised: [Equation]

Original: When calculating the message m st from atom s to atom t , we want to use the information containedin both x ( k ) s and x ( k ) t given the context of the edge’s orientation ˆ d st , ˆ d st = d st / | d st | .
Revised: When calculating the message m st from atom s to atom t , we want to use the information containedin both x ( k ) s and x ( k ) t given the context of the edge’s orientation ˆ d st , ˆ d st = d st / | d st | .

Original: We do thisby rotating the embeddings by R st for which R st ˆ d st = [0 , 0 , 1] ⊤ , i.e ., the direction of the edge’sunit vector ˆ d st is aligned with the z-axis.
Revised: We do thisby rotating the embeddings by R st for which R st ˆ d st = [0 , 0 , 1] ⊤ , i.e ., the direction of the edge’sunit vector ˆ d st is aligned with the z-axis.

Original: Thus when the messages are calculated, the problem issimplified since thenetwork can assume the atoms are aligned along the z-axis.
Revised: Thus when the messages are calculated, the problem issimplified since the atoms are always oriented along the z-axis.

Original: After calculating themessages using a neural network F e , they are rotated back to the global coordinate frame:
Revised: After calculating the messages usinga neural network F m , they are rotated back to the global coordinate frame:

Original: [Equation]
Revised: [Equation]

Original: where the matrices D st and D − 1 st perform the rotation on the embeddings’ coefficients correspondingto the 3D rotation matrices R st and R − 1 st respectively.
Revised: where the matrices A st and A − 1 st perform the rotation on the embeddings’ coefficients correspondingto the 3D rotation matrices R st and R − 1 st respectively.

Original: To simplify notation, we assume the coefficients in x s across degree l and order m are flattened, and D st ∈ R ( L +1) 2 × R ( L +1) 2 is a blockdiagonal matrix constructed from the set of Wigner D-matrices across degrees, l ∈ L .
Revised: To simplify notation, we assume the coefficients in x s across degree l and order m are flattened, and A st ∈ R ( L +1) 2 × R ( L +1) 2 is a blockdiagonal matrix constructed from the set of Wigner D-matrices across degrees.

Original: D − 1 st is theinverse of D st , with D − 1 st = D ⊤ st .
Revised: A − 1 st is the inverseof A st , with A − 1 st = A ⊤ st .

Original: In addition to the rotated embeddings, F e isalso provided inputedge information that is invariant to rotations; | d st | is the magnitude of the distance between atoms sand t , and a s and a t their respective atomic numbers.
Revised: In addition to the rotated embeddings, F m is provided as input edgeinformation that is invariant to rotations; | d st | is the magnitude of the distance between atoms s andt , and a s and a t their respective atomic numbers.

Original: Note that the rotation matrix R st is not unique,since the roll rotation around the vector d st is not specified andis randomly sampled during training.
Revised: Note that the rotation matrix R st is not unique,since the roll rotation around the vector d st is not specified and randomly sampled during training.

Original: The implications of this are discussed below.
Revised: The implications of this are discussed below.

Original: The message function F e is computed using a neural network as illustrated in Figure 2(a).
Revised: The message function F m is computed using a neural network as illustrated in Figure 2(a).

Original: The atomicnumbers a s and a t are used to look up two independent embeddings of size E = 128 , and a set of1D basis functions are used to represent | d st | using equally spaced Gaussians every 0.02 Å from 0 tothe 8 Å with σ = 0 .
Revised: Theatomic numbers a s and a t are used to look up two independent embeddings of size E = 128 , anda set of 1D basis functions are used to represent | d st | using equally spaced Gaussians every 0.02Å from 0 to the 8 Å with σ = 0 .

Original: 04 followed by a linear layer with E outputs.
Revised: 04 followed by a linear layer with E outputs.

Original: Their values are added together andpassed through a neural network to produce a vector of size H .
Revised: Their values are addedtogether and passed through a neural network to produce a vector of size H .

Original: The rotated embeddingsD st x ( k ) s and
Revised: The rotated embeddings

Original: D st x ( k ) t are concatenated and passed through a single layer neural network to produce another vectorof size H .
Revised: A st x ( k ) s and A st x ( k ) t are concatenated and passed through a single layer neural network to produceanother vector of size H .

Original: These two are multiplied together to combine the edge information with the informationcontained in the rotated embeddings.
Revised: These two are multiplied together to combine the edge information withthe information contained in the rotated embeddings.

Original: Two more fully connected layers are performed with eachfollowed by a SiLU non-linear activation function [17].
Revised: Two more fully connected layers are performedeach followed by a SiLU non-linear activation function [17].

Original: Finally, a single linear layer is used to expandthe output to the original size of the embeddings.
Revised: Finally, a single linear layer is used toexpand the output to the original size of the embeddings.

Original: 2.2.1 Message Equivariance

Original: For our message function (Equation (2)) to be equivariant, the output of F e needs to be invariantto global rotations. In general, this is not the case since the matrix D st is not unique, i.e ., the rollrotation around the vector d st is not specified, and in practice it is randomly chosen.
Revised: Note as stated before, the matrix A st is not unique since the rotation around the vector d st is notspecified. Due to this, if all coefficients are included in the calculation of F m the result will not beequivariant with respect to rotations, since their values will change based on the arbitrary rotationaround d st .

Original: The roll rotationcorresponds to a rotation about the z-axis after rotating x ( k ) s and x ( k ) t by D st .
Revised: If rotation equivariance is desired for messages, the input and output coefficients to

Original: Due to this, all of therotated coefficients will vary based on the random rotation chosen, except the m = [-
-]
Revised: F m can be restricted to only those for which m = [-
-]0 .

Original: coefficientsthat are symmetric about the z-axis; see Appendix E andthe m = 0 bases highlighted in green in Figure 1(a).
Revised: The m = 0 spherical harmonic functions aresymmetric on the z-axis; see supplementary material and bases highlighted in green in Figure 1(a).

Original: If rotation equivariance is desired for messages, theinput and output coefficients to F ecan be restricted to only those for which m = 0 .{+
+}
Revised: Since the only remaining degree of freedom are rotations about the z-axis after rotating x ( k ) s and x ( k ) tby A st , the resulting m = {+
+}

Original:  Since the other inputs | d st | , a s and a t to F e are alsoinvariant to rotations, F e is invariant to rotations if only m = 0 coefficients are used.
Revised: coefficients are invariant to rotations. Since the other inputs | d st | , a sand a t to F m are also invariant to rotations, F m is invariant to rotations if only m = 0 coefficientsare used.

Original: The resultingmessages m st are equivariant to rotations once they are rotated back to the global coordinate frameusing D − 1 st .
Revised: The resulting messages m st are equivariant to rotations once they are rotated back to theglobal coordinate frame using A − 1 st .

Original: While equivariance is a desirable property, as we demonstrate later, using m ∈ [ − 1 , 1]coefficients can help improve performance even though equivariance is violated.If we choose to use coefficients beyond just m = 0 for F e , two approaches may be taken. The first isto simply add the coefficients to the inputs and outputs of F e and assume the neural network will learna roughly equivariant mapping through the random sampling of the roll rotation. As we demonstrate in Appendix A, the learned functions are indeed roughly equivariant.
Revised: While equivariance is a desirable property, as we demonstratelater, using m ̸ = 0 coefficients can help improve performance even though equivariance is violated.

Original: The second strategy takes amore direct approach to encouraging the network to learn equivariant mappings by sampling themessages at multiple rotations or “taps” about the z-axis.

Original: See Appendix D for details.

Original: The number of coefficients used during message passing when calculating F e may be varied basedon the edge properties to reduce the memory required by the network.
Revised: The number of coefficients used during message passing when calculating F m may be varied basedon the edge properties to reduce the memory required by the network.

Original: For instance, atoms that arefar away from each other may not need the same resolution of the spherical functions as those closetogether, so a lower maximum degree L may be used.
Revised: For instance, atoms that arefar away from each other may not need the same resolution of the spherical functions as those closetogether, so a lower maximum degree l may be used.

Original: By training several message passing networkswith varying degrees for different ranges of edge distances, memory usage can be reduced while notimpacting accuracy.
Revised: By training several message passing networkswith varying degrees for different ranges of edge distances, memory usage can be reduced while notimpacting accuracy.

Original: Similarly, the range of m can typically be reduced to [ − 1 , 1] or [ − 2 , 2] since mfor values greater than 2 (or less than -2) don’t vary as much in the direction of the z-axis, reducingtheir observed utility to the network.
Revised: Similarly, the range of m can typically be reduced to [ − 1 , 1] or [ − 2 , 2] since mfor values greater than 2 (or less than -2) don’t vary as much in the direction of the z-axis, reducing are converted from a spherical harmonic representation to a spherical grid representation using G .

Revised: The channels are concatenated and passed through a 3 layer 1 × 1 CNN.

Revised: Each layer is followed by a SiLU activation.

Revised: Finally, the channels are converted back to a spherical harmonic representation using G − 1 and added to x ( k ) t to create x ( k +1) t .

Revised: their observed utility to the network.

Original: See Figure 1(b) for examples of the spherical channels whenthe range of m is reduced.
Revised: See Figure 1(b) for examples of the spherical channels whenthe range of m is reduced.

Original: Note that while the number of coefficients used by F e may be reduced,the node embeddings maintain their original size based on the maximum L , so they may be able toaggregate information from all neighboring atoms at different distances and angles.
Revised: Note that while the number of coefficients used by F m may be reduced,the node embeddings maintain their original size based on the maximum L , so they may be able to aggregate information from all neighboring atoms at different distances and angles.

Original: 2.3 Message Aggregation
Revised: 2.3 Message Aggregation

Original: At this point in the network, the per edge messages are computed and rotated back to the globalcoordinate frame.
Revised: At this point in the network, the per edge messages are computed and rotated back to the globalcoordinate frame.

Original: Ideally, to allow for complex interactions between the messages,  a non-linearfunctionwould be applied after summing all messages directed at a target node.
Revised: To allow for complex interaction between the messages, we compute a non-linearfunction after summing all messages directed at a target node.

Original: If we did this bynaively passing the summed coefficients m ( k +1) t 
Revised: If we did this by naively passing the summed coefficients m ( k +1) t [Equation]

Original: = (cid:80) s m ( k +1)
Revised: s m [Equation] k +1)

Original: st through a fully connected neuralnetwork, the network may have difficulties learning representations that are approximately rotationequivariant (a fundamental property of atom forces).
Revised: st through a fullyconnected network, the network may have difficulties learning representations that are approximatelyrotation equivariant (a fundamental property of atom forces).

Original: Another option is to place constraints on thenon-linearities performed to enforce equivariance using Clebsch–Gordan Nets [23] or steerable MLPs[5].
Revised: Another option is to place constraintson the non-linearities allowed to enforce equivariance using Clebsch–Gordan Nets [23] or steerable MLPs [4].

Original: We propose applying an unconstrainednon-linear pointwise function on the sphere, i.e., a functionthat is applied at every orientation without knowledge of the orientation from which the point wassampled [9].
Revised: Instead, we apply an unconstrained pointwise function on the sphere, i.e., a functionthat is applied at every orientation without knowledge of the orientation from which the point wassampled [8].

Original: For our function, we use a non-linear neural network F c ( R C → R C ) that combinesinformation across channels.
Revised: For our function, we use a non-linear neural network F c ( R C → R C ) that combinesinformation across channels.

Original: In practice, F c is applied at a discrete number of orientations, whichresults in a transformation that we numerically demonstrate (see Appendix A) is approximatelyequivariant to rotations: 
Revised: In practice, F c is applied at a discrete number of orientations that resultsin a transformation that we numerically demonstrate (see supplementary material) is approximatelyequivariant to rotations: [Figure]

Original: [Equation]
Revised: [Equation]

Original: where G is a function that converts a spherical function represented by spherical harmonic coefficients,to one represented by point samples on a sphere.
Revised: where G is a function that converts a spherical function represented by spherical harmonic coefficients,to one represented by point samples on a sphere.

Original: For G we use a spherical coordinate system (polarand azimuthal) to generate a 2D discrete representation of the spherical functions, see Figure 3.
Revised: For G we use a spherical coordinate system (polarand azimuthal) to generate a 2D discrete representation of the spherical functions, see Figure 3.

Original: A1 × 1 convolutional neural network F c is applied to each discrete sample containing C channels,and the result is then converted back to the spherical harmonic representation using G − 1 .
Revised: A1 × 1 convolutional neural network F c is applied to each discrete sample containing C channels,and the result is then converted back to the spherical harmonic representation using G − 1 .

Original: This isanalogous to having a 2D image represented in the frequency domain, converting it to the spatialdomain using an inverse discrete Fourier transform, applying a transformation and converting backto the frequency domain.
Revised: This isanalogous to having a 2D image represented in the frequency domain, converting it to the spatialdomain using an inverse discrete Fourier transform, applying a transformation and converting backto the frequency domain.

Original: Since the same operation is applied to all orientations on the sphere, atransformation that is approximately equivariant (up to discrete sampling limitations) to rotationsmay be learned, see Appendix A for a more detailed discussion.
Revised: Since the same operation is applied to all orientations on the sphere, atransformation that is approximately equivariant (up to discrete sampling limitations) to rotationsmay be learned, see the supplementary material for a more detailed discussion.

Original: A diagram of the network is shown in Figure 3.
Revised: An illustration of the network is shown in Figure 3.

Original: To provide additional information to the network,x ( k ) t is also converted to a spherical grid representation, concatenated with G ( m ( k +1) t ) and providedas input to the neural network to compute the final updates to the coefficients x ( k +1) t at iteration k +1 .
Revised: To provide additional information to the network,x ( k ) t is also converted to a spherical grid representation, concatenated with G ( m ( k +1) t ) and providedas input to the neural network to compute the final updates to the coefficients x ( k +1) t at iterationk

Original: The network can learn circular functions on the sphere if varying resolutions of x ( k ) are providedto the network, similar to Difference of Gaussian filters for 2D images [27]. The resolution of x ( k )can be reduced by lowering the degree L before transforming by G . In our experiments, if multipleresolutions or bands are used, we use resolutions of degree L and L − 1 .
Revised: + 1 .

Original: Three layers of 1 × 1convolutions on 2 C channels(or 4 C if two bands are used) are performed with each followed by a
Revised: Three layers of 1 × 1 convolutions on 2 C channels are performed with each followed by a

Original: SiLU non-linear activation function.
Revised: SiLU non-linear activation function.

Original: To avoid aliasing, the spherical grid is sampled at a resolution of2 ∗ ( L + 1) .
Revised: To avoid aliasing, the spherical grid is sampled at a resolution of2 ∗ ( L + 1) .

Original: and are converted from a spherical harmonic representation to a spherical grid representation using G . The channels are concatenated and passed through a 3 layer 1 × 1 CNN. Each layer is followed by a SiLU activation. Finally, the channels are converted back to a spherical harmonic representation using G − 1 and added to x ( k ) t to create x ( k +1) t .
Revised: [Table]

Original: 2.4 Energy and Forces Estimation
Revised: 2.4 Energy and Forces Estimation

Original: We compute the energy E by first estimating a per-atom energy using a pointwise function on thesphere and taking its integral over all possible orientations.
Revised: We compute the energy E by first estimating a per-atom energy using a pointwise function on thesphere and taking its integral over all possible orientations.

Original: The per-atom energy estimates aresummed to obtain the overall system’s energy:
Revised: The per-atom energy estimates aresummed to obtain the overall system’s energy:

Original: [Equation]
Revised: [Equation]

Original:  where F energy is a three layer fully connected neural network ( R C → R 1 ) with SiLU activationfunctions.
Revised: (4) where F energy is a three layer fully connected neural network ( R C → R 1 ) with SiLU activationfunctions.

Original: We calculate forces similarly by estimating a force magnitude | f | =
Revised: We calculate forces similarly by estimating a force magnitude | f | =

Original: F force ( s ( K ) i (ˆ r )) inevery direction ˆ r over the sphere.
Revised: F force ( s ( K ) i (ˆ r )) inevery direction ˆ r over the sphere.

Original: Integration is performed after multiplying the magnitude by theorientation ˆ r to obtain directional vectors:
Revised: Integration is performed after multiplying the magnitude by theorientation ˆ r to obtain directional vectors:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: ) where F force is a three layer fully connected neural network ( R C → R 1 ).
Revised: where F force is a three layer fully connected neural network ( R C → R 1 ).

Original: In practice, a discreteapproximation of the integral is performed for equations (4) and (5) using a set of 128 evenlydistributed points on the sphere (see Appendix C.2).
Revised: In practice, a discreteapproximation of the integral is performed for equations (4) and (5) using a set of 128 evenlydistributed points on the sphere.

Original: Note that the application of a pointwise functionof an arbitrary neural network at a finite number of discrete orientations is not invariant (energies)or equivariant (forces) to rotations.
Revised: Note that the application of a pointwise function of an arbitraryneural network at a finite number of discrete orientations is not invariant (energies) or equivariant(forces) to rotations.

Original: However in practice, numerically this operation does closelyapproximate these properties (see Appendix A.2).
Revised: However in practice, numerically this operation does closely approximate theseproperties (see Supplementary B.2).

Original:  Experiments
Revised:  Experiments

Original: We present results on the Open Catalyst 2020 (OC20) dataset [6] that is released under a Creative
Revised: We present results on the Open Catalyst 2020 (OC20) dataset [5] that is released under a Creative

Original: Commons Attribution 4.0 License.
Revised: Commons Attribution 4.0 License.

Original: OC20 contains over 130M training examples for the task ofpredicting atomic energies and forces for catalysts used in renewable energy storage and otherimportant applications [44].
Revised: OC20 contains over 130M training examples for the task ofpredicting atomic energies and forces for catalysts used in renewable energy storage and otherimportant applications [44].

Original: This dataset is a popular benchmark for the ML community.
Revised: This dataset is a popular benchmark for the ML community.

Original: We begin bycomparing results across all tasks on the test set.
Revised: We beginby comparing results across all tasks on the test set.

Revised: Since several papers only report results on the

Revised: IS2RE task using approaches that directly predict relaxed energies, we also train a model specificallyfor this task and approach, and compare.

Original: Next, we show numerous ablation studies comparingmodel variations on the smaller OC20 2M dataset. 
Revised: Finally, we show numerous ablation studies comparingmodel variations on the smaller OC20 2M dataset. [Table]

Original: Finally, since several papers only report resultson the IS2RE task using approaches that directly predict relaxed energies, we also train a modelspecifically for this task and approach, and compare in Appendix B.
Revised: , which we found to empirically provide a meaningful performance difference. Training time is in GPU days. Median represents the trivial baseline of always predicting the median training energy or force across all the validation atoms. The

Revised: SCN model has L = 8 , 20 layers, C = 128 and E = 256 .

Revised: See supplementary material for SCN results trained on the OC20 All dataset and IS2RS results.

Original: 3.1 Implementation Details
Revised: 3.1 Implementation Details

Original: The implementation of spherical harmonics and their transformations uses the code provided by
Revised: The implementation of spherical harmonics and their transformations uses the code provided by

Original: Euclidean neural networks (e3nn) [13].
Revised: Euclidean neural networks (e3nn) [12].

Original: For message passing, two resolutions of spherical harmonicsare used based on a atom’s nearest neighbors, see Appendix C.1 for exact parameters.
Revised: For message passing, two resolutions of spherical harmonicsare used based on a atom’s nearest neighbors, see the supplementary material (Section A) for exactparameters.

Original: Unlessotherwise stated C = 128 , K = 16 , H = 1024 , E = 128 , and for F e in message passing only ordersm ∈[ − 1 , 1] are used.
Revised: Unless otherwise stated C = 128 , K = 16 , H = 1024 , E = 128 , and for F m inmessage passing only m ∈

Original: [-During training, the coefficients for the force and energy losses are 100 and 
 [Table] The validation results are averaged across the four OC20 Validation set splits. All SCN models are trained on 16 GPUs for 12 epochs with the learning rate reduced by 0.3 at 5, 7, 9, and 11 epochs, except SCN with L = 8 and 16 layers that used 32 GPUs to obtain a larger batch size.-]
Revised: [-During training, the coefficients for the force and energy losses are 100 and 
 [Table] The validation results are averaged across the four OC20 Validation set splits. All SCN models are trained on 16 GPUs for 12 epochs with the learning rate reduced by 0.3 at 5, 7, 9, and 11 epochs, except SCN with L = 8 and 16 layers that used 32 GPUs to obtain a larger batch size.-][ − 1 , 1] .

Original: Batch sizes vary based on the number of instances that can be fit in 32GB RAM.

Original: respectively. Training is performed using the AdamW optimizer [26] with a learning rate of 0.0004.
Revised: Training is performed using the AdamW optimizer [26] with alearning rate of 0.0004.

Original: The effective batch size is increased using data parallelism and PyTorch’s Automatic Mixed Precision(AMP).
Revised: The effective batch size is increased using data parallelism and NVIDIA’s Automatic Mixed Precision (AMP).

Original: All model code will be open sourced with an MIT license in the Open Catalyst Github repo.
Revised: All model code will be open sourced with an MIT license uponacceptance.

Original: Please see the supplementary for details on the PaiNN[33] baseline.
Revised: Please see the supplementary for details on the PaiNN[32] and NequIP[2] baselines.

Original: 3.2 OC20 Tasks
Revised: 3.2 OC20 Tasks

Original: We compare against numerous models trained on the OC20 2M, All and MD datasets across the
Revised: We compare against numerous models trained on the OC20 2M, All and MD datasets across varioustasks [5].

Original: Structure to Energy and Forces (S2EF), Initial Structure to Relaxed Structure (IS2RS) and

Original: Initial

Original: Structure to Relaxed Energy (IS2RE) tasks [6].

Original: OC20 2M contains 2 million, OC20 All has 133 million and OC20 MD has 38 million training examples.
Revised: OC20 2M contains 2M, OC20 All has 133M and OC20 MD has 38M training examples.

Original: The12 and 16 layer SCN models significantlyoutperform state-of-the-art GemNet-OC [12] on S2EF force prediction ( ≈ 14% improvement) and IS2RE energy predictions ( ≈ 10% improvement) when trained on OC20 2M, Table 1. Comparableresults are found for energy MAE on S2EF with GemNet-OC.
Revised: The 16 layer SCN models significantly outperform state-of-the-art GemNet-OC [11] on S2EF force( ≈ 11% improvement) and IS2RE energy predictions ( ≈ 10% improvement) when trained on OC202M. Comparable results are found for energy MAE on S2EF with GemNet-OC.

Original: We also report throughput efficiency.
Revised: We also reportthroughout efficiency.

Original: Note that while the SCN models process fewer samples per second, they are much more sampleefficient than GemNet as shown in Figure 2(b).
Revised: Note that while the SCN models process fewer samples per second, they aremuch more sample efficient as shown in Figure 2(b).

Original: Improved efficiency has also been shown for fullyequivariant models [3].

Original: We compare our largest SCN models trained on theAll and the All+MD datasets (133M + 38Mexamples) across all OC20 tasks in Table 2.
Revised: We compare our largest SCN models trained on the All+MD datasets (133M + 38M examples)across all OC20 tasks in Table 2.

Original: For these experiments we use a deep 1-tap model, and aslightly shallower 4-tap model since the use of 4-taps requires more memory.

Original: Our model achievesstate-of-the-art results for force MAE ( ≈ 9% improvement) and force cosine for the S2EF tasks.
Revised: Our model achieves state-of-the-art results for force MAE ( ≈ 8%improvement) and force cosine for the S2EF tasks.

Original: The

Original: SCN L = 6 model performs similarly to the recently released GemNet-OC model [12] on energy MAE for S2EF.
Revised: Only the recently released GemNet-OC model[11] performs better on energy MAE for S2EF.

Original: On the challengingrelaxation based IS2RE task that requires models to predict bothaccurate forces and energies, SCN outperforms GemNet-OC by over 7% .
Revised: On the challenging IS2RE task that requires modelsto predict both accurate forces and energies, SCN outperforms GemNet-OC by over 7% .

Original: Note that we only train asingle model to predict both energy and forces, where GemNet-OC-L-F+E + MD is the combinationof two models, one trained specifically for forces and the other for energy. 
Revised: Note thatwe only train a single model to predict both energy and forces, where GemNet-OC-L-F+E + MDis the combination of two models, one used for forces and the other for energy. Please see thesupplementary material for results trained on the All dataset and IS2RS results.

Revised: 3.3 OC20 IS2RE Direct

Revised: The task of Initial Structure to Relaxed Energy (IS2RE) may be accomplished using two approaches:1) Use an S2EF model to relax the positions of the atoms (find local energy minima) and output theenergy at the relaxed structure, or 2) directly predict the relaxed energy from the initial structure

Original: [Table]
Revised: [Table]

Revised: MAE)

Revised: and

Revised: Energy within Threshold (EwT).

Original: [Table]
Revised: The SCN model uses L = 6 withlayers and is trained for 21 epochs without performing the relaxation. Empirically, the first approach achieves higher accuracy (see Tables 1 and 2), while the second approach is more efficient both during training and inference.

Revised: Whilesolving the same problem, the second approach of direct prediction is a fundamentally differentproblem than the first.

Revised: The direct approach cannot take advantage of detailed position and angularinformation, since the initial structure only contains a rough placement of the atoms.

Revised: Instead it mustrely more on global knowledge of how the atoms are arranged.

Revised: Thus, an approach that works well forrelaxation based approaches to IS2RE might not work well for direct approaches.

Revised: As shown in Table 3, the SCN model achieves state-of-the-art results for energy MAE across alltest splits for models that do not use ancillary loses [42, 14].

Revised: Energy within Threshold (EwT) iscomparable to SEGNN [4].

Revised: Improvements for SCN are most pronounced for the out-of-domain splits,demonstrating its increased ability to generalize to unseen data.

Original: 3.3 Ablation studies
Revised: 3.4 Ablation studies

Original: We explore numerous model variations in Table 1 trained on the OC20 2M dataset and evaluatedon the OC20 validation set.
Revised: We explore numerous model variations in Table 1 trained on the OC20 2M dataset and evaluated onthe OC20 validation set.

Original: The first set of ablation experiments explores reducing the complexityof our model.
Revised: The first set of ablation experiments explores reducing the complexity of ourmodel.

Original: If messages are not rotated before and after applying F m in Equation 2, i.e ., A stis an identity matrix, the results are significantly worse for energies and even more so forforcessince forces are highly dependent on angular information.
Revised: If messages are not rotated before and after applying F m in Equation 2, i.e ., A st is an identitymatrix, the results are significantly worse for energies and even more so for forces since forces arehighly dependent on angular information.

Original: If we replace the non-linear aggregation(Equation 3) with a simple summation (No 1x1 conv) in a model with m ∈ [ − 1 , 1] , the results alsodegrade across all metrics.
Revised: If we replace the non-linear aggregation (Equation 3) witha simple summation, the results also degrade across all metrics.

Original: Results improve for m ∈ [ − 1 , 1] as compared to only using m = 0coefficients, which demonstrates that if the equivariant to rotation constraint is relaxed, improved results may be achieved.
Revised: Results improve for m ∈ [ − 1 , 1] ascompared to only using m = 0 coefficients, which demonstrates that if the equivariant to rotationconstraint is relaxed, improved results may be achieved.

Original: For higher m , m ∈ [ − 2 , 2] or m ∈ [ − 4] diminishing returns are noticedwith increased computational cost.
Revised: For higher m , m ∈ [ − 2 , 2] or m ∈ [ − 4 , 4]diminishing returns are noticed with increased computational cost.

Original: When scaling the network, increasing L has a more significant impact on force estimation, whiledepth improves energy estimation.
Revised: When scaling the network, increasing L has a more significant impact on force estimation, whiledepth improves energy estimation.

Original: This may be due to higher L resulting in greater angular resolution,which is important to force estimation.
Revised: This may be due to higher L resulting in greater angular resolution,which is important to force estimation.

Original: Greater depth, which allows information to be shared moreoften between atoms, may lead to better energy predictions since energy is function of the entireatomic structure.
Revised: Greater depth, which allows information to be shared moreoften between atoms, may lead to better energy predictions since energy is function of the entireatomic structure.

Original: Sampling multiple z-axis rotations when computing F e with 4-taps (Equations 16, 17 and 18)produces significantly improved force predictions, while using two bands ( L and L − 1 ) whenaggregating messages improves both energy and force prediction.

Original: The use of 2 bands reduces modelthroughput a small amount, while sampling with 4-taps reduces throughput by over 2 × .

Original: However,sample efficiency is further increased with 4-taps, resulting in similar training times for the sameaccuracy, Figure 2(b).

Original:  Related work
Revised:  Related work

Original: Molecular and atomic property prediction has made significant recent progress.
Revised: Molecular and atomic property prediction has made significant recent progress.

Original: Models relying onhand-crafted representations like Behler-Parrinello[4] and sGDML[8] have recently been surpassedby learned feature representations using GNNs [3, 20, 22, 33].
Revised: Models relying onhand-crafted representations like Behler-Parrinello[3] and sGDML[7] have recently been surpassedby learned feature representations using GNNs [2, 20, 22, 32].

Original: Early GNN developments focusedonly on invariant representations.
Revised: Early GNN developments focusedonly on invariant representations.

Original: CGCNN[42] and SchNet[32] make energy and force estimationsusing only atomic species and pair-wise distance information.
Revised: CGCNN[41] and SchNet[31] make energy and force estimationsusing only atomic species and pair-wise distance information.

Original: DimeNet[20, 21], SphereNet[25], and
Revised: DimeNet[20, 21], SphereNet[25], and

Original: GemNet[22, 12] extend this to explicitly capture triplet and quadruplet angles, which are scalars.
Revised: GemNet[22, 11] extend this to explicitly capture triplet and quadruplet angles, which are scalars.

Original: Utilizing invariant representations offers flexibility in model architecture design since the node or edgefeatures are inherently invariant and model constraints are not needed to maintain equivariance.
Revised: Utilizing invariant representations offers flexibility in model architecture design since the node or edgefeatures are inherently invariant and model constraints are not needed to maintain equivariance.

Original: Theenumeration of triplets and quadruplets of atoms to calculate functions of relative or dihedral anglesrequires careful model design to maintain efficiency.
Revised: Theenumeration of triplets and quadruplets of atoms to calculate functions of relative or dihedral anglesrequires careful model design to maintain efficiency.

Original: More recently, equivariant models, capturingboth scalar and equivariant features, have outperformed traditional GNNs on small molecular datasetsincluding MD17[7] and QM9[29].
Revised: More recently, equivariant models, capturingboth scalar and equivariant features, have outperformed traditional GNNs on small molecular datasetsincluding MD17[6] and QM9[28].

Original: While not strictly equivariant, our work falls into this later classof GNNs for atomic property predictions.
Revised: While not strictly equivariant, our work falls into this later classof GNNs for atomic property predictions.

Original: Models that strictly enforce equivariance to the SO(3) (3D rotations) or E(3) group (3D rigidtransformations) share similarities to our use of spherical channels [33, 1, 3, 9, 5].
Revised: Models that strictly enforce equivariance to the SO(3) (3D rotations) or E(3) group (3D rigidtransformations) share similarities to our use of spherical channels [32, 1, 2, 8, 4].

Original: Both representatoms by higher-order tensors and not just scalar information.
Revised: Both representatoms by higher-order tensors and not just scalar information.

Original: For instance, Tensor Field Networks[39], Cormorant [1], NequIP[3], SEGNN [5] and BOTNet [2]  use the same spherical harmonicrepresentations as SCNs, while PaiNN [33] is an l = 1 variation.
Revised: For instance, Tensor Field Networks[38], Cormorant [1], NequIP [2] and SEGNN [4] use the same spherical harmonic representations as SCNs, while PaiNN [32] is an l = 1 variation.

Original: All models place constraints on theoperations that can be performed to ensure equivarance, such as restricting non-linear operators toinvariant scalar inputs, e.g., pair-wise atom distances.
Revised: All models place constraints on the operations thatcan be performed to ensure equivarance, such as restricting non-linear operators to invariant scalarinputs, e.g., pair-wise atom distances.

Original: In networks like TFN, geometric informationof higher degree l are mixed into the l = 0 features via tensor products on which a gated non-linearfunction is applied.
Revised: In networks like TFN, geometric information of higher degreel are mixed into the l = 0 features via tensor products on which a gated non-linear function is applied.

Original: Similar to our approach, the recently proposed SEGNN uses the sphericalharmonic representation to enable steerable features based on edge orientations.
Revised: Similar to our approach, the recently proposed SEGNN uses the spherical harmonic representation toenable steerable features based on edge orientations.

Original: SEGNN introducessteerable MLPs that enforce equivariance by restricting the learnable parameters to those that scalethe Clebsch-Gordan tensor products followed by gated non-linearities [40].
Revised: SEGNN introduces steerable MLPs that enforceequivariance by restricting the learnable parameters to those that scale the Clebsch-Gordan tensorproducts followed by gated non-linearities [39].

Original: In our approach, weincrease the expressivity of the model by identifying a set of m = 0 coefficients across degrees l thatare invariant to rotations given an edge’s orientation to which unconstrained functions may be applied.
Revised: In our approach, we increase the expressivity of themodel by identifying a set of m = 0 coefficients across degrees l that are invariant to rotations givenan edge’s orientation to which unconstrained functions may be applied.

Original: We also demonstrate that improved performance can be achieved by relaxing the requirement forstrict message equivariance by including a broader set of coefficients beyond m = 0 .
Revised: We also demonstrate thatimproved performance can be achieved by relaxing the requirement for strict message equivarianceby including a broader set of coefficients beyond m = 0 .

Original: Furthermore,we project the spherical channels onto a grid and perform a pointwise 1 × 1 convolution followed bya non-linearity.
Revised: Furthermore, we project the sphericalchannels onto a grid and perform a pointwise 1 × 1 convolution followed by a non-linearity.

Original: Although not strictly equivariant, the pointwise operation allows for complex mixingof different degrees of spherical harmonics resulting in rich geometric descriptions.
Revised: Althoughnot strictly equivariant, the pointwise operation allows for complex mixing of different degrees ofspherical harmonics resulting in rich geometric descriptions.

Original:  Discussion
Revised:  Discussion

Original: A limitation of SCN is that it directly predicts forces, so energy conservation is not enforced.
Revised: A limitation of SCN is that it directly predicts forces, so energy conservation is not enforced.

Original: Thismay limit its use in chemistry applications such as molecular dynamics.
Revised: Thismay limit its use in chemistry applications such as molecular dynamics.

Original: Non-energy conservingmodels are still practically useful for applications such as structure relaxations and transition statesearches.
Revised: Non-energy conservingmodels are still practically useful for applications such as structure relaxations and transition statesearches.

Original: The SCN model scales O ( n 2 ) with respect to L , which may limit usingL larger than 8.
Revised: The SCN model scales O ( n 2 ) with respect to L , which may limit using larger L .

Original: Using a larger set of m coefficients during message passing also faces challenges, since the networkmay find it more difficult to learn approximately equivariant mappings.
Revised: Usinga larger set of m coefficients during message passing also faces challenges, since the network mayfind it more difficult to learn approximately equivariant mappings.

Original: In Table 1, we saw thatincreased depth improves results, especially to energy prediction.
Revised: In Table 1, we saw that increaseddepth improves results, especially to energy prediction.

Original: It remains an open question whetherdeeper networks will see further improvement.
Revised: It remains an open question whether deepernetworks will see further improvement.

Original: While this work is motivated by problems we face in addressing climate change, advances in chemistrymay have numerous use cases.
Revised: While this work is motivated by problems we face in addressing climate change, advances in chemistrymay have numerous use cases.

Original: Some of these use cases may have unintended consequences, suchas the development of the Haber-Bosch process for ammonia fertilizer production that enabled theworld to feed a growing population, but over fertilization has recently led to ocean “dead zones”and its production is very carbon intensive.
Revised: Some of these use cases may have unintended consequences, suchas the development of the Haber-Bosch process for ammonia fertilizer production that enabled theworld to feed a growing population, but over fertilization has recently led to ocean “dead zones”and its production is very carbon intensive.

Original: More alarmingly, the knowledge gained from fertilizerproduction was used to create explosives during wartime [16].
Revised: More alarmingly, the knowledge gained from fertilizerproduction was used to create explosives during wartime [16].

Original: We hope to steer research in this areatowards beneficial applications by utilizing datasets such as OC20.
Revised: We hope to steer research in this areatowards beneficial applications by utilizing datasets such as OC20.

Original: In conclusion, we propose a GNN that uses spherical harmonics to represent channel embeddingsthat contain explicit orientation information.
Revised: In conclusion, we propose a GNN that uses spherical harmonics to represent channel embeddingsthat contain explicit orientation information.

Original: While the network is encouraged to learn outputmappings that are equivariant to rotations, best performance is achieved by relaxing the equivariantconstraints and allowing for more expressive non-linear transformations.
Revised: While the network is encouraged to learn outputmappings that are equivariant to rotations, best performance is achieved by relaxing the equivariantconstraints and allowing for more expressive non-linear transformations.

Original: We demonstrate stateof-the-art results across numerous tasks on the large-scale OC20 dataset for modeling catalyst forapplications addressing climate change. 
Revised: We demonstrate stateof-the-art results across numerous tasks on the large-scale OC20 dataset for modeling catalyst forapplications addressing climate change. Checklist

Revised: For all authors...

Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?

Revised: [Yes]

Revised: (b) Did you describe the limitations of your work?

Revised: [Yes] , see section 5.

Revised: (c) Did you discuss any potential negative societal impacts of your work?

Revised: [Yes] , seesection 5.

Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Revised: [Yes]

Revised: If you are including theoretical results...

Revised: (a) Did you state the full set of assumptions of all theoretical results?

Revised: [N/A]

Revised: (b) Did you include complete proofs of all theoretical results?

Revised: [N/A]

Revised: If you ran experiments...

Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Revised: mental results (either in the supplemental material or as a URL)?

Revised: [No] , code will bereleased with MIT license upon acceptance or sooner[N/A] .

Revised: (b) Did you specify all the training details (e

Revised: g., data splits, hyperparameters,

Revised: how they

Revised: were chosen)?

Revised: [Yes]

Revised: (c) Did you report error bars (e

Revised: g., with respect to the random seed after running exper-

Revised: iments multiple times)?

Revised: [No] , experiments are too expensive to run multiple times.

Revised: However, we provide rough estimates for run variations in Table 1.

Revised: (d) Did you include the total amount of compute and the type of resources used (e

Revised: g., type

Revised: of GPUs, internal cluster, or cloud provider)?

Revised: [Yes] , training times and throughput are

Revised: proved.

Revised: If you are using existing assets (e

Revised: g., code, data, models) or curating/releasing new assets...

Revised: (a) If your work uses existing assets, did you cite the creators?

Revised: [Yes]

Revised: (b) Did you mention the license of the assets?

Revised: [Yes]

Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Revised: [No]

Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Revised: using/curating?

Revised: [N/A]

Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Revised: information or offensive content?

Revised: [N/A]

Revised: If you used crowdsourcing or conducted research with human subjects..

Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Revised: applicable?

Revised: [N/A]

Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Revised: Board (IRB) approvals, if applicable?

Revised: [N/A]

Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Revised: spent on participant compensation?

Revised: [N/A]