E FFICIENT A PPROXIMATIONS OF C OMPLETE I NTERATOMIC P OTENTIALS FOR C RYSTAL P ROPERTY P REDICTION
Anonymous authors
Paper under double-blind review
A BSTRACT
We study the problem of crystal material property prediction.
A crystal structure consists of a minimal unit cell that is repeated infinitely in 3D space.
How to accurately represent such repetitive structures in machine learning models remains unresolved.
Current methods construct graphs by establishing edges only between nearby nodes, thereby failing to faithfully capture infinite repeating patterns and distant interatomic interactions.
In this work, we propose several innovations to overcome these limitations.
First, we propose to model physics-principled interatomic potentials directly instead of only using distances as in existing methods.
These potentials include the Coulomb potential, London dispersion potential and
Pauli repulsion potential.
Second, we propose to model the complete set of potentials among all atoms, instead of only between nearby atoms as in prior methods.
This is enabled by our approximations of infinite potential summations with provable error bounds.
We further develop efficient algorithms to compute the approximations.
Finally, we propose to incorporate our computations of complete interatomic potentials into message passing neural networks for representation learning.
We perform experiments on the JARVIS and Materials Project benchmarks for evaluation.
Results show that the use of complete interatomic potentials leads to consistent performance improvements with reasonable computational costs.
I NTRODUCTION
The past decade has witnessed a surge of interests and rapid developments in machine learning for molecular analysis (Duvenaud et al., 2015).
These initial studies mainly focus on the prediction and generation problems of small molecules.
To enable computational analyses, molecules need to be featurized in an appropriate mathematical representation form.
Recently, with the advances of graph neural networks (GNNs) (Gilmer et al., 2017; Battaglia et al., 2018), molecules are more commonly represented as graphs in which each node corresponds to an atom, and each edge corresponds to a chemical bond (Stokes et al., 2020; Wang et al., 2022c).
A variety of molecular graph prediction (Stokes et al., 2020; Wang et al., 2022c) and generation (Shi et al., 2019; Jin et al., 2018; Luo et al., 2021) methods have been developed based on 2D molecular graph representations.
A key limitation of the 2D graph representations is that the 3D geometries of molecules are not captured, but such information may be critical in many molecular property prediction problems (Hu et al., 2021).
To enable the encoding of 3D molecular geometries in GNNs, a series of 3D GNN methods have been developed for prediction (Sch¨utt et al., 2017;
Gasteiger et al., 2019;
Liu et al., 2022b; Wang et al., 2022b) and generation (Liu et al., 2022a; Luo & Ji, 2022; Hoogeboom et al., 2022) problems.
In these 3D graph representations, each node is associated with the corresponding atom’s coordinate in 3D space.
Geometric information, such as distances between nodes and angles between edges, is used during message passing in GNNs.
Recently, these methods have been extended to learn representations for proteins (Jing et al., 2020; Wang et al., 2022a).
Inspired by the success of GNNs on small molecules, Xie & Grossman (2018) developed the crystal graph convolutional neural network (CGCNN) for crystal material property prediction.
Different from small molecules and proteins, crystal materials are typically modeled by a minimal unit cell (similar to a small molecule) that is repeated in 3D space with certain directions and step sizes.
In theory, the unit cell is repeated infinitely in 3D space, but any real-world material has finite size.
However, given that our modeling is at the atomic level, modeling crystal materials as infinite repetitions of unit cells is approximately accurate.
Therefore, a key challenge in crystal material modeling is how to accurately capture the infinite-range interatomic interactions resulted from the repetitions of unit cells in 3D space.
Current GNN-based crystal property prediction methods construct graphs by creating edges only between atoms within a pre-specified distance threshold (Xie & Grossman, 2018; Chen et al., 2019; Louis et al., 2020; Schmidt et al., 2021; Choudhary & DeCost, 2021).
Thus, they fail to capture interactions between distant atoms explicitly.
In this work, we propose a new graph deep learning method, PotNet , with several innovations to significantly advance the field of crystal material modeling.
First, we propose to model interatomic potentials directly as edge features in PotNet, instead of using distance as in prior methods.
These potentials include the Coulomb potential (West, 1988), London dispersion potential (Wagner &
Schreiner, 2015), and Pauli repulsion potential (Krane, 1991).
Second, a distinguishing feature of
PotNet is to model the complete set of potentials among all atoms, instead of only between nearby atoms as in prior methods.
This is enabled by our approximations of infinite potential summations with provable error bounds.
We further develop efficient algorithms to compute the approximations.
Finally, we propose to incorporate our computations of complete interatomic potentials into message passing neural networks for representation learning.
We performed comprehensive experiments on the JARVIS and Materials Project benchmarks to evaluate our methods.
Results show that the use of complete interatomic potentials in our methods leads to consistent performance improvements with reasonable computational costs.
B ACKGROUND AND R ELATED W ORK 2. C RYSTAL R EPRESENTATION AND P ROPERTY P REDICTION
A crystal structure can be represented as the periodic repetition of unit cells in the threedimensional (3D) Euclidean space, where the unit cell contains the smallest repeatable structure of a given crystal.
Specifically, let n be the number of atoms in the unit cell, a crystal can be represented as M =
( A , L ) .
Here, A = { a i } ni =
= { ( x i , p i ) } ni =1 describes one of the unit cell structures of M , where x i ∈ R b and p i ∈ R 3 denote the b -dimensional feature vector and the 3D Cartesian coordinates of the i -th atom in the unit cell, respectively.
L = [ l 1 , l 2 , l 3 ] ∈ R 3 × 3 is the lattice matrix describing how a unit cell repeats itself in the 3D space.
In the complete crystal structure, every atom in a unit cell repeats itself periodically in the 3D space.
Specifically, from an arbitrary integer vector k ∈ Z 3 and the unit cell structure A , we can always obtain another repeated unit cell structure
A k = { a k i } ni =
= { ( x k i , p k i ) } ni =1 , where x k i
=
x
i , p k i = p i + Lk .
Hence, the complete crystal structure (cid:101) A of M with all unit cells can be described as
[Equation]
In this work, we study the problem of crystal property prediction.
Our objective is to learn a property prediction model f : M → y ∈ R that can predict the property y of the given crystal structure M .
We will focus on predicting the total energy, or other energy-related properties of crystals.
2. C RYSTAL P ROPERTY P REDICTION WITH I NTERATOMIC P OTENTIALS
Most of the classical crystal energy prediction methods are based on interatomic potentials.
According to the studies in physics (West, 1988; Daw et al., 1993; Brown, 2016), the total energy of a crystal structure can be approximated by the summation of interatomic potentials in the crystal.
Particularly, the three following categories of interatomic potentials are widely used in crystals, and they are considered as sufficient for accurate energy approximation.
• Coulomb potential , which is caused by the electrostatic interaction of two atoms with charges.
Coulomb potentials are closely related to ionic bonding and metallic bonding in crystals (West, 1988).
For any two atoms a and b , let z a and z b denote the number of charges in the atom a
and b , and let d ( a , b ) be the Euclidean distance between the atom a and b .
The Coulomb potential
V Coulomb is defined as V Coulomb ( a , b ) =
− z a z b e 2 4 πϵ 0 d
( a , b ) .
Here e is the elementary charge constant, and ϵ 0 is the permittivity constant of free space.
• London dispersion potential , which describes the Van der Waals interaction between atoms.
London dispersion potential is often considered in energy estimation because its contribution is cumulative over the volume of crystals (Wagner & Schreiner, 2015) and sometimes very strong in bulk crystals, such as the crystals containing sulfur and phosphorus.
The mathematical form of this potential can be described as V London ( a , b ) = − ϵ/d 6 ( a , b ) , where ϵ is a hyperparameter.
• Pauli repulsion potential , which results from the generally existing Pauli exclusion principle in all crystal structures.
The Pauli exclusion principle forces any two atoms to be sufficiently far away from each other so that the electron orbits of them do not overlap.
Such exclusion interactions lead to Pauli repulsion potential with the form of V Pauli ( a , b ) =
e − αd ( a , b ) , where α is a hyperparameter (Buckingham, 1938; Slater, 1928).
2. C RYSTAL P ROPERTY P REDICTION WITH D EEP L EARNING
While physics-based methods have been used for predicting the crystal energy for a long time, these methods are usually crystal-specific, i.e., one method can only achieve accurate approximation for one certain type of crystals.
Recently, thanks to the advances of deep learning, many studies have been done to develop a general crystal property predictor for a variety of different crystals with powerful deep neural network models.
Some studies (Wang et al., 2021; Jha et al., 2018; 2019; Goodall & Lee, 2020) represent crystals as chemical formulas, and adopt sequence models to predict properties from these string representations.
However, more recent studies consider crystals as 3D graphs and employ expressive 3D GNNs (Sch¨utt et al., 2017; Klicpera et al., 2020b;
Liu et al., 2022b), a family of deep neural networks specifically designed for 3D graph-structured data, to crystal representation learning.
CGCNN (Xie & Grossman, 2018) is the first method that proposes to represent crystals with radius graphs and adopts a graph convolutional network to predict the property from the graph.
Based on the pioneering exploration of CGCNN, a lot of subsequent studies (Schmidt et al., 2021; Louis et al., 2020; Chen et al., 2019; Choudhary & DeCost, 2021; Batzner et al., 2022) propose various 3D GNN architectures to achieve more effective crystal representation learning.
Particularly, by enhancing the input features with angle information, ALIGNN Choudhary & DeCost (2021) develops the currently most powerful 3D GNN architecture for crystals and achieves the best crystal property prediction performance.
M ETHOD
Although existing GNN-based methods have achieved impressive performance in crystal property prediction, they struggle in further boosting the performance due to the approximation of interatomic interactions using functional expansions based on distances and failure in capturing complete interatomic interactions.
In this section, we present PotNet, a novel crystal representation model that can overcome these limitations of prior methods.
Based on the physical modeling of crystal energy, PotNet explicitly uses infinite potential summations as input features to capture complete interatomic interactions.
The infinite potential summations are incorporated into the message passing mechanism of graph neural networks and efficiently approximated by a fast algorithm.
To the best of our knowledge, PotNet is the first work that bridges the classical crystal energy computation methods based on potentials and the data-driven methods based on deep neural networks.
3. A PPROXIMATING C RYSTAL E NERGY WITH C OMPLETE I NTERATOMIC P OTENTIALS
According to the density functional theory (DFT) in physics, for any crystal M =
( A , L ) with the complete structure (cid:101)
A defined in Eqn.
(1), its total energy E ( M ) can be accurately approximated by the embedded atom method (Daw & Baskes, 1984;
Daw et al., 1993; Baskes, 1987; Lee et al., 2016;
Riffe et al., 2018) in the form of
[Equation]
where V ( a , b ) denotes the interatomic potentials between the atoms a and b , capturing the magnitude of interactions; ρ a is the local electron density of the atom a , determined by the coordinate and number of charges of the atom a according to the Hohenberg-Kohn theorem; F ( · ) is a parametrized function to embed the electron density ρ a .
Actually, existing studies (Jalkanen & M¨user, 2015) show that ρ a can be considered as a function of (cid:80) b ̸
= a , b ∈ (cid:101)
A V ( a , b ) mathematically.
Hence, the
Eqn.
(2) can be rewritten in the following form as
[Equation]
where G ( · ) is a parametrized function.
Eqn.
(3) can be considered as a way to compute the energy from the complete interatomic potential summation (cid:80) b ̸
= a , b ∈ (cid:101)
A V ( a , b ) of every atom a in the unit cell A .
However, in practice, the function G is computationally expensive if not infeasible.
Hence, more and more studies have turned to the powerful learning capability of modern deep neural network models to approximate it effectively.
3. L IMITATIONS OF E XISTING D EEP L EARNING M ETHODS
Currently, most of the existing graph deep learning methods (Xie & Grossman, 2018; Chen et al., 2019; Louis et al., 2020; Choudhary & DeCost, 2021) for crystals takes the radius graph representation and the distance-based features as inputs to predict the crystal energy in Eqn.
Specifically, for a crystal M =
( A , L ) , the radius graph is constructed by adding edges between any atom a in the unit cell A and any other atom b in the complete crystal structure (cid:101)
A whose distances are smaller than a pre-specified distance threshold r .
In addition, some functional expansions of distances, e.g., radial basis functions (RBF), are used to model interatomic interactions and form the input edge features to the 3D GNN model.
Hence, let a = ( x a , p a ) , b = ( x b , p b ) , the crystal energy prediction ˆ E ( M ) of these methods can be generally described as
[Equation]
where N r ( a ) = { b : b ̸ = a , b ∈ (cid:101) A ,
|| p a −
p b
|| 2 < r } , ϕ ( · ) denotes the functional expansions, and
H ( · ) is a non-linear function based on 3D GNN models.
However, we argue that predicting or approximating the energy with Eqn.
(4) is a suboptimal solution.
Actually, compared with Eqn.
(3) that is physics-principled, there exist non-negligible approximation errors in Eqn.
First, Eqn.
(4) captures the interatomic interactions based on interatomic distances, while the energy can be more accurately approximated by a function of interatomic potentials as in Eqn.
Though according to Sec. 2.2, interatomic potentials themselves are also functions of distances, we argue that directly using functional expansions of distances is not the best solution to crystal energy prediction.
The commonly used functional expansions in existing methods, such as RBF ϕ ( · ) , have different mathematical forms from potentials defined in Sec. 2.2.
Intuitively, this will pose more challenges and burdens to 3D GNN models since they need to learn a mapping from ϕ ( · ) to the energy E while the energy E is not the direct function of ϕ ( · ) .
Hence, we argue that directly employing the physics-principled potential functions instead of ϕ ( · ) as input features is more suitable for crystal energy prediction.
Second, different from Eqn.
(3), Eqn.
(4) does not capture the complete set of interatomic interactions because the summation set N r ( a ) of atoms b is constrained to be the atoms whose distances to the atom a are smaller than r .
This can lead to a significant approximation error due to ignoring long-range interatomic potentials, i.e., interatomic potentials between distant atoms.
Different from molecules with finite structures, long-range interatomic potentials cannot be ignored for crystals with infinite structures.
By the first principles in physics, interatomic potentials decay algebraically when pairwise interatomic distances become larger.
Hence, for a finite structure like molecules, the potentials from atoms that are far away from a given atom is limited and can be ignored.
However, long-range interatomic potentials have a significant influence on a given atom in the infinite crystal structure.
Taking Coulomb potentials as an example, assuming that we are given a 1D crystal structure where there is only one atom repeating itself with Euclidean distance of 1, each atom has only one unit of charge, and the total energy is simply the sum of all interatomic
Coulomb potentials.
As defined in Sec. 2.2, Coulomb potential energy V ( a , b ) between atoms a and b satisfies V ( a , b ) ∝ 1 /d , where d is the distance between atoms a and b .
Considering the
Coulomb potentials between a given atom and all other atoms, the total potential (cid:101) V of them satisfies (cid:101) V ∝ (cid:80) ∞ n =1 1 /n .
If only the pairwise atoms within the distance threshold r are considered, an infinite error of energy calculation is introduced.
Specifically, the smallest possible prediction error ∆ V satisfies ∆ V ∝ (cid:80) ∞ n = ⌊ r +1 ⌋ 1 /n , which is infinite.
In other words, ignoring interatomic
Coulomb potentials between atoms with distances larger than r will cause a significant prediction error of the total energy.
From this example, we can easily find that the failure to capture complete interatomic potentials due to using radius graphs is a key factor that prevents accurate energy prediction in existing GNN-based methods.
3. M ESSAGE P ASSING WITH C OMPLETE I NTERATOMIC P OTENTIALS
It follows from the analysis in Sec.
3.2 that the major limitations of existing deep learning methods for crystal representation learning lie in (1) not making the prediction from physics-principled interatomic potentials, and (2) not considering complete interatomic interactions.
To overcome these limitations, we propose to explicitly use complete interatomic potential summations in GNN models.
Since our proposed method is tightly related to potentials, we name it PotNet.
By reformulating Eqn.
(3), our PotNet incorporates the crystal energy computation with complete interatomic potentials into the message passing scheme of GNN models.
For any material structure
M =
( A , L ) , we can rewrite the definition of its complete structure (cid:101) A in Eqn.
(1) as
[Equation]
where A b = (cid:83)
k ∈
Z 3 { b k } denotes the set of atoms containing the atom b from the unit cell A and all its periodically repeated duplicates in the complete crystal structure.
With Eqn.
(5), we can reformulate Eqn.
(3) as
[Equation]
where the infinite potential summation S ( a , b )
= (cid:80) c
̸
= a , c ∈
A b V ( a , c ) denotes the sum of the interatomic potentials from the atom b together with its all periodic duplicates to the atom a .
Eqn.
(6) can be integrated into the message passing scheme of GNN models.
Specifically, we can create a graph
G for M =
( A , L ) , where each atom in the unit cell
A corresponds to a node in the graph.
For any two nodes u , v in the graph, there is an edge connecting them, and every node u in the graph is also connected to itself by a self-loop edge.
If we consider the infinite potential summation S ( a , b ) as the feature of the edge from the node b to a , we can use the message passing based non-linear neural network model in GNN to fit the function 12 (cid:80) b ∈ A S [Equation]
a , b ) + G [Equation] b ∈ A S [Equation]
a [Equation] (cid:1) . Based on this design of directly using interatomic potentials as edge features, our PotNet employs a GNN model with multiple message passing layers on the graph G to predict the crystal energy of
M .
The computational process of the ℓ -th message passing layer for the node a can be described as
[Equation]
where h ( ℓ ) a denotes the node embedding vector of the node a outputted from the ℓ -th message passing layer, h (0) a is initialized to the atom feature vector of the atom a , and g φ ( · ) , f θ ( · ) are both neural network models with the trainable parameters φ and θ , respectively.
Here, the model f θ plays the role of capturing information from both atomic features and complete interatomic potentials.
Detailed information about model architectures of f θ and g φ is provided in Appendix D.1. Note that our PotNet is actually a 3D GNN model even though 3D geometric information is not explicitly involved in Eqn.
This is because the edge feature S ( a , b ) is related to potential functions, and by Sec.
2.2 we know that they are computed from interatomic distances.
In other words, PotNet
⋯ ⋯ ⋯ ⋯ � 𝒘𝒘∈𝑨𝑨 𝒖𝒖 𝑉𝑉 ( 𝒘𝒘 , 𝒗𝒗 ) (a) (b) (c) ⋯ ⋯ ⋯ ⋯ 𝒖𝒖 𝒗𝒗 � 𝒘𝒘∈𝑨𝑨 𝒗𝒗 𝑉𝑉 ( 𝒘𝒘 , 𝒖𝒖 ) � 𝒘𝒘∈𝑨𝑨 𝒖𝒖 , 𝒘𝒘≠𝒖𝒖 𝑉𝑉 𝒘𝒘 , 𝒖𝒖 � 𝒘𝒘∈𝑨𝑨 𝒗𝒗 , 𝒘𝒘≠𝒗𝒗 𝑉𝑉 ( 𝒘𝒘 , 𝒗𝒗 ) 𝑆𝑆 𝒖𝒖 , 𝒖𝒖 𝑆𝑆 ( 𝒗𝒗 , 𝒗𝒗 ) 𝑆𝑆 ( 𝒖𝒖 , 𝒗𝒗 ) 𝑆𝑆 ( 𝒗𝒗 , 𝒖𝒖 ) 𝒖𝒖 𝒗𝒗 𝒖𝒖 𝒗𝒗 can be grouped into compute can be considered to encode 3D geometric information with potential functions, though our direct motivation of using potential functions comes from the physical modeling of crystal energy.
Intuitively, the message passing process in Eqn.
(7) over the graph G can be considered as a general case of employing a radius graph where the distance threshold r goes to infinity, i.e., r → + ∞ .
In this case, as shown in Fig. 1(a), for any atom in the crystal, all the other atoms in the complete crystal structure will be included to interact with it.
If we follow the radius graph construction process in the previous methods (Xie & Grossman, 2018; Chen et al., 2019; Louis et al., 2020; Choudhary &
DeCost, 2021), we will obtain a graph (cid:101) G in which there exist an infinite number of edges between every pair of nodes.
However, PotNet simplifies this complicated graph (cid:101) G to the graph G in which only one edge exists between every node pair.
Specifically, PotNet directly models interatomic interactions as potentials and for any two nodes in (cid:101) G , PotNet aggregates all edges between them to a single edge by the use of infinite potential summation S ( a , b ) (see Fig. 1(b)).
In other words,
PotNet provides an effective solution that enables GNN models to capture complete interatomic interactions through the use of infinite potential summations.
For our implementation of infinite potential summations S ( a , b ) in Eqn.
(7), we add up the infinite summations of three potentials, including the Coulomb potentials, London dispersion potentials, and
Pauli repulsion potentials described in Sec.
We simplify the mathematical form of Coulomb potentials V Coulomb ( a , b ) =
− z a z b e 2 4 πϵ 0 d
( a , b ) to V Coulomb ( a , b ) = − ϵ 1 /d ( a , b ) where ϵ 1 is a hyperparameter, because e, π, ϵ 0 are all known constants, and z a , z b can be learned from atomic features.
The detailed hyperparameter settings of London dispersion potentials V London ( a , b )
= − ϵ/d 6 ( a , b ) and Pauli dispersion potentials V Pauli =
e − αd ( a , b ) can be found in Appendix D.1.
3. E FFICIENT C OMPUTATION OF I NFINITE P OTENTIAL S UMMATION
Although we have effectively incorporated infinite potential summations into the message passing based GNN models, the computation of these infinite potential summations is not easy.
Basically, there are two challenges to achieve accurate and efficient computation of the infinite potential summations.
For accuracy, the computation algorithm needs to have provable error bounds.
For efficiency, it needs to be fast for scalable GNN training and fast crystal property prediction.
To tackle these two challenges, we derive a fast approximation algorithm for infinite potential summations based on the Ewald summation method (Ewald, 1921).
Specifically, we unify the summations of three infinite potentials between atom a and all duplicated positions of atom b into an integral form such that the Ewald summation method can be applied for efficient implementation in PotNet (Fig. 1(c)). The core idea of Ewald summation is that a slowly converging summation in real space is guaranteed to be converted into a quickly converging summation in Fourier space (Woodward, 2014). Based on that, the Ewald summation method divides a summation into two parts. One part has a quicker converging rate in real space than the original summation.
The other ”slower- [Table] to-converge” part is transformed into the Fourier space and becomes quickly convergent. In our method, the Ewald summation method is used with the infinite summations by dividing the integral into two parts, including one part that converges fast in Fourier space and another part that converges fast in real space, to obtain fast approximations with provable error bounds.
To apply the fast approximation algorithm of infinite summations proposed by Ewald (1921) [Table] a unified integral view of infinite potential summations is needed.
Following notations in Sec.and 3.3, we denote the positions of the atoms in the set
A b as P b = { p kb | p kb = p b + Lk , k ∈ Z} .
The Euclidean distances between the atom a and all atoms in A b can be represented as { d | d = ∥
p b + Lk − p a ∥ , k ∈ Z 3 } .
As described in Sec.
3.3, we simplify the Coulomb potential function as V Coulomb ( a , b ) = − ϵ 1 /d ( a , b ) , where ϵ 1 is a hyperparameter scaling the potential.
Based on that, we can represent Coulomb potentials from all atoms in A b to the atom a as {− ϵd | d = ∥
p b + Lk − p a ∥ , k ∈ Z 3 } .
Similarly, London dispersion potentials from all atoms in A b to the atom a can be represented as {− ϵ ′
d 6 | d = ∥ p b + Lk − p a ∥ , k ∈ Z 3 } .
It is worth noting that Coulomb potentials and London dispersion potentials can be represented in a unified view as { constant d p | d = ∥ v ab + Lk ∥ , v ab = p b − p a , k ∈ Z 3 } , where p is a positive number.
And we represent
Pauli potentials from all positions of atom j to atom i as { e − αd | d = ∥
p b + Lk − p a ∥ , k ∈ Z 3 } .
We provide detailed proofs in
Appendix C.1 that the summations of these three potentials can be unified in an integral form as
[Equation]
where A, B, C, D are constants derived from the corresponding specific potential forms.
We then apply the Ewald summation method (Ewald, 1921) to Eqn.
(8) and split it into two parts:
[Equation]
[Equation] where S direct denotes the part that converges fast in real space, and S Fourier denotes the other part that quickly converges in Fourier space when the total summation converges as shown by Ewald (1921).
Based on this, we further prove that S direct and S Fourier can be deduced into summations of incomplete Bessel functions K ν ( x, y ) in Appendix C.2 and the approximation error can be analyzed and proved to be bounded.
Note that for London dispersion potentials and Pauli potentials, the transformed summations of incomplete Bessel functions can be directly approximated.
However, for Coulomb potentials, the direct summation diverges.
Concretely, when p > 3 for potentials of form { constant d p | d = ∥ v ab + Lk ∥ , v ab = p b − p a , k ∈ Z 3 } , the corresponding potential summation converges, and when p ≤ 3 , the corresponding potential summation diverges.
To tackle this problem, we follow previous mathematical derivations (Harris, 2008; Slevinsky & Safouhi, 2010; Jones, 2007) and use the analytically continued incomplete Bessel functions to approximate infinite summations of Coulomb potentials as shown in Appendix C.3.
We then provide detailed mathematical proofs that the summations of incomplete Bessel functions are convergent and can be approximated with an error bounded by the Gaussian Lattice Sum in Appendix B.2.
Detailed implementation of the [Table] proposed summation algorithm can be found in Appendix C.4.
It is worth noting that PotNet is the first method to use the incomplete Bessel function to compute the potential summations and is able to compute Pauli potential summation, while previous methods (Crandall, 1998; Lee & Cai, 2009;
Nestler et al., 2015) cannot achieve it.
Also, we are able to compute other interatomic potential summations including Lennard-Jones potential, Morse potential
[Table]
and screened Coulomb potential by using our method as shown in Appendix C.
[Table]  E XPERIMENTAL S TUDIES 4. E XPERIMENTAL S ETUP
We conduct experiments on two material benchmark datasets, including The Materials Project and
JARVIS.
Baseline methods include CFID (Choudhary et al., 2018), SchNet (Sch ¨ utt et al., 2017),
CGCNN (Xie & Grossman, 2018), MEGNET (Chen et al., 2019), GATGNN (Louis et al., 2020), and ALIGNN (Choudhary & DeCost, 2021).
Unless otherwise specified, we report the results reported by referred papers or provided by original authors.
All PotNet models are trained using the
Adam (Kingma & Ba, 2014) optimizer with weight decay (Loshchilov & Hutter, 2017) and one cycle learning rate scheduler (Smith & Topin, 2019) with a learning rate of 0.001, training epoch of 500, and batch size of 64.
We use Pytorch to implement our models.
For all tasks on two benchmark datasets, we use one NVIDIA RTX A6000 48G GPU for computing.
Other detailed configurations of PotNet for different tasks are provided in Appendix D.1.
To capture the global infinite-range interactions without losing details of local interactions, PotNet uses both local and infinite crystal graphs.
Specifically, for the local crystal graph, we use the radius crystal graph proposed by CGCNN but replace Euclidean distances with interatomic potentials for edge features.
To be concrete, because the influences of London dispersion potentials and Pauli potentials are limited and can be ignored when only considering nearby regions, we only use the
Coulomb potentials in the radius crystal graph.
The infinite crystal graph is constructed as described in Sec.
3.3, where Coulomb potentials, London dispersion potentials and Pauli potentials are used as edge features.
4. E XPERIMENTAL R ESULTS
The Materials Project .
We first evaluate PotNet on The Materials Project-2018.6.1, which is a widely used large-scale material benchmark with 69239 crystals.
We follow previous works (Xie &
Grossman, 2018; Chen et al., 2019; Choudhary & DeCost, 2021; Louis et al., 2020) and use four crystal properties including formation energy, band gap, bulk moduli, and shear moduli.
We notice that previous works (Xie & Grossman, 2018; Chen et al., 2019; Choudhary & DeCost, 2021; Louis et al., 2020; Sch ¨ utt
et al., 2017) compare with each other using different versions of splitting training, evaluation, and testing datasets with different random seeds.
For instance, the original CGCNN paper only uses 28046 training samples for formation energy prediction, resulting in the original result of 0.039 as shown in Table 1.
To make the comparisons fair, we follow the settings of the previous state-of-the-art (SOTA)
ALIGNN (Choudhary & DeCost, 2021) for tasks including formation energy and band gap prediction, and retrain all other baselines using the same dataset setting for these two tasks.
For bulk and shear moduli, we follow the dataset setting of GATGNN (Louis et al., 2020) which has the best prediction performances for these two tasks and retrained all other baselines.
We use test mean-absolute-error (MAE) as the evaluation metric following previous studies.
Detailed configurations are also shown in Appendix D.2. We present our results in Table 1, where PotNet consistently outperforms other SOTA methods on all four tasks by large margins.
JARVIS Dataset .
We then evaluate PotNet on JARVIS, a newly released benchmark dataset proposed by Choudhary et al.
(2020) with 55722 crystals. We evaluate PotNet on five crystal property prediction tasks including formation energy, bandgap (OPT), bandgap (MBJ), total energy, and Ehull.
We follow ALIGNN and use the same training, validation, and test sets for these tasks.
Since there are missing results for baseline methods, we retrain corresponding baseline methods following the same settings with ALIGNN as discussed in Appendix D.2..
As shown in Table 2, PotNet achieves the best performances on all five tasks consistently by significant margins.
[Table]
Efficiency of PotNet .
Beyond the superior modeling capacity for crystals, our PotNet is faster and more efficient than ALIGNN.
To demonstrate the efficiency of PotNet, we compare PotNet with ALIGNN in terms of training time per epoch, total training time, and inference time for the task of JARVIS formation energy prediction.
From Table 3, PotNet is four times faster in terms of total training time and inference time compared with ALIGNN. [Table]
We also analyze the time cost of infinite summations as shown in Table 4. We calculate the infinite summations during the preprocessing.
Unlike previous methods like ALIGNN, we need to calculate the complete potential set besides constructing graphs.
Therefore, we cannot avoid spending more time computing. However, for a single material, the preprocessing time of our method is at the level of milliseconds. Even if we additionally compute the infinite potential summations, the preprocessing time of our method and ALIGNN is still within the same order of magnitude. If considering both preprocessing time and model time for single material screening, we have a faster inference speed than ALIGNN.
Overall, the computational cost of our method is reasonable.
4. A BLATION S TUDIES
In this section, we demonstrate the importance of two core components of PotNet, including interaction modeling using potentials and infinite summation of potentials for crystal prediction.
We conduct experiments on the JARVIS formation energy task, and use test MAE as the evaluation
[Table]
Interaction Modeling using Potentials .
We demonstrate the importance of interaction modeling using potentials by replacing potentials with Euclidean distances used by previous works in our PotNet with exactly the same model architecture.
Specifically, we denote
PotNet with only local crystal graph as the base model.
We use ‘Base + Euclidean’ to represent the base model with Euclidean distances as edge features and ‘Base + Potential’ to represent the base model using Coulomb potentials as edge features.
As shown in Table 5, by replacing Euclidean distances with Coulomb potentials, PotNet without considering infinite potential summation already obtains a significant performance gain from 0.0363 to 0.0318, revealing the importance of interaction modeling using potentials in PotNet.
Infinite Summation of Potentials .
The importance of infinite summation of potentials is demonstrated by comparing the previous base models with ‘Base + Potential + Infinite’, denoting the full
PotNet model with infinite summation in infinite crystal graph.
It can be seen from Table 5 that by using infinite crystal graphs introduced in Sec. 3.3, the global repeating patterns of crystal structures are captured, resulting in a performance gain from 0.0318 to 0.0308 for formation energy prediction in JARVIS.
A G AUSSIAN L ATTICE S UM
The Gaussian Lattice Sum (B´etermin et al., 2021) computes the summation of Gaussian functions centered at the points given by a shifted lattice, formally defined as
[Equation]
where L ∈ R d × d is the lattice matrix, v ∈ R
d is a vector inside a unit cell and c ∈ R + is a prefixed constant.
One of the characteristics of the Gaussian Lattice Sum is that the term e − c | Lk
+ v | 2 rapidly decays as k becomes large, resulting fast convergence of G Ω ( L , v , c ) .
As shown by Deconinck et al.
(2004), for R ∈ R + , we have
[Equation]
where Γ( z, x ) = (cid:82) ∞ x t z − 1 e − t dt is the incomplete Gamma function and ρ = min {√ c | Lk | | k ∈
Z d , k ̸ = 0 } .
We can obtain the upper bound of the Gaussian Lattice Sum by setting R =
such that
[Equation]
B I NCOMPLETE B ESSEL F UNCTION
The incomplete Bessel function (Harris, 2008) is defined as
[Equation]
(13) where x, y ∈ R , x ≥ 0 , y ≥ 0 , and ν > 0 .
In existing studies (Harris, 2008; Slevinsky & Safouhi, 2010; Jones, 2007), K ν ( x, y ) can be analytically continued to ν ∈ R .
In this work, we follow these studies and also use the analytical continuation for our calculations, i.e., we also consider ν ∈ R .
B. F AST A PPROXIMATION OF THE I NCOMPLETE B ESSEL F UNCTION
Computing the incomplete Bessel function is extremely challenging as there does not exist any explicit closed-form solution.
In this work, we investigate a fast approximation of the incomplete
Bessel function.
Specifically, we adopt the algorithm in the work by Slevinsky & Safouhi (2010), where the incomplete Bessel function K ν ( x, y ) is approximated by the G ( m ) n transformation with the linear time complexity of O ( n ) , and n is the number of iterations.
As shown by Levin & Sidi (1981), the approximation G ( m ) n to (cid:82) ∞
f ( t ) dt is given as the solution of
[Equation]
where β i,k and G ( m ) n are unknowns, d l dx l G ( m ) n = 0 , ∀ l > 0 , σ k = min ( s k , k + 1) , and s k is the largest of s ∈ Z
such that lim x →∞ x s f ( k ) ( x ) = 0 for k = 0 , 1 , · · · , m − 1 .
Slevinsky & Safouhi (2010) proved that the incomplete Bessel function satisfies Eqn.
(14) with m = 1 , through which we can obtain
[Equation]
To eliminate all the unknowns β i,k , Slevinsky & Safouhi (2010) applied the ( x 2 ddx ) operator n times such that
[Equation]
By doing this, we obtain
[Equation]
in which we can have
[Equation]
with
[Equation]
[Equation]
This leads to a recursive algorithm to approximate G (1) n .
To compute the incomplete Bessel function
K ν ( x, y ) , Slevinsky & Safouhi (2010) investigated the following property
[Equation]
in which the term (cid:82) ∞ 0 t − ν − 1 e − t − xyt dt can be approximated by G (1) n .
Therefore [Equation] to approximate
K ν ( x, y ) we have
[Equation]
[Equation] As a result, we obtain the approximation ˜ G (1) n to K ν
( x, y ) by recursively solving ˜ N n ( x ) and
D n ( x ) (Gaudreau et al., 2012).
The detailed expressions of ˜ N n ( x ) and D n ( x ) are given in Slevinsky & Safouhi (2010).
In addition, we follow Nestler et al.
(2015) to further optimize the approximation of the incomplete Bessel function when ν = 0 and x, y are both small, e.g. x 2 + y 2 < 1 .
In this case, the remaining part of the Taylor expansion of K 0 ( x, y ) is small and we can approximate K 0 ( x, y ) by the first m terms of Taylor series such that
[Equation]
The detailed error bound by this expansion is shown in Nestler et al.
B. C ONVERGENCE OF I NCOMPLETE B ESSEL F UNCTION S UMMATION
We define the summation of incomplete Bessel functions on a lattice as
[Equation]
where ν, α, β, γ ∈ R are constants, α > 0 , β ≥ 0 , γ ≥ 0 , v ∈ R
d is a vector inside a unit cell and
L ∈ R d × d is the lattice matrix.
We aim to prove the summation of incomplete Bessel functions is convergent and can be approximated with an error bounded by the Gaussian Lattice Sum introduced in Appendix A.
Proof.
The incomplete Bessel function has an upper bound such that
[Equation]
where x > 0 and Γ is the incomplete Gamma function described in Appendix A. Based on this
[Equation]
we obtain
[Equation]
where α | Lk + v | 2 + γ > 0 .
As shown by Borwein & Chan (2009) [Equation] | Γ( z,x ) x z | has an upper bound such that
[Equation]
where z ∈ R , x ∈ R and x > 0 .
For a prefixed value R ∈ R , R 2 > − ν , R 2 > 1 and R 2 > γ , we have
[Equation]
where G Ω ( L , v , α ) is the Gaussian Lattice Sum as described in Appendix A.
Therefore, the incomplete Bessel function summation can be divided into two parts such that
[Equation]
where the first part is a finite part inside an ellipsoid with a size of (cid:112) R 2 /α − γ/α , and the second part ϵ ( R ) is bounded by the Gaussian Lattice Sum G Ω ( L , v , α ) which is convergent.
Thus, the incomplete Bessel function summation is convergent.
Consequently, to approximate the incomplete
Bessel function summation, we choose to evaluate the summation inside an ellipsoid with the size of (cid:112) R 2 /α − γ/α for a prefixed R ∈ R , such that R 2 > − ν , R 2 > 1 and R 2 > γ .
Then the error ϵ ( R ) is bounded by Gaussian Lattice Sum G Ω ( L , v , α ) .
We can further bound the error by the inequality (11) of Gaussian Lattice Sum as introduced in Appendix A such that
[Equation]
where ρ = min {√ α | Lk | | k ∈ Z d , k ̸ = 0 } .
This completes the proof.
C F AST A LGORITHM OF P OTENTIAL S UMMATION C. I NTEGRAL T RANSFORMATION
Here, we prove that the summation of the three introduced potentials can be transformed into an integral form as
[Equation]
where A, B, C, D are constants derived from the corresponding specific potential forms.
Proof.
We first prove that both the potential forms V ( L , v ) = 1 / | Lk + v | 2 p and V ( L , v ) =
e − α | Lk
+ v | can be written in the following integral form
[Equation]
For the potentials in the form of V ( L , v ) = 1 / | Lk
+ v | 2 p , we apply the Mellin transform such that
[Equation]
Thus, we obtain
[Equation]
Apparently, we can obtain A = 1 /π , B = 0 , C = p and D = 1 / Γ( p ) for the integral form of
V ( L , v ) = 1 / | Lk
+ v | 2 p .
For the potentials in the form of V ( L , v ) =
e − α | Lk
+ v | , we consider the inverse Laplace transform on e − α √ s as shown by Bateman (1954), such that
[Equation]
Therefore, we can apply the Laplace transform in Eqn.
(34) to derive the integral form of e − α | Lk
+ v |
[Equation]
Apparently, we can obtain A = 1 , B = α 2 4 π , C = − 12 and D = α 2 π for the integral form of
V ( L , v ) =
e − α | Lk
+ v | .
Finally, we conduct a summation for these two types of potentials V ( L , v ) in the space
k ∈ Z d as
S ( L , v ) = (cid:88) k ∈ Z d
V ( L , v ) .
This completes the proof.
C. C ALCULATING I NTEGRAL S UMMATION
As shown in Sec. 3.4, S ( L , v ) can be written as the summations in the Euclidean space directly and then Fourier space:
[Equation]
where S Fourier ( L , v ) = D (cid:80)
k ∈
Z d (cid:82) 1 0 t C − 1 e − Aπ |
Lk + v | 2 t − Bt dt denotes the summation in Fourier space, and S direct ( L , v ) =
D (cid:80) k
∈
Z d (cid:82) ∞ 1 t C − 1 e −
Aπ |
Lk
+ v | 2 t − B t dt denotes the summation in direct space.
Apparently, S direct ( L , v ) is already the form of the incomplete Bessel function summation.
We apply the analytic continuation to S direct ( L , v ) to expand the domain of C in S direct ( L , v ) as detailed in Appendix C.3 and we have
[Equation]
for all constant C ∈ R .
Below, we prove that S Fourier ( L , v ) can be deduced into the incomplete
Bessel function summation.
Proof.
Inspired by the Ewald summation (Ewald, 1921; Crandall, 1998), we consider S Fourier ( L , v ) on the reciprocal lattice using the Poisson summation (Crandall, 1998):
[Equation]
where w ∈ R d is a vector and w = 0 in our case, and L ′ =
L ( L T L ) − 1 is the lattice matrix for the reciprocal lattice.
As a result, We obtain
S Fourier ( L , v
[Equation]
D [Equation] We also apply the analytic continuation to S Fourier ( L , v ) in the last step.
Apparently, S Fourier ( L , v ) is deduced into the incomplete Bessel function summation, and this completes the proof.
Therefore, both S Fourier ( L , v ) and S direct ( L , v ) can be expressed by the incomplete Bessel function summation forms:
[Equation]
As shown in Appendix B.2, the incomplete Bessel function summation (cid:80) k ∈
Z d K ν ( α | Lk
+ v | 2 + γ, β ) is convergent and can be approximated.
Therefore, S ( L , v ) can also be approximated.
C. A NALYTIC C ONTINUATION OF P OTENTIAL S UMMATIONS
To represent the series that is not convergent, including the inverse summation (cid:80)
k ∈
Z d 1 | Lk + v | as shown in Sec.
3.4, we need to investigate the analytic continuation of potential summation.
As shown in Appendix C.2, the potential summation can be written as
[Equation]
These two parts can be deduced into two incomplete Bessel function summations by analytic continuation.
Analytic continuation is a technique to extend the domain P of a given analytic function f ( x ) .
Consequently, we denote ˆ f ( x ) as an analytic continuation of f ( x ) to Q , where we denote the domain Q containing P , and denote a function ˆ f ( x ) that is analytic on Q , and ˆ f ( x ) = f
( x ) holds for all x in P .
As shown by Kung & Yang (2003), the analytic continuation is unique and satisfies the permanence of functional relationships, i.e., the equations holding for f ( x ) will also hold for ˆ f ( x ) .
In our case, we can expand the domain of constant C in S ( L , v ) to C ∈ R such that
S ( L , v ) is well-defined for any C ∈ R .
For example, for summation of the potentials in the form of (cid:80) k ∈
Z d 1 / | Lk
+ v | 2 p , we have C = p as shown in Appendix C.1.
Analytic continuation enables us to compute the summation when p = 0 . 5 , which is initially divergent.
Formally, we assume that C is originally defined in the domain P ⊂ R .
As derived in Appendix C.1, for C ∈ P , we have
[Equation]
Furthermore, we have ν = − C
and ν = C − d 2 in the incomplete Bessel function, which is analytically continued to ν ∈ R (Jones, 2007).
That is, P is contained by ν ’s domain R .
Therefore, the incomplete Bessel function summations are the analytic continuation of the corresponding potential summations.
To explain the advantage of analytic continuation, we consider the infinite potential summation inside a unit cell such that
[Equation]
For a non-convergent S , analytic continuation will result in an unusual value such as negative energy instead of infinity.
This is useful in practice since the infinity will cause a numerical explosion in training.
And for a convergent series S , due to the permanence of functional relationships, analytic continuation will also produce a reasonable result.
Specifically, if we use an analytically continued function to approximate convergent summation S ( a , b )
= (cid:80) c
̸
= a , c ∈
A b V ( c , a ) , we can approximate S by deriving
[Equation]
∈ A S ( a , b ) . As for non-convergent summation S ( a , b ) = (cid:80) c ̸ = a , c ∈
A b V ( c , a ) , due to the permanence of functional relationships of analytic continuation, the equation (cid:80) a ∈ A (cid:80) b ̸ = a , b ∈ (cid:101)
A V ( a , b ) = (cid:80) a ∈ A (cid:80) b ∈
A S ( a , b ) still holds.
A famous example of using analytic continuation for crystal energy prediction is the Madelung constant of NaCl, which is derived by the summation of Coulomb potentials among Na and Cl ions and calculated by analytic continuation of an absolutely convergent series (Borwein et al., 1985).
C. I MPLEMENTATION AND N UMERICAL E XAMPLES OF A PPROXIMATION
Here we describe the implementation of our algorithm as in Eqn.
Considering the fact that solving the inverse of the incomplete Gamma function is complicated, instead, we provide a proper value R and then calculate its corresponding error bound ϵ based on Eqn.
Given a lattice matrix L ∈ R d × d , a vector v ∈ R d inside a unit cell, and the constants A, B, C, D derived from specific potential functions as described in Appendix C.1, we aim to evaluate these two parts
D (cid:80) k
∈
Z d K − C
( Aπ | Lk
+ v | 2 , B ) and
[Equation]
To evaluate S direct ( L , v )
=
D (cid:80) k
∈
Z d K − C
( Aπ | Lk
+ v | 2 , B ) , we derive the following steps.
Step 1: Determine the value R such that R 2 > C , R 2 > 1 , and calculate the error bound ϵ
=
d 2 ( 2 ρ ) d Γ( d 2 , ( R − ρ 2 ) 2 ) , where ρ = min {√ Aπ | Lk | | k ∈ Z d , k ̸ = 0 } .
Step 2: Select points inside an ellipsoid such that P = { k | |
Lk + v | ≤ R/ √ Aπ } .
Step 3: Evaluate the incomplete Bessel function summation S direct ( L , v ) by calculating the every term DK − C
( Aπ | Lk
+ v | 2 , B ) based on Appendix B.1.
To evaluate S Fourier ( L , v )
=
[Equation]
(cid:80) [Equation]
K
[Equation]
[Equation] + B, 0) , we derive the following steps. Step 1: Determine the value R such that R 2 > d 2 − C , R 2 > 1 , R 2 > B , and calculate the error bound ϵ > d 2 ( 2 ρ ) d Γ( d 2 , ( √ R 2 − B − ρ 2 ) 2 ) , where ρ = min { (cid:112) πA | L ′ k | | k ∈ Z d , k ̸ = 0 } .
Step 2: Select points inside an ellipsoid such that P = { k | |
Lk + v | ≤ (cid:112)
A ( R 2 − B ) /π } .
Step 3: Evaluate the incomplete Bessel function summation S Fourier ( L , v ) by calculating every term 1 det L D
A d 2 e 2 πi L ′ k · v K C − d 2 ( π | L ′ k | 2 A + B, 0) based on Appendix B.1.
Our implementation is based on Cython and GNU Scientific Library (Galassi et al., 2002), in which the native incomplete Gamma function and Bessel function are used to implement the incomplete
Bessel function.
We conduct numerical experiments on Intel Xeon Gold 6258R CPU.
We show the evaluation examples in Table 6 with the corresponding error bound and evaluation time.
The running time is at the scale of milliseconds.
C. P OTENTIAL S UMMATION E XTENSIONS
To highlight the generality of our potential summation method, in this section, we introduce additional potentials that can be converted to our general integral form in Eqn. 31, including LennardJones potential, Morse potential, and screened Coulomb potential.
These potentials are only used for some specific types of materials, such as gas and fluid materials.
Lennard-Jones Potential (Lennard-Jones & Dent, 1928) is an intermolecular pair potential that is usually used for gas or organic materials.
The commonly used expression for the Lennard-Jones potential is
[Equation]
where ϵ and σ are hyperparameters.
And the summation of V LJ ( L , v ) can be converted to two potential summations of type 1 / |
Lk + v | 2 p with p = 3 and p = 6 , such that
[Equation]
where we show the calculation of the potential summation of type 1 / |
Lk
+ v | 2 p in Appendix C.1.
Morse Potential (Morse, 1929) is an interatomic potential of a diatomic molecule and can be used for simple molecular materials.
The Morse potential has a mathematical form of
[Equation]
can be found in Appendix C
[Table]
[Equation]
where D e and r e are hyperparameters.
Similarly, the summation of V Morse ( L , v ) can be converted to two potential summations of type e − α | Lk + v | with α = a and α = 2 a , such that
[Equation]
where we show the calculation of the potential summation of type e − α |
Lk
+ v | in Appendix C.1.
Screened Coulomb Potential or Yukawa potential (Yukawa, 1935) represents the Coulomb interactions with damping of electric fields.
It is an important potential reflecting the behaviors of chargecarrying fluids or particles in semiconductors.
The screened Coulomb potential has an analytic form of V ( r ) =
z a z b e 2 r exp( − r/a ) where r is the distance, z a , z b are charges of particle a and particle b , e is elementary charge constant and a is a scaling hyperparameter.
Since z a , z b , e are constants and can be extracted outside the summation, we can obtain the simplified screened Coulomb potential
[Equation]
Consider the inverse Laplace transform on e − α √ s / √ s , by using Bromwich contour with branch points, we obtain
[Equation]
Therefore, we can apply the Laplace transform in Eqn.
50 such that
[Equation]
Then we can obtain A = 1 /π , B
= α 2 , C = 12 and D = 1 / √ π in Eqn. 30 to fit screened Coulomb potential into our potential summation method.
D M ODEL I MPLEMENTATION D. N ETWORK S TRUCTURE
The employed network architecture is shown in Fig.
Since our major contribution is to consider infinite interatomic potentials, we simply design our network architecture following the commonly Linear Interaction Interaction
Interaction
Linear
MLP I n t e r a c t i o n B l o c k 𝒙𝒙 𝒊𝒊 𝒛𝒛 𝒊𝒊 Output AvgPooling 𝒙𝒙 𝒋𝒋 𝒆𝒆 𝒊𝒊𝒋𝒋𝒄𝒄 , 𝒔𝒔 𝒊𝒊𝒋𝒋𝒄𝒄 , 𝒔𝒔 𝒊𝒊𝒋𝒋𝒍𝒍 , 𝒔𝒔 𝒊𝒊𝒋𝒋𝒑𝒑 ‖
MLP ⊙ 𝒙𝒙 𝒊𝒊 𝒆𝒆 𝒄𝒄 , 𝒔𝒔 𝒄𝒄 , 𝒔𝒔 𝒍𝒍 , 𝒔𝒔 𝒑𝒑 ‖� 𝑗𝑗       768   ‖ 𝑎𝑎 : Concatenate along 𝑎𝑎 𝑡𝑡𝑡 dimension ⊙ : Hadamard Product ∑ 𝑗𝑗 : Aggregate over 𝑗𝑗
Embedding
BN
ReLU +
[Figure]
𝜎𝜎 𝜎𝜎 : Sigmoid + : Plus used settings.
Specifically, existing methods for 3D graphs (Xie & Grossman, 2018;
Sch ¨ utt et al., 2017; Klicpera et al., 2020b;a;
Liu et al., 2022b; Gasteiger et al., 2021; Sch¨utt et al., 2021) share a similar architecture, which usually contains an input block, an interaction block, and an output block.
Without loss of generality, we take the updating process for node i as an example to illustrate the network.
The Inputs contain atomic features and potentials.
z i is the 92-dimensional atomic feature for any atom i following CGCNN (Xie & Grossman, 2018).
Below we denote d as the interatomic distances.
As explained in Sec. 4.1, e c = − ϵ ′ 1 /d denotes the Coulomb potentials for the local crystal graph.
s c = − (cid:80) d
ϵ 1 /d, s l = − (cid:80) d ϵ/d 6 , s p
=
(cid:80) d e − αd denote the summations of Coulomb potentials,
London dispersion potentials, and Pauli potentials for the infinite crystal graph, and we set ϵ ′ 1 = 4 .for
e c , ϵ 1 = − 1 .
for s c , ϵ = 4 .[-
-]
0 for s l and α = 3 . [-
-]0 for s p , respectively.
For simplicity, we apply them to the RBF embeddings with the same cutoff.
Since e c and s l are negative values and have values exceeding the RBF cutoff, we use an exponential function to make their absolute values smaller.
The input block contains a Linear layer and an Embedding layer.
For each node i , the Linear layer is employed to generate a 256-dimensional vector as the input node features to the first interaction layer.
For each edge, the Embedding layer is employed to map the Coulomb potentials and summations of Coulomb potentials, London dispersion potentials, and Pauli repulsion potentials to 256-dimensional embeddings by using 256 RBF kernels with centers from -4.0 to 4.0.
The Interaction block contains several Interaction layers.
Each layer updates the feature vector of node i based on features of the neighboring nodes and potential embeddings of the connected edges.
Particularly, for any of the neighboring node j of node i , the corresponding potential embeddings e cij , s cij , s lij , and s pij for edge ij are all produced by the Embedding layer.
The Readout block contains an AvgPooling layer and another Linear layer.
We first use the AvgPooling layer to aggregate features from all nodes in a graph and then use the Linear layer that maps the hidden dimension of 256 to the final output which is a scalar. D. C ONFIGURATIONS OF R ETRAINED M ODELS
In this section, we show detailed configurations of retrained models of CGCNN (Xie & Grossman, 2018), SchNet (Sch¨utt et al., 2017), MEGNET (Chen et al., 2019), GATGNN (Louis et al., 2020) and ALIGNN (Choudhary & DeCost, 2021).
If not specified, models are trained with a radius cutoff of 8.0 using the Adam (Kingma & Ba, 2014) optimizer with weight decay (Loshchilov & Hutter, 2017) and one cycle learning rate scheduler (Smith & Topin, 2019).
CGCNN (Xie & Grossman, 2018).
We directly use the publicly available code from Xie & Grossman (2018) to build and train the CGCNN model.
We build the model with 128 hidden dimensions and three message-passing layers and train the model for 1000 epochs with a batch size of 256 and an initial learning rate of 1e-2.
SchNet (Sch¨utt et al., 2017).
We directly adopt the SchNet model from PyTorch Geometric (Fey &
Lenssen, 2019) with 128 hidden dimensions and six message-passing layers following the original paper.
We train SchNet with an initial learning rate of 5e-4 and batch size of 64 for 500 epochs.
GATGNN (Louis et al., 2020).
We directly use the publicly available code from Louis et al.
(2020) to build and train the GATGNN model.
We build the model with 128 hidden dimensions and three message-passing layers following the original default settings.
We train the model with an initial learning rate of 5e-3 and batch size of 64 for 500 epochs.
MEGNET (Chen et al., 2019).
We directly adopt the MEGNET model from the publicly available code from Chen et al.
Following the original paper, we use three message-passing layers with the same feature dimensions as mentioned in the original paper and use Set2Set readout function.
We train MEGNET with an initial learning rate of 1e-3 and batch size of 128 for 1000 epochs following the configuration settings mentioned in the original paper.
ALIGNN (Choudhary & DeCost, 2021).
We directly use the publicly available code from Choudhary & DeCost (2021).
We use the official best model configurations of ALIGNN to train ALIGNN models with an initial learning rate of 1e-3 and batch size of 64 for 500 epochs.
E L INEAR E NERGY M ODELING USING I NFINITE P OTENTIAL S UMMATION
𝑁𝑁𝑎𝑎
[Figure]
𝐶𝐶𝑙𝑙
[Figure]
𝑐𝑐 𝑑𝑑
In this section, we provide examples of calculations where the true energies of these materials can be directly approximated by linear combinations of our infinite potential sums.
These are special cases of Eqn.
6 with a linear embedded function G .
Specifically, we evaluate the total energy per atom of NaCl and two other materials ( MgO , LiF ) whose crystal structures are similar to NaCl .
Since they are pure ionic crystals and Coulomb interactions dominate the system, we first consider the total electrostatic energy and Coulomb potential summations.
For a neutral system, the electrostatic energy is convergent because the Coulomb potentials cancel each other.
To calculate the energy of a crystal, we cannot derive it directly due to its complexity.
Instead, we divide it into many individual infinite potential summations.
Although these individual summations might be originally divergent, they will all be well-defined under analytical continuation and the total energy will also be convergent.
As shown in Appendix C.3, analytical continuation allows us to compute potential summations that are initially divergent, and also allows us to compute the convergent summations by analytically continued potential summations.
For example, we can calculate (cid:80) n ∈ Z ,n ̸ =0 ( − 1) n n by analytic con- tinuation such that
[Equation]
where ζ ( s, a ) is Hurwitz zeta function and ζ ( s, a ) = (cid:80) ∞ n = 1 ( n + a ) s when s > 1 , a ̸ = 0 , − 1 , − 2 , ... , and its analytic continuation elsewhere.
That is, we can use the analytically continued zeta function ζ ( s, a ) to precisely approximate a convergent series.
Inspired by analytical continuation, we can approximate the total energy per atom of crystals of
NaCl by analytically continued infinite potential summations.
Due to the symmetry of the NaCl cell, we only involve atoms a, b, c, d in our calculation.
Specifically, as shown in Fig.
3, atom a represents the body center Na + , atom b represents the face center Cl − , atom c represents the edge center Na + and atom d represents the corner Cl − .
Based on this, assuming that the side length of the unit cell is 1, the total energy of Na + is approximated by the total Coulomb interactions with atom a such that
[Equation]
[Equation]
where N A is Avogadro constant, d ( a , u ) is the distance between atom a and u , r 0 is the minimum distance between Na and Cl , ˜ d
= d/r 0 is the normalized distance, A a denotes the set of atoms containing atom a and all its repetitions, ˜ S is the infinite potential summation, approximated by our infinite potential summation method in Sec.
3.4, z Na , z Cl are charges of Na +
and Cl − , e is the elementary charge constant, ϵ 0 is the permittivity constant of free space, and the coefficients 1 , 12 6 , 14 12 , 18 8 denote the fraction of atoms in a unit cell.
We finally obtain a constant − 1 .from our infinite potential summations.
In fact, this constant is exactly the famous Madelung constant M (Borwein et al., 1985).
Also, by considering an additional repulsion term, we can derive the calculation result in Eqn.
54 to the famous Born-Land´e equation (Born, 1921)
[Equation]
[Table]
where z + , z − are the charges of cation and anion, M is the Madelung constant computed from infinite Coulomb potential summations, and n is the Born exponent measuring the effect of repulsion.
By choosing n = 9 , we can derive an approximation for the total energy of NaCl of 7.84 eV. Similarly, we can apply Eqn. 55 to MgO and
LiF .
We further show these approximations in Table.
7 and it can be noticed that these approximations already give rough results compared to the ground truth energy.
This implies that our features can serve as a good starting point for machine learning models to learn the ground truth energy.
Apparently, previous methods cannot achieve this due to the lack of such informative features.
It is worth noting that the Madelung constant is typically unknown because those coefficients for the infinite potential summations depend on the charge distribution in the system, which we do not know at the beginning.
Also, these crystals are special cases of Eqn.
6 with a linear embedded function G , while G is typically a nonlinear function (Daw & Baskes, 1984).
Therefore, the network serves the purpose of learning those coefficients to learn the Madelung constant and providing nonlinearity.
F C ONCLUSION AND O UTLOOK
We study the problem of how to capture infinite-range interatomic potentials in crystal property prediction directly.
As a radical departure from prior methods that only consider nearby atoms, we develop a new GNN, PotNet, with the message passing scheme that takes efficient approximations to capture the complete set of potentials among all atoms.
Experiments show that the use of complete potentials leads to consistent performance improvements.
Altogether, our work provides a theoretically principled and practically effective framework for crystal modeling.
In the future, we expect our approximations may be further improved to obtain a lower error bound.
We also expect our algorithms for computing the summation could be further improved.