S ELF - CONSISTENT G RADIENT - LIKE E IGEN D ECOMPOSITION IN S OLVING S CHR ¨ ODINGER E QUATIONS
Anonymous authors
Paper under double-blind review
A BSTRACT
The Schr¨odinger equation is at the heart of modern quantum mechanics.
Since exact solutions of the ground state are typically intractable, standard approaches approximate
Schr¨odinger’s equation as forms of nonlinear generalized eigenvalue problems F ( V ) V = SV Λ in which F ( V ) , the matrix to be decomposed, is a function of its own top- k smallest eigenvectors V , leading to a “selfconsistency problem”.
Traditional iterative methods heavily rely on high-quality initial guesses of V generated via domain-speciﬁc heuristics methods based on quantum mechanics.
In this work, we eliminate such a need for domain-speciﬁc heuristics by presenting a novel framework, Self-consistent Gradient-like Eigen
Decomposition (SCGLED) that regards F ( V ) as a special “online data generator”, thus allows gradient-like eigendecomposition methods in streaming k -PCA to approach the self-consistency of the equation from scratch in an iterative way similar to online learning.
With several critical numerical improvements, SCGLED is robust to initial guesses, free of quantum-mechanism-based heuristics designs, and neat in implementation.
Our experiments show that it not only can simply replace traditional heuristics-based initial guess methods with large performance advantage (achieved averagely 25x more precise than the best baseline in similar wall time), but also is capable of ﬁnding highly precise solutions independently without any traditional iterative methods.
I NTRODUCTION
While the many-body Schr ¨ odinger equation governs all quantum-mechanical systems, it can be solved analytically only for extremely simple ones like an isolated hydrogen atom.
Nonetheless, a series of important approximation theories are developed so that it can be solved numerically for many relevant molecules and materials including tens to thousands of electrons, making it possible for physicists and chemists to explore the properties of matters in an ab-initio or ﬁrst-principle way, serving as a foundation of computational physics, chemistry and material science research.
Among those approximation theories, two mainstream directions involve the Hartree-Fock (Hartree, 1928; Hartree & Hartree, 1935) and density functional theories (Hohenberg & Kohn, 1964) that either approximate the Schr¨odinger equation as a “Hartree-Fock” equation or as a “Kohn-Sham equation” (Kohn & Sham, 1965).
While very different in nature, both approximated equations are ﬁnally formed as a nonlinear, generalized eigenvalue problem F ( V ) V = SV Λ in which F ( V ) and
S are N × N real symmetric matrices, Λ = diag( λ 1 , · · · , λ k ) is a k × k matrix containing the top- k smallest eigenvalues, and
V =
[ v 1 , · · · , v k ] is an N × k matrix containing the corresponding top- k eigenvectors.
Interestingly, the matrix F which we aim to decompose is explicitly deﬁned as a given function of the eigenvectors V (denoted as F ( V ) : R N × k
→ R N × N ), leading to a “causeand-effect dilemma” or “self-consistency problem”.
That is, the form of the equation is deﬁned by its ﬁnal solution, but how does one know the solution if the form of the equation is not determined?
The dominant method in solve such equations is generally referred to as Self-Consistent Fields (SCFs).
SCFs are the main component of nearly all mainstream quantum chemistry and solid-state physics software.
The general idea is to perform a ﬁxed-point iteration, that is, guess an initial solution V 0 from scratch, then solve F ( V k − 1 )
V k = SV k Λ k repeatedly to obtain eigenvectors
V k at the k th iteration, k = 1 , 2 , · · · , until (cid:107) F ( V k ) − F ( V k − 1 ) (cid:107) is less than a given convergence threshold.
Unfortunately, there is no guarantee that such an iteration can always lead to a converged solution
(Froese Fischer, 1987).
Actually, SCFs are extremely sensitive to the selection of initial guess V 0 . Without a high-quality initialisation, SCFs easily fail and oscillate between two or more different results.
Till now, all popular choices for the initial guess are heuristics-based, highly reliant on specialized prior knowledge of quantum mechanics.
For example, the SAD method (Van Lenthe et al., 2006) applies the physical intuition that molecular electron densities can be approximated via the sum of atomic electron densities.
Here, the core Hamiltonian method ignores all interactions between electrons, which simpliﬁes the equation to a standard eigen-decomposition problem.
More methods and details can be found in (Lehtola, 2019).
Besides the prerequisite of domain-speciﬁc theories, such methods are also sophisticated in implementation to achieve satisfactory convergence performance, which is a main barrier for practitioners to build high-performance solvers from scratch.
Meanwhile, in the machine learning community, researchers are facing the challenge of analyzing real-time, online data represented as streams.
In contrast to the normal case that data is fully available ofﬂine, online data is received sequentially and may even change over time, thus specialized methods are developed to dynamically adapt to new patterns in the data stream, which is called stochastic, streaming or online algorithms.
Specially, a series of works (Oja, 1982; Tang, 2019;
Allen-Zhu & Li, 2017; Gemp et al., 2021) focus on stochastic k -PCA, which aim to estimate the top- k principal components of a data stream in a real-time manner.
To handle the stochasticity of data, such stochastic k -PCA methods usually contain gradient-like eigen decomposition.
When a new data sample is available, the eigenvectors will be updated towards a gradient-like direction computed by the new sample, with a small learning rate.
In this work, we noticed that F ( V ) in the aforementioned eigen-decomposition problem is subject to concurrent update during the sequential update of V towards self-consistency: V 1 F = F ( V 1 ) −−−−−−→
V 2 F = F ( V 2 ) −−−−−−→ · · · .
In this case F ( V ) can be regarded as a special “online data generator”, which shares similarities with the online learning setting that online data is concurrently updated during the sequential learning process.
In this way, we explore the possibility of applying gradient-like eigen-decomposition in stochastic k
-PCA methods to handle the self-consistency of approximated
Schr ¨ odinger equations.
Together with several numerical improvements to enhance the smoothness of optimization, we developed Self-consistent Gradient-like Eigen Decomposition (SCGLED), an efﬁcient solver for the self-consistent approximated Schr ¨ odinger equations, SCGLED is robust to initial guesses V 0 , free of quantum-mechanism-based heuristics design, and neat in implementation.
While it can simply replace traditional heuristics-based initial guess methods with performance advantage, it is also capable of ﬁnding highly precise solutions without any traditional SCF iteration.
R ELATED WORK
The computational theories for quantum many-body systems, especially for determining the wave function of Schr¨odinger equations, has a long history starting from 1920s.
There are three mainstream theories: Hartree-Fock theory (Hartree, 1928; Hartree & Hartree, 1935), density functional theory (Hohenberg & Kohn, 1964) and quantum Monte Carlo.
With the prosperity of deep learning and differentiable optimization in recent years, there are a series of works focusing on deep-learningaided wave function representation of quantum Monte Carlo methods such as FermiNet (Pfau et al., 2020) and PauliNet (Hermann et al., 2020).
For density functional theory, there are also some works using neural networks to learn the exchange-correlation functional (Li et al., 2021; Kasim & Vinko, 2021).
However, these works focus more on improving simulation accuracy towards physical reality by using neural networks as a better functional approximator, while our work’s focus is very different, stressing on the optimization efﬁciency while the equation is completely given.
There are also works on direct optimization for Kohn-Sham equations by total energy minimization
(Yang et al., 2006), which model the problem as a constrained optimization problem instead, and more domain-speciﬁc knowledge is involved, while our work tries to solve the equation from a purely optimization-based aspect.
For gradient-like eigen-decomposition, most of the works focus on stochastic k -PCA, which estimates the top- k principal components of a data stream.
Let x ∈ R d denote a random data sample at time step t , v i ∈ R d , i = 1 , 2 , · · · , k denote the estimate of i th principal component (eigenvector of 𝜆𝜆𝐹(𝑉) 𝑉 =
𝑆 𝑉 Λ 𝑣 1 𝑣+ 𝑈 eff ( )
2𝑉𝑉 ⊤ 𝐻 ≝ 𝑣 1 𝑣𝐹(𝑉)
[Figure]
E [ XX (cid:62) ] ) at time t , corresponding to the i th largest eigenvalue. η denotes the learning rate.
Oja’s algorithm: v (cid:48)
i ← v i + η ( xx (cid:62) v i ) , ∀ i = 1 , · · · , k and ( v 1 , · · · , v k ) ← QR ( v (cid:48) 1 , · · · , v (cid:48) k )
Here QR ( v 1 , · · · , v k ) is the Gram-Schmidt decomposition that orthonormalizes v t 1 , · · · , v tk .
There are also several follow-up works (Sanger, 1989; Tang, 2019) that improve the performance, in which a most recent one is EigenGame (Gemp et al., 2021)
[Equation]
R v i and v i ← v (cid:48)
i (cid:107) v i (cid:107) , ∀ i = 1 , · · · , k.
P ROBLEM DESCRIPTION
In this paper, while what we want to solve is the approximated Schr ¨ odinger Equations, whose physical background is elaborated in Appendix A, we abstract it mathematically as the following type of nonlinear generalized eigenvalue problem
[Equation]
(1) where
• F ( V ) : an N × N real symmetric matrix to be decomposed, which is deﬁned in equationas a function of V .
• S : an N × N positive semi-deﬁnite matrix, which is a constant input in the problem.
• Λ : Λ = diag( λ 1 , · · · , λ k ) is a k × k diagonal matrix containing the top- k smallest eigenvalues.
• V : V =
[ v 1 , · · · , v k ] is an N × k matrix containing k column eigenvectors corresponding to the top- k smallest eigenvalues.
The deﬁnition of F ( V ) :
R N × k
→ R N × N is as follows:
[Equation]
in which H is an N × N real symmetric matrix, which is given in this problem.
U eff
R N × N → R N × N is a given function.
We also deﬁne P ( V )
= 2 V V (cid:62) for convenience.
To conclude, the input of the problem is S , H , U eff ( · ) and k , and the output of the problem is the eigenvectors V ∗ that obeys both equation 1 and equation 2.
The top- k smallest eigenvalues of F ( V ∗ ) are guaranteed to be negative.
To make it more clear, a toy example is provided as follows:
[Equation]
[Equation]
[Equation]
[Equation] (cid:20) 0 . 5697 0 .0 . 4441 0 .(cid:21) in which P = 2 V V (cid:62) , U eff ( P ) ∈ R 2 × 2 , E is a 2 × 2 × {+
+}
× 2 tensor.
The solution of the toy example should be V ∗ =
(0 . 5489 , 0 . 5489) (cid:62) , in this case F ( V ∗ ) =
(cid:20) − [Equation]
3655 −
[Equation]
5939 − 0 .
5939 − 0 .
3655 (cid:21) , and the result of top-1 eigen-decomposition (cid:20)
[Equation] [Equation]
− 0 .
[Equation]
0 .(cid:21)
V [Equation] (cid:20) 1 .0 .0 .1 .(cid:21)
V Λ will happen to be exactly V ∗ =
(0 . 5489 , 0 . 5489) (cid:62) with the smallest eigenvalue λ 1 = − 0 .
5782 , while the other one is [1 . 2115 , − 1 .
2115] (cid:62) with eigenvalue λ 2 = 0 .
S OLVING APPROXIMATED S CHR ¨ ODINGER EQUATIONS USING SELF - CONSISTENT GRADIENT - LIKE EIGEN DECOMPOSITION
First, notice that the stochastic k -PCA methods usually can be generalized to decompose a real symmetric matrix M .
For Oja’s algorithm (Oja & Karhunen, 1985), that is v (cid:48)
i ← v i +
[Equation]
i ∀ i
= 1 , · · · , k and ( v 1 , · · · , v k ) ← QR ( v (cid:48)
[Equation] [Equation] [Equation] (3) in which v ti is the eigenvector corresponding to the i th largest eigenvalue.
Due to the lack of efﬁciency, such decomposition methods are not favorable for traditional eigendecomposition ( M is a given constant input) compared with classical algorithms such as QR iteration.
However, as stochastic algorithms, they have a unique advantage that they can adapt to dynamic change of M , which suits the sticking point in equation 1 that F ( V ) is subject to concurrent update when V changes.
This motivates us to apply them to tackle the approximated Schr¨odinger equations.
Together with the orthogonalization technique introduced in Appendix B which transforms the generalized eigenvalue problem into a standard one, we replace M in equation 3 with F (cid:48)
=
X (cid:62) F ( V ) X , and update F (cid:48) in each time step immediately after each update of V to maintain the self-consistency, which forms the initial version of our proposed algorithm, self-consistent gradient-like eigen decomposition (SCGLED), shown in Algorithm 1.
Note that we need the eigenvectors corresponding to top- k smallest eigenvalues rather than the largest ones, so we decompose − F (cid:48) instead of F (cid:48) .
Different from traditional SCF method shown in Appendix B where V (cid:48) t is purely a temporary variable that should be discarded after each iteration, V (cid:48) t in our proposed algorithm is more likely a “training variable” that is ﬁrst randomly initialized and then trained during the whole iterative process.
While the convergence analysis of Algorithm 1 is heavily blocked by the intrinsic nonlinearity of approximated Schrodinger equations, which is fundamentally complex in computational physics, we instead use heuristics to boost the empirical convergence performance in later paragraphs, and prove the correctness of our algorithm under some robustness assumption of F ( V ) on the convergence point, with the following proposition
Proposition 1.
If Algorithm 1 converges to a stable convergence point V ∗ and for a small perturbation (cid:15) towards V ∗ , [ X (cid:62) F ( V ∗ ) X − X (cid:62) F ( V ∗ + (cid:15) ) X ]
( V ∗ + (cid:15) ) = O ( (cid:15) 2 ) , then V ∗ is the solution of equation 1.
The proof is deferred to Appendix C.
However, this initial version is of poor efﬁciency and stability.
The computational efﬁciency is blocked by the Gram-Schmidt process which is highly order-preserving thus cannot be decentralized/vectorized.
Also, due to the existence of F ( V ) =
H + U eff ( P ( V )) that is sensitive to the change of V , the iteration process is highly nonlinear.
In this case oscillation will easily occur which hinders the iteration process toward convergence.
To tackle these problems, we made several improvements to Algorithm 1 as follows.
First, we replace the classical Oja’s algorithm with the decentralized version of EigenGame (Gemp et al., 2021).
EigenGame replaces the time-consuming QR decomposition by a generalized GramSchmidt step in its gradient ∇ v i , which makes it theoretically harder to analyse (since the orthonormality of eigenvectors is not guaranteed during the iteration), but with better empirical performance due to its decentralized nature.
Its procedure is as follows:
[Equation]
v i [Equation] Second, to reduce oscillation during the iteration process, we introduce the damping technique for the update of F .
That is,
[Equation]
in which α is the mixing hyperparameter between 0 and 1.
Here we set it as 0.2.
Third, the selection of learning rate η is highly tricky for different molecule inputs.
To enhance the robustness of the algorithm towards learning rate, we introduce the momentum method for the update of V (cid:48) as follows:
[Equation]
in which the momentum term β is set to 0.9.
Fourth, instead of updating F in every iteration, we control the update interval of F via a parameter I F .
The ﬁrst reason is for efﬁciency, since V (cid:48) only update slightly in each time step with a small learning rate η , it may not be necessary to do a fresh computation of F ( V ) in each time step, especially considering that the computation of U eff ( P ( V )) is relatively time-consuming.
The second reason is for stability, since the damping technique in equation 6 will degenerate if previously updated F t − 1 and current F ( V ) are too close.
Summarizing all considerations above, our proposed algorithm is shown in Algorithm
E XPERIMENTS
In this section, we perform extensive performance benchmarks on W4-17 dataset(Karton et al., 2017), which is also applied in prior benchmark work (Lehtola, 2019).
All the 160 singlet molecules in the W4-17 dataset are used to evaluate the performance of our proposed algorithm.
Our algorithm
Input: H, S, U eff ( · ) in equation 1 and equation 2, total number of iterations T , learning rate η , F ’s update ratio I F
Output: V ∗ , the solution of equation1: Initialize V (cid:48)
∈
R N × k randomly.
2: F ← H 3: m ← 0 N × k 4: Find X satisfying X (cid:62) SX
= I .
5: for t = 0 , 1 , 2 , · · · , T do 6: if t mod I F = 0 then 7:
V ← XV (cid:48) 8:
F ← (1 − α ) F + αF ( V ) 9:
F (cid:48) ← X (cid:62) FX 10: end if 11:
Obtain ∇ R V (cid:48) for one-step decomposition of − F (cid:48) using equation 4.
12: m ← βm + η ∇ R V (cid:48) 13:
V (cid:48) ← V (cid:48) + m t 14:
Normalize all column vectors in V (cid:48) following equation 5.
15: end for 16: V ∗ ← XV (cid:48) is implemented in Python with NumPy, and optimized via Numba (Lam et al., 2015). The learning rate η is set to be 10 − 2 . The effective potential matrix function U eff ( · ) in Algorithm 2 is based on Hartree-Fock theory and provided by PySCF (Sun et al., 2018).
We use the standard 6-31G basis set (Ditchﬁeld et al., 1971) for the computation of all molecules.
All experiments are run on a AMD
Ryzen 7 5800H CPU (8 cores, 16 threads, 3.2-4.4GHz) with 16GB memory.
For the reproducibility of the result, in all experiments we initialize V (cid:48) in Algorithm 2 by (cid:20) I (cid:21) where I is a k × k identity matrix.
5. SCGLED AS AN INITIAL GUESS METHOD
A direct application of our proposed method is to provide an initial guess for the traditional SCF method shown in Algorithm 3.
We compare with the following popular initial guess methods that are widely applied in quantum chemistry and solid-state physics software.
All of them highly rely on domain-speciﬁc heuristics
• H core : Core Hamiltonian method, which obtains the guess orbitals from the diagonalization of the core Hamiltonian H , ignoring all interelectronic interactions.
• atom (Van Lenthe et al., 2006):
Superposition of atomic HF density matrices.
• minao:
Superposition of atomic densities projected in a minimal basis obtained from the ﬁrst contracted functions in the cc-pVTZ or cc-pVTZ-PP basis set.
• huckel (Karton et al., 2017):
Parameter-free H¨uckel guess.
In the experiment, we use PySCF’s integrated implementation for all the initial guess baselines.
PySCF (Python-based Simulations of Chemistry Framework, (Sun et al., 2018)) is an open-source, highly popular quantum chemistry software in Python, whose computationally critical parts are implemented and optimized in C to guarantee efﬁciency.
We evaluate the performance of an initial guess method from two aspects
• Precision: the distance between the initial guess and the ﬁnal converged solution of equation 1, measured by the averaged energy error over 160 molecules.
(expected to be as small as possible) • Convergence: the ratio that the initial guess can successfully lead to a converged solution via traditional SCF iteration, measured by the number of molecules that failed to converge in SCF iteration with a certain initial guess method.
(expected to be as few as possible)
[Figure] Initial guess methods are expected to be very rapid compared with main SCF iterations, so we restrict the range of T from 200 to 10,000, corresponding to 5-250 milliseconds, while traditional methods take around 30 milliseconds.
We also set F ’s update interval I F = 50 in this experiment.
The result of performance evaluation is shown in Figure 2.
While traditional initial guess methods are parameter-free heuristics, SCGLED is iterative thus the performance and time cost are directly inﬂuenced by the number of iterations T .
As a result, SCGLED is shown as a curve in timeperformance space while traditional methods are shown as points.
In both Figure 2a and Figure 2b,
SCGLED’s curve lies at the lower-left direction of traditional methods, indicating that our method achieves better results for both precision and convergence ability.
Comparing the result of the best baseline “minao” and SCGLED at T = 1000 in Figure 2a, we can ﬁnd that averagely SCGLED is 25x more precise than minao (0.013379 Hartree vs. 0.343194 Hartree in energy error) while the wall time are very close (26.3 ms vs. 28.6 ms).
Our proposed method not only performs better or close to traditional methods given the same time, but also provides more ﬂexible options, such as paying more time to achieve better performance in time-rich tasks, or sacriﬁcing performance to satisfy rigorous time limitation, which is inconceivable for traditional methods due to their heuristics nature.
A more detailed, per-molecule result is shown in Appendix E.
[Figure]
For SCGLED, we also study the inﬂuence of I F , the parameter to control F ’s update interval, to the performance, shown in Figure 3.
The I F -performance relation is shown as a U-shaped curve, which indicates that I F should be set to a moderate value, neither too large nor too small.
The best value of I F (the bottom of the U-shaped curve) is relevant to the total number of iterations T : I F should be smaller for small T so F can be updated for reasonable times in very limited iterations, and should be moderately larger for large T so that the convergence acceleration technique (such as the damping technique in equation 6) works better.
5. SCGLED AS A FULL SOLVER
While gradient-based methods are widely used in machine learning in ways that do not require extreme precision, it is usually computationally intractable in other ﬁelds that require high precision, especially scientiﬁc computing, due to its low convergence rate.
However, a counterintuitive result is that our proposed gradient-like method can produce highly precise solutions of equation 1 in a reasonable number of iterations.
[Figure] [Figure]
The result that our proposed method acts as a full solver (without any traditional SCF iteration) is shown in Figure 4.
The convergence criterion is selected as the default setting in PySCF.
That is, the difference of a molecule’s total energy before and after a single-step SCF iteration should be less than 10 − 10 , which is extremely strict.
I F is set as 100 since T is relatively large in this scenario.
It shows that our proposed method successfully solved more than half of the molecules within 20,000 iterations, and 155 out of 160 molecules within 1 million iterations.
If we replace the simple damping technique in Algorithm 2 with a more powerful DIIS method (Pulay, 1980) for the last 10% iterations 2 , we can successfully solve all 160 molecules with very high precision.
Some of the highly precise solutions are visualized in Figure 5 by VMD
(Humphrey et al., 1996), in which the coordinate and type of atoms are given for each molecule, and the electron densities are solved by SCGLED, represented as isosurfaces shown as grey surface around the molecule.
C ONCLUSION
In this work, we solve the approximated Schr¨odinger equations from scratch with gradient-like eigen-decomposition.
This work contributes to both the ﬁeld of computational physics and machine learning as follows
• For computational physics, beside the performance advantage, this work shows the possibility to solve the approximated Schr¨odinger equations from a purely optimization-based aspect, without any heuristics method based on prior quantum mechanism knowledge to bootstrap the solving stage.
In this way the solving of approximated Schr¨odinger equations can be stripped out from its physical background, studied independently as a mathematical optimization problem.
• For machine learning, this work explores a brand new area of “self-consistent” eigenvalue problems, especially approximated Schr¨odinger equations, for stochastic eigendecomposition methods such as Oja’s algorithm and EigenGame, which are previously regarded as specialized methods for k -PCA.
While such methods can properly handle stochasticity, this work shows that they are also capable of handling self-consistency, which leads to a potential of application in a broader ﬁeld of scientiﬁc computing.
A P HYSICAL B ACKGROUND
In this section, we brieﬂy introduce the physical background of the problem in section 3.
We refer to (Szabo & Ostlund, 1996; Martin, 2004) for more details.
In computational physics, a foundation problem is to solve the Schr¨odinger equation of a many-body quantum system
[Equation]
where H is the Hamilton operator for a system of M nuclei and N electrons described by their coordinates R A and r i .
With the Born-Oppenheimer approximation (nuclei are much heavier than electrons), we consider the electrons to be moving in the ﬁeld of ﬁxed nuclei, thus the kinetic energy of the nuclei (approximated as zero) and the repulsion between the nuclei (approximated to be constant) can be neglected.
In this case we write H as
[Equation]
and nuclei +
N (cid:88) i =
N (cid:88)
j = i +  | r
i − r j | (cid:124) (cid:123)(cid:122) (cid:125) repulsion between electrons and Ψ( r 1 , · · · , r N ) the electronic wave function, which should be normalized (i.e., (cid:104) Ψ | Ψ (cid:105) =
(cid:82) Ψ ∗ ( r 1 , · · · , r N )Ψ( r 1 , · · · , r N ) dr 1 · · · dr N = 1 ).
For convenience, let operator h ( i )
= − 12 ∇ 2 i − (cid:80) MA =
Z A | r i − R A | so H can be rewritten as H
= (cid:80)
Ni =1 h ( i ) + (cid:80)
Ni =1 (cid:80)
Nj =
i + 1 | r
i − r j | .
According to the variational principle, the problem of ﬁnding solution Ψ for the ground state energy E 0 can be transformed to the following constrained optimization problem
[Equation]
Since the multi-electron wave function Ψ( r 1 , · · · , r N ) is computationally intractable when
N is large, a basic way is to approximate it as the product of N orbital wave function ψ 1 ( r 1 ) ψ 2 ( r 2 ) · · · ψ N ( r N ) satisfying (cid:104) ψ
i | ψ i (cid:105) =
(cid:82) ψ ∗ i ( r )
ψ i ( r ) dr = 1 , ∀ i ∈ 1 · · · N , which is called “Hartree approximation”.
We also expand ψ i ( r )
= (cid:80) K u =1 C ui φ ( r ) as a linear combination of K “basis functions” which is ﬁxed and given, in this case our task becomes to determine the value of all C ui so as to determine the approximated wave function.
To construct a Lagrange multiplier
[Equation]
we have
[Equation] [Equation]
and
[Equation]
Let
H uv =
[Equation]
S
[Equation]
= (cid:82) φ
[Equation]
u ( r ) φ
[Equation]
( r ) dr and
E uvλσ =
[Equation]
[Equation]
2 which are all constants since { φ i } are given functions, we have [Equation] = [Equation]
uvλσ
[Equation]
C
[Equation] − [Equation] S
[Equation]
Let ∂L∂C ui = 0 , ∀ i = 1 · · · N, u = 1 · · · K and we have (cid:88)
[Equation]
Let P λσ =
(cid:80) j
( (cid:54) = i )
C λj C σj , [ U eff ( P )] uv = (cid:80) λ,σ P λσ E uvλσ , F uv = H uv
+ [ U eff ( P )]
uv and Λ = diag ( (cid:15) 1 , · · · , (cid:15) N ) , P, U eff ( P ) ,
F ∈ R K × K , we have the matrix form of the above equation
[Equation]
which is a generalized eigenvalue problem where the matrix F to be decomposed is deﬁned by the eigenvectors C .
Actually, there are several improved approximation theories based on the Hartree approximation mentioned above, in which the deﬁnitions of P and U eff ( P ) are different.
A most inﬂuential one is the Hartree-Fock theory in which the multi-electron wave function is approximated by a slater determinant
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) χ 1 ( x 1 ) χ 2 ( x 1 ) · · · χ N ( x 1 ) χ 1 ( x 2 ) χ 2 ( x 2 ) · · · χ N ( x 2 ) . χ 1 ( x N ) χ 2 ( x N ) · · · χ N ( x N ) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) to conform to the antisymmetry principle Ψ( · · · , x i , · · · , x j , · · · ) = − Ψ( · · · , x j , · · · , x i , · · · ) whose famous representation is Pauli exclusion principle.
In this way P uv = 2 (cid:80) N/ 2 i
C λj C σj (or in matrix form, P = 2 CC (cid:62) ) and [ U eff ( P )] uv = (cid:80) λ,σ P λσ E uvλσ − 12 (cid:80) λ,σ P λσ E uλσv , in which an “exchange term” − 12 (cid:80) λ,σ
P λσ E uλσv is added.
B S ELF -C ONSISTENT F IELD (SCF) METHOD
Self-Consistent Field (SCF) is a standard method to solve equation 1.
An initial density matrix
P 0 is generated via heuristics based on prior quantum mechanism knowledge, then the generalized eigen-decomposition problem F ( P t − 1 ) V t =
SV t Λ t is repeatedly solved to obtain eigenvectors V t and density matrix P t
= 2 V t V (cid:62) t at the t -th iterations, t = 1 , 2 , · · · , until | P t − P t − 1 | is less than the convergence threshold.
The detailed routine is shown in Algorithm 3.
To solve the generalized eigen-decomposition problem, it should be transformed to standard form ﬁrst.
To achieve this, the orthogonalization technique introduced in (Szabo & Ostlund, 1996) is applied to eliminate the overlap matrix S .
First, ﬁnd a linear transformation X so that X (cid:62) SX
=
I .
There are several ways to achieve this, a popular one is named “canonical orthogonalization” which lets X = U diag ( s − 1 / 2 ) , where U and s are all eigenvectors and eigenvalues of S .
Note that all eigenvalues of S are positive so there is no difﬁculty of taking square roots.
Then let
V (cid:48)
=
X − 1 V ( V = XV (cid:48) ), we have F ( V ) XV (cid:48) =
SXV (cid:48) Λ .
Multiply X (cid:62) on the left
and we have X (cid:62) F ( V ) XV (cid:48) =
X (cid:62) SXV (cid:48) Λ .
Let F (cid:48)
( V ) =
X (cid:62) F ( V ) X , we have F (cid:48) ( V
[Equation]
(cid:48)
[Equation]
(cid:48)
[Equation]
(9) which is a standard eigen-decomposition problem.
Algorithm
Input: H, S, U eff ( · ) in equation 1 and equation 2.
Converge threshold (cid:15) .
Output: V ∗ , the solution of equation
Obtain initial density matrix P 0 via initial guess methods (e.g., SAD or core Hamiltonian).
Find X satisfying X (cid:62) SX
= I .
t ←while | P t − P t − 1 | < (cid:15) do
F t ← H + U eff ( P t )
F (cid:48)
t ← X (cid:62) F t X t ← t +
Obtain precise eigenvectors V (cid:48) t of F (cid:48)
t − 1 corresponding to the top- k smallest eigenvalues via classical eigen-decomposition methods such as QR iteration.
V t ← XV (cid:48) t
P t ← 2 V t V (cid:62) t end while
V ∗ ← V t
C P ROOF
Proof of Proposition 1:
Proof.
V ∗ being the solution of equation 1 means that V ∗ contains the generalized eigenvectors corresponding to the top- k smallest generalized eigenvalues of [ F ( V ∗ ) , S ] .
In Algorithm 1, we denote the converged V (cid:48) in the ﬁnal step as V ∗(cid:48) .
With the orthogonalization technique introduced in
Appendix B, V ∗ being the solution of equation 1 is equivalent to 1.
V ∗(cid:48)
=
X − 1 V ∗ contains the eigenvectors corresponding to the top- k smallest eigenvalues of F ∗(cid:48) .
F ∗(cid:48)
=
X (cid:62) F ( V ∗ ) X .
When V (cid:48) in the iteration of Algorithm 1 is equal to the converged solution V ∗(cid:48) , F ∗(cid:48)
=
X (cid:62) F ( V ∗ )
X holds.
Since V ∗(cid:48) is already converged, the one-step decomposition of − F ∗(cid:48) will produce no update for V ∗(cid:48) .
For Oja’s algorithm equation 3, letting M = − F ∗(cid:48) , this means ( v ∗ 1 , · · · , v ∗ k ) =
QR ( v ∗ 1 + ηMv ∗ 1 , · · · , v ∗ k + ηMv ∗ k ) , η > 0 .
Note that v ∗ 1 , · · · , v ∗ k are orthonormal guaranteed by the
QR decomposition.
Considering that the QR decomposition is implemented via the Gram-Schmidt process, starting from i = 1 , we have
[Equation] [Equation]
in which β 1 = (cid:107) v ∗ 1 + ηMv ∗ 1 (cid:107) , β 2 = (cid:107) v ∗ 2 + ηMv ∗ 2 − [( v ∗ 2 + ηMv ∗ 2 ) (cid:62) v ∗ 1 ] v ∗ 1 (cid:107) , · · · are normalization factors.
Therefore, v ∗ 1 , · · · , v ∗ k are all eigenvectors of − F ∗(cid:48) .
To show that v ∗ 1 , · · · , v ∗ k are eigenvectors corresponding to the top- k eigenvalues, we mainly follow (Oja & Karhunen, 1985) and (Hertz et al., 2018).
We start from k = 1 to show that v ∗ 1 should correspond to the largest eigenvalue λ 1 of − F ∗(cid:48) to be a stable convergence point.
First, for v 1 , the QR decomposition degenerates to normalization, that is
[Equation]
Assuming η is small enough, it can be expanded as a power series of η .
By ignoring O ( η 2 ) terms, we have
[Equation]
While we have a converged solution v ∗ 1 , we already know that it is an eigenvector but do not know which eigenvalue it corresponds to.
In this case, we assume that after a series of iterations, v 1 is very close to the eigenvector corresponding to the α th largest eigenvalue (denoted as v α ) of the ﬁnal converged matrix − F ∗(cid:48)
= − X (cid:62) F ( v ∗ 1 ) X , that is
[Equation]
where (cid:15) is a very small perturbation vector.
Since we already know that v ∗ 1 is an eigenvector, we have v ∗ 1 = v α .
Let M = − F ∗(cid:48) = − X (cid:62) F ( v ∗ 1 ) X and E = X (cid:62) F
( v ∗ 1 ) X − X (cid:62) F ( v 1 ) X
=
− F (cid:48) − M ⇒ − F (cid:48) = M + E .
{+From the assumption in the proposition, we have Ev 
+}
= O ( (cid:15) 2 ) .
Then we do one-step iteration as follows:
[Equation]
Now we want to analyse the direction of ∆ v 1 to see whether it can conquer the perturbation (cid:15) and converge to v ∗ 1 = v α stably.
We do it by multiplying another eigenvector v β (cid:62) on the left (ignoring the O ( (cid:15) 2 ) term)
[Equation]
We notice that, if β > α (i.e., λ β > λ α ), there will always exist a direction (cid:15) so that both v β (cid:62) (cid:15) and v β (cid:62) ∆ v 1 are larger than zero.
In this case v 1 will ﬂip to the other eigenvector thus cannot converge to v α stably.
Therefore, α should be 1 so that v ∗ 1 = v α corresponds to the largest eigenvalue of − F ∗(cid:48) .
For top- k case, take k = 2 as an example, notice that the Gram-Schmidt process not only normalize the eigenvector v 2 but also orthogonalize it towards v 1 , so ( v 2 + ∆ v 2 ) (cid:62) v 1 = (∆ v 2 ) (cid:62) v 1 = 0 , and we already know that v 1 will converge to the eigenvector corresponding to the largest eigenvalue.
Therefore we can only select other eigenvectors out of v 1 to analyse the direction of ∆ v 2 , which shows that v 2 will converge to the eigenvector corresponding to the second largest eigenvalue.
D F ULL - SOLVER P ERFORMANCE B ENCHMARK
While SCGLED is not targeted to be an extremely precise (convergence threshold less than 10 − 10 ), end-to-end solving technique due to its ﬁrst-order nature, we still compare its performance toward traditional SCF methods in a full-solver setting for reference.
We test three full-solving schemes
• Independent SCGLED: applying our proposed method, SCGLED, as a full solver without any tranditional SCF methods.
• SCF + minao: applying the traditional SCF method with “minao” as the initial guess method, which performs best within traditional methods as shown in subsection 5.1, and acts as the default setting in PySCF.
• SCF + SCGLED: applying the traditional SCF method with SCGLED as the initial guess method.
with three settings for heuristics convergence acceleration techniques
• Vanilla: no convergence acceleration method is applied.
Damping: the damping technique shown in equation 6 is applied in SCGLED or SCF iteration.
• DIIS: the DIIS technique (Pulay, 1980) is applied in independent SCGLED or SCF iteration.
Note that DIIS is not applied when SCGLED is served as an initial guess method.
So excluding vanilla SCGLED in Algorithm 1 whose empirical performance is too poor for benchmarking, we have 8 test methods in total to be benchmarked.
In Table 1, we evaluate these methods from two aspects
• Convergence: evaluated via the number of successfully solved molecules within the total 160 singlet molecules in W4-17 dataset.
• Efﬁciency: evaluated via the averaged 3 end-to-end time cost of the method along the molecules that are successfully solved.
For the parameter setting, when SCGLED is served as an initial method, we set T = 5000 and
I F = 50 for Vanilla and Damping cases, which is relatively large since their performance are more sensitive to the quality of initial guess.
{+For the DIIS case which are relatively not so sensitive to the initial guess, we set T = 
+}
and I F = 25 .
When SCGLED is running independently without traditional SCF iterations, the parameter setting stays the same as in subsection 5.2.
Vanilla
Damping
DIIS
Independent SCGLED — 155 /
[Table]
ms
[Table]
ms
SCF + minao 141 / 332.6 ms 159 /
[Table]
ms
[Table]
ms
SCF + SCGLED 149 / 190.3 ms 159 /
[Table]
ms 160 /
[Table]
ms
The benchmark result is shown in Table 1 and Figure 6.
While independent SCGLED works in the full-solver setting, there is still a performance gap with other methods that target for full-solving performance.
Instead, SCF + SCGLED works best, especially in the Vanilla case that both the convergence performance (unsolved molecules from 19 to 11) and efﬁciency performance (averaged time cost from 332.6 ms to 190.3 ms) are signiﬁcantly improved.
For the Damping and DIIS case, the performance advantages are narrowed since the heuristics convergence acceleration techniques themselves become a more dominant factor for end-to-end performance.
Also note that, as an
[Figure]
optimization-based method
, SCGLED brings more randomness compared with traditional heuristics initial guess methods that are built on domain knowledge.
Therefore, while SCF + SCGLED performs better for most of the molecules, its distribution of time costs is wider, shown as curves with smaller slope in Figure 6, and the end of the curve in Damping and DIIS settings can correspond to larger time cost compared with SCF
+ minao. E R ESULT P ER M OLECULE In this section, we provide the description of the full numerical result listed in the supplementary material of this paper, for all the 160 singlet molecules in the W4-17 dataset.
For each molecule and method, we list four values as follows:
• Energy: the total Hartree-Fock energy of the molecule computed with the method (Unit: Hartree, 1 Hartree = 4 . 3598 × 10 − 10 Joule).
• Energy error: the difference between the computed energy above and the exact HartreeFock energy listed in the second column of the table (expected to be as small as possible).
Note that we only keep 6 decimals due to the space limit, while the converge criterion is stricter, so there will be some cases that the error is 0 .
000000 while the number of iterations in the fourth line is not equal to zero.
• Wall time: the wall time to run the method for the corresponding molecule.
• Convergence: whether the computed result can reach to a converged solution with the vanilla SCF iteration shown in Algorithm 3 (cannot reach a converged solution via SCF: (cid:55) ; can reach a converged solution via SCF: (cid:51) ; already a converged solution: (cid:51)(cid:51) ).
If it can reach to a converged solution, we also show the number of SCF iterations it costs.
If the number of SCF iterations is equal to 0, it means that the method already reaches a precise (converged) solution before the SCF iteration.
In the result table, we bold the result if it is a dominated result over the baselines (the energy error and wall time are both smaller than the best of the baseline methods while the number of iterations is not larger than the best baseline).
We underline the result if it takes more time than the best baseline while the other target values are dominated.
The H core method are not taken into account here since both the energy error and the convergence performance are very unsatisfactory, even though it is extremely fast.
The table shows that
• 75 out of 160 molecules contain at least one dominated result (in bold).
• 155 out of 160 molecules contain at least one more precise result (marked with underline).
• 72 out of 160 molecules contain at least one result that already reached the converged solution (marked with (cid:51)(cid:51) , the number of SCF iterations is equal to 0) within 10,000 iterations of SCGLED.
That is, we already obtained the converged, precise solution.
No further SCF iterations are needed.
• 14 out of 160 molecules are successfully led to convergence by SCGLED while all the four baseline methods failed to do so.
The full table of numerical result can be found in “ full numerical result.pdf ” as a supplementary material of this paper.