Self-consistent Gradient-like Eigen Decomposition in Solving Schrödinger Equations
Authors: Xihan Li
Xiang Chen
Rasul Tutunov
Haitham Bou Ammar
Lei Wang
Jun Wang
Venue: ICLR 2022 Submitted (2022)
Keywords: Schrödinger Equation
Hartree-Fock
Self-consistent
Eigen-decomposition
online learning
stochastic k-PCA
Oja's Algorithm
EigenGame

Abstract: The Schrödinger equation is at the heart of modern quantum mechanics. Since exact solutions of the ground state are typically intractable, standard approaches approximate Schrödinger's equation as forms of nonlinear generalized eigenvalue problems $F(V)V = SV\Lambda$ in which $F(V)$, the matrix to be decomposed, is a function of its own top-$k$ smallest eigenvectors $V$, leading to a ``self-consistency problem''. Traditional iterative methods heavily rely on high-quality initial guesses of $V$ generated via domain-specific heuristics methods based on quantum mechanics. In this work, we eliminate such a need for domain-specific heuristics by presenting a novel framework, Self-consistent Gradient-like Eigen Decomposition (SCGLED) that regards $F(V)$ as a special ``online data generator'', thus allows gradient-like eigendecomposition methods in streaming $k$-PCA to approach the self-consistency of the equation from scratch in an iterative way similar to online learning. With several critical numerical improvements, SCGLED is robust to initial guesses, free of quantum-mechanism-based heuristics designs, and neat in implementation. Our experiments show that it not only can simply replace traditional heuristics-based initial guess methods with large performance advantage (achieved averagely 25x more precise than the best baseline in similar wall time), but also is capable of finding highly precise solutions independently without any traditional iterative methods.

Under review as a conference paper at ICLR 2022
SELF-CONSISTENT GRADIENT-LIKE EIGEN DECOM- POSITION IN SOLVING SCHRÖDINGER EQUATIONS
Anonymous authors Paper under double-blind review
ABSTRACT
The Schrödinger equation is the heart of modern quantum mechanics. Since exact solutions of the ground state are typically intractable, standard approaches approximate Schrödinger's equation as forms of nonlinear generalized eigen- value problems F(V) SV A in which F(V), the matrix to be decom- posed, is a function of its own top-k smallest eigenvectors V, leading to a "self- consistency problem". Traditional iterative methods heavily rely high-quality initial guesses of V generated via domain-spectio heuristics methods based on quantum mechanics. In this work, eliminate such need for Nomain-specifft heuristics by presenting a novel framework, Self-consistent Gradient-like Figen Decomposition (SCGLED) that regards F(V) as special "online data genera- tor", thus allows gradient-like cigendecomposition methods in streaming k-PCA approach the self-consistency of the equation from scratch in an iterative way similar to online learning. With several critical numerical improvements, SC- GLED is robust to initial guesses, free of quantum-mes mechanism-base heuristics designs, and neat in implementation. Our experiments show that it not only can simply replace traditional heuristics-based initial guess methods with large perfor- mance advantage (achieved averagely 25x more precise than the best baseline in similar wall time), but also is capable of finding highly precise solutions indepen- dently without any traditional iterative methods.
INTRODUCTION
While the many-body Schrödinger equation governs all quantum-mechanical systems, can be solved analytically only for extremely simple ones like an isolated hydrogen atom. Nonetheless, series of important approximation theories are developed so that it can be solved numerically for many relevant molecules and materials including tens to thousands of electrons, making it possible for physicists and chemists to explore the properties of matters in an ab- initio or first- first-principle way, serving as a foundation of computational physics, chemistry and material science research.
Among those approximation theories, two mainstream directions involve the Hartree-Food (Hartree 1928 Hartree & Hartree 1935) and density functional theories (Hohenberg & Kohn 1964) that
either approximate the Schrödinger equation as a "Hartree-Fock equation or as a "Kohn-Sham equation" (Kohn & Sham 1965). While very different in nature, both approximated equations are finally formed as a nonlinear, generalized eigenvalue problem F(V)V = SVA A in which F(V) and S are N x N real symmetric matrices, A diag(A1, top-k smallest eigenvalues, and V [e1.--- uys] is an N x k matrix containing the corresponding top-k eigenvectors. Interestingly, the matrix which we aim to decompose explicitly defined as given function of the eigenvectors V (denoted F(V) N - NxN), leading to a "cause- and-effect dilemma" or selff-consitteny problem". That is, the form of the equation is defined by its final solution, but how does one know the solution if the form of the equation is not determined?
k x k matrix containing the
The dominant method in solve such equations is generally referred to as Self-Consistent Fields (SCFs). SCFs are the main component of nearly all mainstream quantum chemistry and solid-state physics software. The general idea is to perform a fixed-point iteration, that is, guess an initial solution Vú from scratch, then solve F(Ve-1)V, the kth iteration, l = 1, 2, ... until [[&&) F(Vx-1) is less than a given convergence threshold. Unfortunately, there is guarantee that such iteration can always lead converged solution
SVAA, repeatedly to obtain eigenvectors Vi at

---

Under review as a conference paper at ICLR 2022
(Froese Fischer 1987.. Actually, SCFs are extremely sensitive to the selection of initial guess Vo- Without a hugh-quality initialisation, SCFs easily fail and oscillate between two or more different results.
Till now, all popular choices for the initial guess are heuristics-based, highly reliant on specialized prior knowledge of quantum mechanics. For example, the SAD method (Van Lenthe et al., 2006) applies the physical intuition that molecular electron densities can be approximated via the sum of atomic electron densities. Here, the core Hamiltonian method ignores all interactions between elec- trons, which simplifies the equation to a standard eigen-decomposition problem. More methods and details can be found in (Lehtola 2019). Besides the prerequisite of fomain-specifie theories, such methods are also sophisticated in implementation to achieve satisfactory convergence performance, which is a main barrier for practitioners to build perforrefe solvers from scratch.
Meanwhile, in the machine learning community, researchers are facing the challenge of analyzing real-time, online data represented as streams. In contrast to the normal case that data is fully avail- able offline, online data is received sequentially and may even change over time, thus specialized methods are developed to dynamically adapt to new patterns the data stream, which called stochastic, streaming
online algorithms. Specially, series of works (Ojal 1982 Tangg 2019 Allen-Zhi & Li 2017] Gemp et al. 202I) focus on stochastic k-PCA, which aim to estimate the top-A principal components of data stream in real-time manner. To handle the stochasticity of data, such stochastic k-PCA methods usually contain gradient-like eigen decomposition. When new data sample is available, the eigenvectors will be updated towards gradient-like direction computed by the new sample, with a small learning rate.
In this work, we noticed that F(V) in the aforementioned eigen-decomposition problem is sub- ject to concurrent update during the sequential update of towards self-consistency: Vi F=P(V),
V22 F=
In this case F(V) can be regarded as a special "online data generator", which shares similarities with the online learning setting that online data is concurrently updated during the sequential learning process. In this way, we explore the possibility of applying gradient-like sigen-decomposition in stochastic k-PCA methods to handle the seff-consistency of approximated Schrödinger equations. Together with several numerical improvements to enhance the smoothness of optimization, we developed Self-consistent Gradient-like Eigen Decomposition (SCGLED), an efficient solver for the self-consistent approximated Schrödinger equations, SCGLED is robust to initial guesses Va, free of quantum quantum-mechane mechanism bebsedd heuristics design, and neat in implementation. While it can simply replace traditional heuristics- initial guess methods with performance ad- vantage, it also capable of finding highly precise solutions without any traditional SCF iteration.
RELATED WORK
The computational theories for quantum many-body systems, especially for determining the wave function of Schrödinger equations, has a long history starting from 1920s. There are three main- stream theories: Hartree-Fock theory (Hartree 1928 Hartree & Hartree 1935), density functional theory (Hohenberg & Kohn| 1964) and quantum Monte Carlo. With the prosperity of deep learning and differentiable optimization in recent years, there are a series of works focusing on deep-learning- aided wave function representation of quantum Monte Carlo methods such as FermiNet (Pfau et al.) 2020) and PauliNet (Hermann et al. 2020). For density functional theory, there also some works using neural networks to learn the exchange-correlation functional (Li et al., [021]] Kasim & Vinko, 2021). However. these works focus more on improving simulation accuracy towards physical re- ality by using neural networks as better functional approximator, while our work's 's focus is very different, stressing on the optimization efficiency while the equation completely given. There also works direct optimization for Kohn-Shan equations by total energy minimization (Yang) ctt al. 2006),, which model the problem as a constrained optimization problem instead, and more domain-specifie knowledge is involved, while our work tries to solve the equation from purely optimization-bl based aspect.
For gradient- like eigen- decomposition, most of the works focus stochastic k-PCA, which esti- mates the top-k principal components of a data stream. Let x e Rd denote a random data sample at time step 1, U2 Rdd i .... denote the estimate of vth principal component (eigenvector of
2

---

Under review as a conference paper at ICLR 2022
F(V)
V
V
A
Generalized Eigen-
decomposition
-Pp
<
"Seff consistent"
Definition F(V)
F(V)
+ Vet(( 2VVT
Figure 1: An visualization of the problem. S, E RNXN and Uar RNxN + RR given input. The solution V should obey both the generalized eigen-decomposition and the definition of F(V), which leads to "self-consistent" problem.
E[XX at time t, corresponding to the ith largest cigenvalue. 77 denotes the learning rate.
Oja's algorithm: v/ f D; + 7(x 0z). Vi (1) 1, . and (V1, .x) + QR(V) ,
Here Q 8k) is the Gram-Schmidt decomposition that orthonormalizes uf vt- There are also several follow-up works (Sanger 1989 TTngg 2019) that improve the performance, in which
most recent one is EigenGame (Gemp al. 2021
(x Us x Uz) Tu]]] Vi = 1,...
EigenGame: Vu,
2x x vi
Ug}
(x Uy, x vz))
j<
R 0g t Vo u),, 1/ + Us + n py and vi +
Vi 1, ... k: |
3 PROBLEM DESCRIPTION
In this paper, while what we want to solve is the approximated Schrödinger Equations, whose phys- ical background is elaborated in Apppediix A we abstract it mathematically as the following type of nonlinear generalized eigenvalue problem
F(VV==SH) =
(1)
where
F(V): an N x N real symmetric matrix to be decomposed, which is defined in equation as function of V.
S: N N positive semi-definie matrix, which is constant input in the problem.
A: A = diag()), Av) is k x l diagonal matrix containing the top-k smallest eigen-
values.
V: V [vi,... ve] is N matrix containing & column eigenvectors corresponding to the top-k smallest eigenvalues.
The definition of F(V): NN RNN is follows:
F(V) def H Unn(
in which H is an N x N real symmetric matrix, which is given in this problem. Ven NNN + N is a given function. We also define P(V) T for convenience. conclude, the input
While the regular eigenvalue problem can be described as finding V that obeys AV VA in which is an N N matrix to be decomposed, A diag(Ai, ... ,Ax) contains all eigenvalues and V = (v....., UN) contains the corresponding cigenvectors, generalized eigenvalue problem adds additional matrix B with the form AV A.A. When B T, degenerates regular eigenvalue problem.
3

---

Under review as a conference paper at ICLR 2022
of the problem is S. H, Uer(-) and k. and the output of the problem is the eigenvectors V* that obeys both equation | and equation The top-k smallest eigenvalues of F(V*) are guaranteed to be negative.
To make it more clear, a toy example is provided as follows:
1.0
0.6953
1.1204
-0.9584
S
H
|t =
0.6953
1.0
-0.9884
-1.1204
(P))o = Σ
Pxo EuoEa
1
Σ
Pxo Eusoos
2
A,
00.7746 0.4441
0.4441
0..441
0.2970
0.5697
0.4441
F11
F11
F.21
Egg
0.5697
0.2970
0.4441
0.4441
0.7746
in which P = 2VVT, (P)(P) E IR2 iE a x 2
tensor.
The solution of the toy example
-0.3655
-0.5939
should be V* (0.5489, (.54489,055489) 0.5489) in this case F(V*)
and the result of -0.3655
-..999
top-1 eisin-ddeddcooooiiiionn
0.3655 -0 -0.5939
1.0
0.6953
VA
0.5939
-0.3655
0.6953
1.0
will happen to be exactly V* (0.5489. 0.5489) T with the smallest eigenvalue Å -0.5782, while the other one is [1.2115, 1122115 with eigenvalue X2 0.6703.
SOLVING APPROXIMATED SCHRÖDINGER EQUATIONS USING SELF-CONSISTENT GRADIENT-LIKE EIGEN DECOMPOSITION
First, notice that the stochastic k-PCA methods usually can be generalized to decompose a real symmetric matrix M. For Oja's sa's algorithm (Oja & Karhunen 1985), that is
of - v, + nM
Vi = 1, ... A and (VI, , + QR(c1 , %
(3)
in which of is the eigenvector corresponding to the ith largest eigenvalue.
Due to the lack of efficiency, such decomposition methods not favorable for traditional eigen- decomposition (M is given constant input) compared with classical algorithms such as QR itera- tion. However, stochastic algorithms,, they have unique advantage that they can adapt dynamic change of M, which suits the sticking point in equation that F(V) is subject to concurrent update when V changes. This motivates us apply them tackle the approximated Schrödinger equations.
Together with the orthogonalization technique introduced in An/ppendix B which transforms the gen- eralized eigenvalue problem into standard one, we replace M in equation with /" = X F(V)),, and update F' in each time step immediately after each update of V to maintain the self self-consistency, which forms the initial version of our proposed algorithm, self-consistent gradient- eigen decom- position (SCGLED), shown in Algorithm top-k smallest eigenvalues rather than the largest ones, so we decompose - F² instead of F'. Dif- ferent from traditional SCF method shown in Appendix B where V. is purely temporary variable that should be discarded after each iteration, V/ in our proposed algorithm is more likely "training variable" that is first randomly initialized and then trained during the whole iterative process.
Note that we need the eigenvectors corresponding to
While the convergence analysis of Algorithm is heavily blocked by the intrinsic nonlinearity of approximated Schrodinger equations, which fundamentally complex in computational physics, instead use heuristics to boost the empirical convergence performance in later paragraphs, and prove the correctness of our algorithm under some robustness assumption of F(V) the convergence point, with the following proposition
Proposition !/AAlgrithmm converges to stable convergence point V* and for a small pertur- bution e lowards V F(V XT +e)X] +
O(e2), then V is the solution
of equation
The proof is deferred to Appendix C
4

---

Under review as a conference paper at ICLR 2022
Algorithm The vanilla version of self-consistent gradient-like eigen decomposition (SCGLED)
Input: H,S, (-) in equation 1 and equation 2 learning rate 7) 0
Output: V*, the solution of equation]
the
1: Initialize V N randomly.
2: Find X satisfying X xT =
while V is not converged do
4: V + XVV'
5: F'+ + F(V) following equation
6: Update V for one-step decomposition of - FF using equation
3
7: end while
8: V* t XV
However, this initial version is of poor efficiency and stability. The computational efficiency is blocked by the Gram-Schmidt process which highly order-preserving thus cannot be decentral- ized/vectorized. Also, due to the existence of F(V) Hl HUnUn(((V)) that is sensitive to the change of V, the iteration process is highly nonlinear. In this case oscillation will easily occur which hinders the iteration process toward convergence. To tackle these problems, made several improvements to Algorithm as follows.
First, we replace the classical Oja's algorithm with the decentralized version of EigenGame (Gemp et al. 2021). EigenGame replaces the time-consuming QR decomposition by generalized Gram- Schmidt step in its gradient Ve,, which makes it theoretically harder analyse (since the orthonor- mality of eigenvectors is not guaranteed during the iteration), but with better empirical performance due to its decentralized nature. Its procedure
follows:
M Moj 01
Vu; 2M U4
Σ 12i
V R
Vu - (Ve, u))0
Vi 1,.
(4)
- u ワ and Us
of
Vi= 1, 1. ...
(5)
|
Second, to reduce oscillation during the iteration process, introduce the damping technique for the update of F. That is,
= (1 a)--a aV.),,
in which is the mixing hyperparameter between 0 and 1. Here set it 0.2.
Third, the selection of learning rate 7 highly tricky for different molecule inputs. To enhance the robustness of the algorithm towards learning rate, we introduce the momentum method for the update of y' as follows:
m: 3m VVV,
v'= V' + m4
(8)
which the momentum term is set to 0.9.
Fourth, instead of updating F in every iteration, control the update interval of F via param- eter Ip. The first reason is for efficiency, since V only update slightly in each time step with a small learning rate 17, may not be necessary to do a fresh computation of F(V) in each time step, especially considering that the computation
Uen( P(V)) is relatively time-consuming. The sec- ond reason is for stability, since the damping technique in equation will degenerate if previously updated Fi-1 and current F(V) are too close.
Summarizing all considerations above, our proposed algorithm is shown in Algorithm 2
EXPERIMENTS
In this section, perform extensive performance benchmarks W4-17 Karaset et al.) 2017). which is also applied in prior benchmark work [Lchtola[2019) All the 160 singlet molecules in the W4-17 dataset used to evaluate the performance of our proposed algorithm. Our algorithm
5

---

Under review as a conference paper at ICLR 2022
Algorithm 2 Self-consistent gradient-like eigen decomposition (SCGLED)
Input: H.S, Uer(-) in equation | and equation total number of iterations T, learning rate 77, F's
update ratio 1F
Output: V*, the solution of equation]
1: Initialize V" E NR randomly.
2: F f-t H
3: m +- 0N*
4: Find X satisfying X SX I.
5: for t = 0,
T do 6: if mod IF then
7:
V + XV"
8:
F + (1=a) + aF((V F' < 1 FX
9:
10:
end
11:
Obtain RRy for one-step decomposition of -F' using equation 4
12:
m + 3m + t
V" +
14:
Normalize all column vectors in V' following equation 5
end for
16:
V + XV-
implemented in Python with NumPy, and optimized via Numba (Lam et al. (015)) The learning rate 7) is set to be 10-2, The effective potential matrix function U.r(.) in Algorithm 2 is based on Cartree-Fock theory and provided by PySCF (Sun et al. 2018). We use the standard 6-31G basis set (Ditchfield et al. 197T) for the computation of all molecules. All experiments are
AMD
Ryzen 5800H CPU (8 cores, 16 threads, 3.2-4.4GHz) with 16GB memory. For the reprodueibility
where I is k k identity [
of the result, in all experiments we initialize V in Algorithm by matrix.
0
5.1 SCGLED AS AN INITIAL GUESS METHOD
direct application of our proposed method is to provide an initial guess the traditional SCF method shown in Algorithm 3 We compare with the following popular initial guess methods that are widely applied in quantum chemistry and solid-state physics software. All them highly rely on domain-specific heuristics.
Heore: Core Hamiltonian method, which obtains the guess orbitals from the diagonalization of the core Hamiltonian H, ignoring all interelectronic interactions.
atom (Van Lenthe et al. 2006); Superposition of atomic HF density matrices.
minao: Superposition of atomic densities projected in a minimal basis obtained from the first contracted functions in the cc-pVTZ or cc-pVTZ-PP basis set.
huckel (Karton et al. 2017): Parameter-free Hückel guess.
In the experiment, we use PySCF's integrated implementation for all the initial guess baselines.
PySCF (Python-based Simulations of Chemistry Framework, (Sun al. 2018)) is an open-source, highly popular quantum chemistry software in Python, whose computationally critical parts are guarantee efficiency.
implemented and optimized
We evaluate the performance of an initial guess method from two aspects
the distance between the initial guess and the final converged solution of equa- [ion[]] measured by the averaged energy error over 160 molecules. (expected to be as small as possible)
Precision:
Convergence: the ratio that the initial guess successfully lead converged solution via traditional SCF iteration, measured by the number of molecules that failed to converge in SCF iteration with certain initial guess method. (expected to be few possible)

---

Under review as a conference paper at ICLR 2022
glmmo aoalcccllia that not cotvoroeleee
200
100
Energy errer (Hartree)
500
80
200
10-2
10-1
2000
40
20
Home
How
10
huckel
T 5000
10000
minao
huckel
To 500
1000
T 2000
000
20000
10
10
10
10
10
Well
(5)
Wall Time
(a) Time-precision evaluation
(h) Time-convergence evaluation
Figure 2: Performance evaluation of initial guess methods
Initial guess methods are expected to be very rapid compared with main SCF iterations, so we restrict the range of T from 200 to 10,000, corresponding to 5-250 milliseconds, while traditional methods take around 30 milliseconds. We also set F's update interval Tp = 50 in this experiment.
The result of performance evaluation shown in Figure 2 While traditional initial guess meth- ods are parameter-free heuristics, SCGLED is iterative thus the performance and time cost are di- rectly influenced by the number of iterations T. As result, SCGLED is shown curve in time- performance space while traditional methods are shown points. In both Figure 2nd Figure 2b SCGLED's curve lies the lower-left direction of traditional methods, indicating that our method achieves better results for both precision and convergence ability. Comparing the result of the best baseline "minao" and SCGLED T 1000 in Figure 2a we can find that averagely SCGLED is 25x more precise than minao 10.013799 Hartree vs. 0.343194 Hartree in energy error) while the wall time are very close (26.3 ms vs. 28.6 ms). Our proposed method not only performs better or close to traditional methods given the same time, but also provides more flexible options, such as paying more time to achieve better performance in time-rich tasks, or sacrificing performance to sat- isfy rigorous time limitation, which is inconceivable for traditional methods due to their heuristics nature. A more detailed, per-molecule result is shown in Appendix
T- 200
conlecclessse that are not convergest
100
T- 200
109.
1= 500
1 500
Ereegy (Hartree)
80-
T- 2000
T- 2000
7
T 5000
T 5000
40
10-4
10-4
15.00
50
100
1.5.0
25
100
update
update
(a) JF- I--reciiioon relation
(b) Tp-convergence relation
Figure 3: 1p-performance relation of SCGLED
For SCGLED, we also study the influence of Ir. the parameter to control F's update interval, to the performance, shown in Figure 3] The /=-performance relation shown as U-shaped curve, which indicates that IF should be set moderate value, neither too large too small. The best value of Ie (the bottom of the U-shaped curve) is relevant to the total number of iterations T: If should be smaller for small T so F can be updated for reasonable times in very limited iterations, and should be moderately larger for large so that the convergence acceleration technique (such as the damping technique in equation ( works better.

---

Under review as a conference paper at ICLR 2022
5.2 SCGLED AS FULL SOLVER
While gradient-based methods are widely used in machine learning in ways that do not require extreme precision, it is usually computationally intractable in other fields that require high precision. especially scientific computing, due to its low convergence rate. However, counterintuitive result that our proposed gradient-like method can produce highly precise solutions of equation reasonable number of iterations.
#lmolccules that are not salved)
140
100
60
40
20
SCGLED (with DISS)
10a
10
10c
107
literations)
Figure 4: Performance of SCGLED for solving molecules independently.
(a) Benzene (CoHe)
D-Pentane (1,
(c) Chloroethane
(d) Formonitrile oxide (HCNO)
(C2CHH;)
Figure 5: Visualization of electron density for some molecules, solved via SCGLED.
The result that our proposed method as a full solver (without any traditional SCF iteration) is shown in Figure 4 The convergence criterion is selected as the default setting in PySCF. That is, the difference of a molecule's total energy before and after single-step SCF iteration should be less than 10 10, which is extremely strict. 1F is set 100 since T' is relatively large in this scenario. shows that our proposed method successfully solved more than half of the molecules within 20,000 iterations, and 155 out of 160 molecules within million iterations. If replace the simple damping technique in Algorithhm 2 with more powerful DIIS method (Pulay) 1980) for the last 10% iteration& we can successfully solve all 160 molecules with very high precision. Some of the highly precise solutions are visualized in Figure by VMD (Humphrey et al. 1996), in which the coordinate and type of atoms given for each molecule, and the electron densities solved by SCGLED, represented as isosurfaces shown as grey surface around the molecule.
CONCLUSION
In this work, we solve the approximated Schrödinger equations from scratch with gradient-lit like cigen-decomposition. This work contributes both the field computational physics and machine learning as follows:
F.gg, for T 10000, damping is applied for the first 9000 iterations, and apply DIS for the last 1000 iterations (10 actual DIIS computations done since 1F 100).
8

---

Under review as a conference paper at ICLR 2022
For computational physics, beside the performance advantage, this work shows the possi- bility to solve the approximated Schrödinger equations from a purely optimization based aspect, without any heuristics method based prior quantum mechanism knowledge to bootstrap the solving stage. In this way the solving of approximated Schrödinger equations be stripped out from its physical background, studied independently mathematical optimization problem.
For machine learning, this work explores brand new area of "self-consistent" eigen- value problems, especially approximated Schrödinger equations, for stochastic eigen- decomposition methods such as Oja's algorithm and EigenGame, which are previously regarded as specialized methods for k-PCA. While such methods can properly handle stochasticity, this work shows that they are also capable of handling self-consistency. which leads potential of application in broader field of scientific computing.
REFERENCES
Zeyuan Allen-Zhu and Yuanzhi Li. First efficient convergence for streaming k-pea: A global, gap- free, and near-optimal rate. In 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS), pp. 487-492, 2017. doi: 10.1109/FOCS.2017.51.
R. Ditchfield, W. J. Hchre, and J. A. Pople. Self-consistent molecular-orbital methods. ix. ex- tended gaussian-type basis for molecular-orbita. studies of organic molecules. The Journal of Chemical Physics, 54(2):724-728, 1971. doi: 10.1063/1.1674902. URL https://doi. orr//
1..1066/11.174441
Charlotte Froese Fischer. General hartree-fock program. Computer Physics Communications, 43(3):355-363 1987. ISSN 0010-4655. doi: http:///o..agg///000000000066669 90053-1. URL
0010465587900522
Ian M. Gemp, Brian McWilliams, Claire Vernade, and Thore Graepel. Eigengame: PCA as nash equilibrium. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL /tL.:ttppp///eeeneevieew.. 9SYbNq
D. R. Hartree. The wave mechanics of an atom with a non-coulomb central field. part i. theory and methods. Mathematical Proceedings of the Cambridge Philosophical Society, 24(1):89-110; 1928. doi: 10.
Douglas Rayner Hartree and W. Hartree. Self-consistent field, with exchange, for beryl- lium. Proceedings of the Royal Society of London. Series A Mathematical and Physical Sciences, 150(869):9-3) -33,, 1935. doi: 10. 10.1098/rspa.1935 1935.009990000..0 coyalsocietypub. shinngg weg.ing... orgg doi / //as///s// 1000100.. 100...1988/80 098/rspa....... 1935 .35.0085
URL https::/
Jan Hermann, Zeno Schätzle, and Frank Noé. Deep-neuual-networr. solution of the electronic Schrödinger equation. Nature Chemistry, 12(10):891-897, October 2020. ISSN 1755- 4349. doi: 10. 101100//41157--00055444444
URL [RLLhttps:/ http:///wwwwwaarreee/mme www.raaaree.... cwm acom/art... ices// Number: 10 Publisher: Nature Publishing Group
John Hertz, Anders Krogh, and Richard G Palmer. Introduction to the theory of neural computation. CRC Press, 2018.
Hohenberg and W. Kohn. Inhomogeneous electron gas. Phys. Rev., (66-884-B8777 Nov 1964. doi: 10. 10311y3//yss 13..ev......8.4... URL htt.::ttp/////mm //link:///kkk.////// aps PhysRev. 136. B864
William Humphrey, Andrew Dalke, and Klaus Schulten. VMD Visual Molecular Dynamics. Journal of Molecular Graphics, 14:33-38, 1996.
Amir Karton, Nitai Sylvetsky, and Jan M. L. Martin. W4-17: A diverse and high-confidence
dataset of atomization energies for benchmarking high-level electronic structure methods. Journal
of Computational Chemistry, 38(24):2063 2075, 2017. doi: https:/dbb.ooggll..00////.2222 URL htRl:ttts:://cc in
w abrrar.rr.r. ley. com do: http://ccccc/////////////.001000 /ttp://..../..0000....../22

---

Under review as a conference paper at ICLR 2022
M. F. Kasim and S. M. Vinko. Learning the exchange-correlation functional from nature with fully differentiable density functional theory. Phys. Rev. Lett.. 127:126403, Sep 2021. doi: 10. htwp://hhyyseLeet........... PhysRevLett. 126433
URL
W. Kohn and L. J. Sham. Self-consistent equations including exchange and correlation effects. Phys. Rev., 140:A1133-A1138, Nov 1965. doi: 10.1 10.1103/PhysRev. 100.:/PhhhRe.....111.... 140.A1133, URL https:
aps /
Siu Kwan Lam, Antoine Pitrou, and Stanley Seibert. Numba: A Ilvm-based python jit compiler. In Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC, LLVM "15
New York, NY, USA, 2015. Association for Computing Machinery. ISBN 9781450340052. doi: https:/s::/dot org/10 1145/2833157 2633166
10.144/2833157:28331662
Susi Lehtola. Assessment of Initial Guesses for Self-Consistent Field Calculations. Superposition of Atomic Potentials: Simple yet Efficient. Journal of Chemical Theory and Computation, 15
())15533-6044 March 2019. ISSN 1549-9618, doi: 10. 10.1021/aaa 1/acs//c.cccc/ccccoooo//ooo wat.aacsccc...o000... 8b01089. URL htthtt:::
//g/1.0rg/10 1021/ accp////.sccaaeeeee/e jetc. 8b01089 Publisher: American Chemical Society.
Li Li. Stephan Hoyer, Ryan Pederson, Ruoxi Sun, Ekin D. Cubuk, Patrick Riley, and Kieron Burke. Kohn-sham equations as regularizer: Building prior knowledge into machine-learned physics. Phys. Rev. Lett., 126:036401, Jan 2021. doi: 10.1 10.1100/hysseel.ett. 126.036401. URL https:
7/link. aps. www//oogoooooooooooooo http///o//..../ 1tt.1/..113PPphP ht..///yyseeeeettt.mm 126. 036401
1103/1
Richard M. Martin. Electronic Structure: Basic Theory and Practical Methods. Cambridge Univer-
sity Press, 2004. doi: 10.1017/CB098805110007
Erkki Oja. Simplified neuron model as a principal component analyzer. Journal of mathematical biology, 15()):677-27,, 1982.
Erkki Oja and Juha Karhunen. On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix. Journal of Mathematical Analysis and Applications, 106(1), )): 69 84, February 1985. ISSN 0022-247X. doi: 10. 10.101660022-247X(555991111
URL https:
www. www. www.scceeneedirect..cmm wttp:///csaaooeee....caaa///////
David Pfau, James S. Spencer, Alexander G. D. G. Matthews, and W. M. C. Foulkes. Ab ini-
tio solution of the many-electron schrödinger equation with deep neural networks. Phys. Rev. Research, 2:033429, Sep 2020. doi: 10.1 Re
URL https:
7/1ink aps www.app.......o.......
http://....10///yyteeeeeeeeeeere 2.33
Péter Pulay. Convergence acceleration of iterative sequences. the case of scf iteration.
Chemical Physics Letters, 73(2):393-398, 1980. ISSN 0009-2614.
doi: https://dii.or
10.1016/00099-224440000666666 https:: // /www.w.w..
3riicie/pi1/00095544888333
Terence D. Sanger. Optimal unsupervised learning in single-layer linear feedforward neu- ral network. Neural Networks, ((6):459-477 1989. ISSN 0893- doi: https:/doi..og!
10.1016/0893 00010016608936600006899990000000 6080(89)90044.00. URL (tRL:h/ts:/ //t::///wwwwww. www.ssecieeecee...c..m.. wwwprrettttccceeeeeeeeeeeeeeeeee commsscience.
Qiming Sun, Timothy C. Berkelbach, Nick S. Blunt, George H. Booth, Sheng Guo, Zhendong Li, Junzi Liu, James D. McClain, Elvira Sayfutyarova, Sandeep Sharma, Sebastian Wouters, and Garnet Kin-Lic Chan. Pysef: the python-based simulations of chemistry framework. WIREs Computational Molecular Science, 8(1),e1340, 2018. doi: https:://oi. http://dbbo.rg//1..000/wwwmm 1340.
URL httpt:t/:/// / // wellinnellle wwweeiibrrr.....eee wi wiley. com/ httm//oo/abb//inn 10. 1002 wcms 1340
Attila Szabo and Neil S Ostlund. Modern quantum chemistry: introduction to advanced electronic
structure theory. Dover Publications, Inc., 1996.
Cheng Tang. Exponentially convergent stochastic k-pca without variance reduction. In H. Wallach, H. Larochelle, A. Beygelzimer, F. V'Alcé-BBuc E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Asso- ciates, Inc., 2019. URL ht h htwp://eeedddag..neiii........ 13ff http:///ocaaa..............prrap
10

---

Under review as a conference paper at ICLR 2022
JH Van Lenthe, R Zwaans, Huub JJ Van Dam, and MF Guest. Starting scf calculations by superpo- sition of atomic densities. Journal of computational chemistry, 27(8):926-933) 2006.
Chao Yang, Juan C. Meza, and Lin- Wan-ang Wang- A constrained optimization algorithm for total en- ergy minimization in electronic structure calculations. Journal of Computational Physics, 217(2): 709-721, 2006. ISSN 0021-9991. doi: http::///w..ooo////.0000011/.p.. www.
https::
A PHYSICAL BACKGROUND
In this section, we briefly introduce the physical background of the problem in section 3 to (Szabo & Ostlund 1996. Martin 2004) for more details.
We refer
In computational physics, foundation problem is to solve the Schrödinger equation of a many-body quantum system
H ( E (
where H the Hamilton operator for a system of M nuclei and N electrons described by their coordinates R and rę. With the Born- Noon-Oppenheimer approximation (nuclei much heavier than electrons), we consider the electrons to moving in the field of fixed nuclei, thus the kinetic energy of the nuclei (approximated constant) can be neglected. In this case we write H as
zero) and the repulsion between the nuclei (approximated to be
ZA r RA
H
j=i+1 rr ryl
==1
kinetic energy of the tween electrons and nu-
coulomb attraction be-
repulsion between trons
clei
and (ri, .N) the electronic wave function, which should be normalized (i.e., ( (V) I T (ri, ... ,N (ri, ,N ,drrdr ... drr = 1). For convenience, let operator h(i) 2 V?
Za ΣN 1:: Rt so can be rewritten as H h(i) h(1) +
-.- According to
the variational principle, the problem of finding solution for the ground state energy Eg can be transformed to the following constrained optimization problem
min ( H (M))
s.t. (V/
Since the multi-electron wave function (ri. ,NN) is computationally intractable when N is large, a basic way is to approximate it as the product of N orbital wave function 4/2 (11)))22222) ... vh (PN) satisfying (V//1/)) J (r) = 1, Vi E 1... N, which is called "Hartree approximation". We also expand ()) = Σ Curo(r) as linear combination of K "basis functions" which is fixed and given, in this case our task becomes to determine the value of all Cui so as to determine the approximated wave function. To construct a Lagrange multiplier
L ( H V)
c((1)))))) 1)
we have
N
(V)
V = Σ
Te (ri, (rr,""r,,)))))) (r) ...dr....dry
+ f Vrrr," (N) Σ [rs - ryl V(r1. ,,, ...drN
(1:))))775))) 1
Σ
(F)))()
(15))))
ri ri-ryl

---

Under review as a conference paper at ICLR 2022
N
Σ CuaCos f (1()))())))))))))
ΣE
E
CurCs,CosCoo (21610000)) b vi-ryl (rid))o
jæh U,U,A,00
and
N
N - f ((())
Σ (((1)161)-1)
N CuiCes
Σ
ca
Let Huo = (r))(( (r))), Suv (
( and EuvAo
mi p ( (12) jo. (r (nndddrrdd which are all constants since {oi} are given functions, o
we have
N N
I Σ
Σ
Hur
E Σ CuCCs,CrCCrruuess
M,Y
jti u.v.A.a
Σ E E Cvs Cua
0L
-2 Coi(Huo CN (wew)
dc
}(z1) A,o
Let aL = 0, Vi = 1... .,., muu=.... K and we have
BC
Cos Coi(Hun (Huv
Fruudo) = Σ CorSues Vi M......N.. K...K
5(+1) Ai
Let PAo Ej(40 CxCoj. [Uar(P)lur =Er, ProEuvsas Fuv Hav + [Unn( (P) and diag(E), ,EN), P. P,Un) e RK*k we have the matrix form of the above equation
=C= = SCA
which is a generalized eigenvalue problem where the matrix F to be decomposed is defined by the eigenvectors C.
Actually, there are several improved approximation theories based on the Hartree approximation mentioned above, in which the definitions of P and (P) are different. A most influential one is the Hartree-Fock theory in which the multi-electron wave function is approximated by slater determinant
(1(11)
Xe(a1)
(x1) (x(22)
1
x1
x2(22)
(Z IN)
VN!
(IN) Xe(aN)
X (*()
to conform to the antisymmetry principle V.... "i,"", ,......) = -V(..., Fj..' ..., whose famous representation Pauli exclusion principle. In this way Puo = =2 CryCr (or in matrix form, P TCCT) and [Uer(P))uv = E>,a PAo EuoEo - 2 Ena Pro Enrovs in which an "exchange term" Ex.o Pxo Eudoo is added.
B SELF-CONSE CELF-CONSISTENT FIELD (SCF) METHOD
An initial density matrix Po is generated via heuristics based prior quantum mechanism knowledge, then the generalized
Ceff-Consistent Field (SCF) is a standard method to solve equation |
12

---

Under review as a conference paper at ICLR 2022
digen-decomposition problem F(P SVA, is repeatedly solved to obtain eigenvectors Vi and density matrix P. = V at the t-th iterations, t = =1,2. until P. Pt-1 is less than the convergence threshold. The detailed routine is shown in AAlgoithh 3
To solve the generalized eigen-dccomposition problem. it should be transformed to standard form first. To achieve this, the orthogonalization technique introduced in (Szabo & Ostlund, (996) is applied to eliminate the overlap matrix S. First, find a linear transformation X so that X SX I. There are several ways achieve this, a popular one is named "canonical orthogonalization" which lets X that all eigenvalues of S are positive so there no difficulty of taking square roots. Then let V X-VV (V XV"), we have F(V)XV
(ffag(ss/3)) where U and
all cigenvectors and cigenvalues of S. Note
SXVA. A. Multiply X the left and have X F(V)XVV' = X SXV'A. Let (V) = F we have
F'(V)V' =VA
(9)
which is a standard eigen-decomposition problem.
Algorithm Self-Consistent Field (SCF) method
Input: H, S. Uer(() in equation 1 and equation 2
Converge threshold c.
Output: V*, the solution of equation
Obtain initial density matrix Po via initial guess methods (e.g., SAD or core Hamiltonian).
Find X satisfying X S
t
- 0
while
Pt - P:-1l do Fg t H + (P)
F + FXX
1+t+1
Obtain precise eigenvectors V2 Fi corresponding to the top-k smallest eigenvalues via classical cigen-decomposittonn methods such as QR iteration.
V -XVV
P. 2 T
end while V V,
C PROOF
Proof of Proposition
Proof. V being the solution of equation I means that V contains the generalized eigenvectors corresponding to the top-k smallest generalized eigenvalues of [F(V*), SJ. In Algorithm 1, we denote the converged V in the final step as V", With the orthogonalizattion technique introduced in Appendix B V being the solution of equation is equivalent to
1. V" = X-IV* contains the eigenvectors corresponding to the top-k smallest eigenvalues F"
2. F = =
When V in the iteration of Algorithm 1 is equal to the converged solution V F" XT holds. Since 4o already converged, the one-step decomposition date for V*, For Oja's algorithm equation 3 letting M - F this means (ef, .uf) QR((f + ... oi + r)M > Note that v ... ut are orthonormal guaranteed by the QR decomposition. Considering that the QR decomposition is implemented via the Gram-Schmidt process, starting from = 1, we have
- will produce no up-
Bioj vi Mo Mvj 31 - vj
" Bang viz M [1vō nM53) 1il0i
13

---

Under review as a conference paper at ICLR 2022
22 + nMa [02 + 1912
T
M oilui
oj + nM - [M0] B1
Dil
A3
0y + nMa Mu
1/2
[(o5 n)M ufjof]l normalization in which 31 lof + MM 11 .2 1 + n factors. Therefore, u ... 2% are all eigenvectors of F
To show that .i.... vt cigenvectors corresponding the top-k cigenvalues, mainly follow (Oja Karhunen 1985) and (Hertz et al., 2018). We start from A: 1 to show that 2e should correspond to the largest eigenvalue d of F" to be stable convergence point.
First, for U1, the QR decomposition degenerates to normalization, that is
-' x' of f ((-F')o1 U1
Assuming 1 is small enough, it can be expanded power series of 1). By ignoring O17)) terms, we have
F
e1 + 01 + ndvr, Aoj = (-F')a - (-F'ee1u
While we have converged solution of, we already know that it is an eigenvector but do not know which eigenvalue it corresponds to. In this case, we assume that after series of iterations, es is very close to the eigenvector corresponding to the wthh largest eigenvalue (denoted as v²) of the final converged matrix -F = -x F(uf)X, that is
0 +
where c is a very small perturbation vector. Since we already know that of is an eigenvector, we have o7 n0h
Let M F -X F(t)) and = X( - F = =F - =
M + E. From the assumption the proposition, have Evi O(e2). Then do one-step iteration as follows:
Av1 (-F'))i [o] (-F')oiju
= (M E)v1 - [o] (M + Eyvilv1
M(u" +c) - -((0- +c)TM(u M(u" + +c)](0 +
(M Ev + - - +c) E)(u +c) +c)
-" + Me - T - -D" - Mr*)
ME 2x
" o0) - x
Now we want to analyse the direction of Ar₁ to see whether can conquer the perturbation and converge to of = v stably. We do it by multiplying another eigenvector Ti on the left (ignoring the O term)
Au (M c--2 ( 00 00)8ao Xº ST BT
= -T 2X (cT, )2)6.8
We notice that, 3 > a (i.e., A 2a), there will always exist a direction e so that both 9T and ST AUj are larger than zero. In this case will flip to the other eigenvector thus cannot converge to 1 stably. Therefore, a should be 1 so that of = a corresponds to the largest eigenvalue of F*
as an example, notice that the Gram-Schmidt process not only normalize the eigenvector 12 but also orthogonalize it towards U1, (U2 Avg) 21 (Ax)) 01 and we already know that in will converge to the eigenvector corresponding to the largest eigenvalue. Therefore we can only select other eigenvectors out of y! to analyse the direction of Ary, which shows that 02 will converge to the eigenvector corresponding to the second largest eigenvalue.
For top-k case, take
14

---

Under review as a conference paper at ICLR 2022
D FULL-SOLVER PERFORMANCE BENCHMARK
While SCGLED not targeted to be an extremely precise (convergence threshold less than 10-10), end-to-end solving technique due to its first-order nature, still compare its performance toward traditional SCF methods in a full-solver setting for reference.
We test three full-solving schemes
Independent SCGLED: applying our proposed method, SCGLED, as a full solver without any tranditional SCF methods.
SCF + minao: applying the traditional SCF method with "minao" the initial guess method, which performs best within traditional methods as shown in subsection 5.1 and acts as the default setting in PySCE.
SCF + SCGLED; applying the traditional SCF method with SCGLED as the initial guess method.
with three settings for heuristics convergence acceleration techniques
Vanilla: no convergence acceleration method is applied.
Damping: the damping technique shown in equation is applied in SCGLED or SCF iteration.
the DIS technique (Pulay 1980) is applied in independent SCGLED or SCF itera- tion. Note that DIIS applied when SCGLED is served an initial guess method.
DII::
So excluding vanilla SCGLED in Algorithm whose empirical performance is too poor for bench- marking, we have 8 test methods total be benchmarked. In Table I we evaluate these methods from two aspects
Convergence:
evaluated via the number of successfully solved molecules within the total 160 singlet molecules in W4-17 dataset.
Efficiency: evaluated via the averaged end-to-end time cost of the method along the molecules that are successfully solved.
For the parameter setting, when SCGLED is served
initial method, we set 7 5000 and IF = 50 for Vanilla and Damping cases, which relatively large since their performance are more sensitive to the quality of initial guess. For the DIIS case which relatively not sensitive to the initial guess, we set T = 200 and T = 25. When SCGLED running independently without traditional SCF iterations, the parameter setting stays the same as in subsection 5.2)
Vanilla
Damping
DIS
Independent SCGLED
155 / 574.3 ms
160 657.9 ms
SCF minao
/ ms
159 244.3 ms
160 160.9 ms
SCF SCGLED
/90.3 ms
159 / 207.1 ms
160 / 148.7 ms
Table 1: Benchmark results for three full-solving schemes with three heuristics convergence accel- eration settings. In each cell, the former is the number of successfully solved molecules within 160 test molecules, and the latter is the averaged time cost.
The benchmark result shown in Table and Figure 6) While independent SCGLED works in the full-solver setting. there is still performance gap with other methods that target for full-solving performance. Instead, SCF + SCGLED works best, especially in the Vamilla case that both the convergence performance (unsolved molecules from 19 to 11) and efficiency performance (aver- aged time cost from 332.6 ms to 190.3 ms) are significantly improved. For the Damping and DIIS case, the performance advantages are narrowed since the heuristics convergence acceleration tech- niques themselves become a more dominant factor for end-to-end performance. Also note that, as an
NNot that the time cost grows exponentially towards the size of the molecule, so the distribution of time enst is quite skewed along different size of molecules. For this reason we take the logarithm of the time cost hefore the average.
15

---

Under review as a conference paper at ICLR 2022
160
SCCLED (Damping)
SCCLED (DIIS)
140
#moloccuss that are solvnd)
SCF minao (Damping)
(DIIS)
SCF SCGLED (Varilla)
SCF SCGLED (Damping)
100
SCGLED (DIIS)
40
20
10-2
10a
10:
10
time cost (s)
Figure 6: Full-solver Performance Benchmark.
optimization- based method, SCGLED brings more randomness compared with traditional heuristics initial guess methods that are built on domain knowledge. Therefore, while SCF + SCGLED per- forms better for most of the molecules, its distribution of time costs is wider, shown as curves with smaller slope in Fiiuue 6, and the end of the curve in Damping and DIIS settings correspond to larger time cost compared with SCF + minao.
E RESULT PER MOLECULE
In this section, we provide the description of the full numerical result listed in the supplementary material of this paper, for all the 160 singlet molecules in the W4-17 dataset. For each molecule and method, we list four values as follows:
Energy: the total Hartree-Fock energy of the molecule computed with the method (Unit: Hartree, 1 Hartree = 4.3598 x 10 10 Joule).
Energy error: the difference between the computed energy above and the exact Hartree- Fock energy listed in the second column of the table (expected to be small possible). Note that we only keep decimals due to the space limit, while the converge criterion is stricter, there will be some cases that the error 0.000000 while the number of iterations in the fourth line is not equal to zero.
Wall time: the wall time to run the method for the corresponding molecule.
Convergence: whether the computed result can reach to converged solution with the vanilla SCF iteration shown in Algorithm (cannot reach converged solution via SCF: can reach converged solution via SCF: v: already converged solution: W). If it can reach to a converged solution, we also show the number of SCF iterations it costs. If the number of SCF iterations is equal to O, it means that the method already reaches precise (converged) solution before the SCF iteration.
In the result table, we bold the result it is a dominated result over the baselines (the energy error and wall time both smaller than the best of the baseline methods while the number of iterations is not larger than the best baseline). We underline the result it takes more time than the best baseline while the other target values dominated. The Hame method both the energy error and the convergence performance are very unsatisfactory, even though it is extremely fast.
taken into account here since
The table shows that
out of 160 molecules contain least one dominated result (in bold).
1.55 out of 160 molecules contain at least precise result (marked with underline).
16

---

Under review as a conference paper at ICLR 2022
72 out of 160 molecules contain at least one result that already reached the converged solu- tion (marked with V, the number of SCF iterations is equal to 0) within 10,000 iterations of SCGLED. That is, we already obtained the converged, precise solution. No further SCF iterations are needed.
14 out of 160 molecules are successfully led convergence by SCGLED while all the four baseline methods failed to do
The full table of numerical result can be found in "fulLnumerica..reuu pdd" as a sup- plementary material of this paper.
17


Reviews:
[{"review_id": "kg88-JGeNo9", "reviewer": "Reviewer_sMJ3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Efficient computation of the Schr\\\"odinger eigenvalue problems  is a fundamental problem in computational  physics is certainly  worth to study. This paper made an interesting attempt for approximating the top $k$-smallest eigenvalues and the corresponding eigenvectors of a generalized nonlinear eigenvalue problem associated to the Hartree-Fock approximation. of the Schr\\\"odinger problem. However, from my perspective, the contribution of this paper is incremental and may not be sufficiently novel for ICLR. Here are some concerns and comments. \n\n- First, it seems that the core of the proposed algorithm is the Eigengame for finding the first $k$-PCA of the linear eigenvalue problem, although there are other minor adaptions, such as introducing the damping and momentum. To me, a simple application of an existing method from others to a new problem is not novel enough for ICLR.  \n\n- Second, the authors claim that the proposed \"SCGDLED is robust to initial guesses\". However, this was only demonstrated in empirical results, but not justified rigorously. Can the authors prove the convergence of the proposed algorithm? Note that the Eigengame paper contains solid theoretical convergence analysis. I am wondering if any rigorous analysis can be done here. \n\n- Another question I had is about the original Schr\\\"odinger problem. In the present paper, the authors seeks an  Hartree-Fock approximation of the Schr\\\"odinger problem and considers a finite dimensional discretization of the Hartree approximation. Can the authors comment on whether (and how) the proposed method, especially the idea of Eigengame can be extended to solving the original  (infinite dimensional) Schr\\\"odinger problem without resorting the Hartree approximation? \n\n- In the statement of Proposition 1, is the assumption simply saying that  $F$ is continuous? If so, please  state it in this way. The current statement seems not precise. \n\n- Regarding Appendix D, I do not think it is a good idea to  include a 10-page long table to show the numerical results. I would recommend the authors select some representative data and perhaps leave the rest to some open repository. ", "summary_of_the_paper": "This paper proposes a numerical algorithm for solving a class of  nonlinear generalized eigenvalue problems  that arises from the Hartree-Fock approximation of the Schr\\\"odinger equation. here the focus is on finding the first $k$-smallest eigenvalues and eigenvectors. The general idea is to perform  fixed point iterations (or solve a sequence of linear eigenvalue problems) in the framework of Self-Consistent Fields. To solve for the top $k$-smallest eigenvalues and eigenvectors of the linear problems, the authors replace the classical Oja’s algorithm with the recently developed EigenGame. The advantage of EigenGame is that the time-consuming QR decomposition step in Oja’s algorithm is replaced by a generalized Gram-Schmidt step in the computation of gradient updates. The efficiency of the proposed method is demonstrated in extensive numerical experiments.  ", "summary_of_the_review": "This paper provides an interesting attempt for accelerating the computation of nonlinear eigenvalue problems relevant to the Schr\\\"odinger equation. However, the contribution of this paper is incremental and may not be sufficiently novel for ICLR. Therefore I am inclined to rejecting the submission.  ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "jcop_jcwly4", "reviewer": "Reviewer_pQwL", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nS1. Some experiments have shown that the proposed method outperforms the traditional heuristics-based initial guess methods in terms of accuracy and efficiency. \nS2. Sufficient technical details of the gradient-like eigendecomposition algorithm are included.\n\nWeaknesses :\nS1. The authors just simply apply Oja & Karhunen's gradient method algorithm to solve their nonlinear eigenvalue problem. Such extensions seem incremental.\nS2. No convergence analysis for Algorithm 1 is provided.\n\n\n\n", "summary_of_the_paper": "This paper considers a gradient-like eigendecomposition algorithm for solving nonlinear eigenvalue problems. The proposed method can eliminate the need for domain-specific heuristics for classical methods, it is robust to initialization, free of heuristics designs, and neat in implementation. Some experiments have shown that it is more efficient and effective.\n\n", "summary_of_the_review": "See above.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "RhkQcuoa9uf", "reviewer": "Reviewer_9vsL", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Proposition 1 is confusing. The statement of the proposition itself does not make any sense. What exactly does \"negligible\" mean? The gradient to be 0? If so, then F = 0 everywhere and the proposition is meaningless. The proof is also hard to understand. It seems that the authors are trying to prove something along the lines of \"a stable local convergence point is the solution of equation 1\", but I am not sure.\n\nJudging from the experiment results, we still need to combine the proposed method with the SCF to achieve a good solution. Can the authors give more information on the full solver case? For example, a comparison between SCF and SCGLED. How many of the examples the two algorithms fail to converge? If converge, what is the perfermance?\n\nSome minor problems: Change the symbol to distinguish clearly $V_{eff}$ and $V$.\n\nEquation (2) (3) can be written in the matrix form to be more consistent with the rest of the paper.\n", "summary_of_the_paper": "The paper studies the Schrodinger equations and its related nonlinear generalized eigenvalue problem. It proposes the Self-consistent Graident-like Eigen Decomposition and shows that it achieves better numerical permanance for solving Schrodinger equations.", "summary_of_the_review": "I think both the theoretical and empirical contribution of the paper is limited. I consider this paper to be marginally below the acceptance threshold.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "IMHfTb2dVlA", "reviewer": "Reviewer_2D5z", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper is interesting that the proposed SCGLED can replace traditional heuristics-based initial guess methods with large performance advantages. I recommend this paper to be published after addressing my below comments.\n\nMy comments are as follows:\n\n1.\tFor Equation (3) on Page 4, v_i’ is used, but in the description v_i_t is used. This will cause confusion.\n2.\tFor Figure (4) on Page 8, it only displayed how fast in terms of iterations the SCGLED solves the molecules. I think it will also be interesting to show how fast in terms of absolute time the molecules are solved by 1) SCGLED, 2) SCGLED (with DIIS), and 3) traditional method\n\n\nConclusion: \n\nThis paper proposed a new method to solve approximated Schrodinger equations with gradient-like eigen-decomposition. It shows it not only can simply replace traditional heuristics-based initial guess methods with large performance advantage, but also is capable of finding highly precise solutions independently without any traditional iterative methods. \n", "summary_of_the_paper": "The manuscript entitled “SELF-CONSISTENT GRADIENT-LIKE EIGEN DECOMPOSITION IN SOLVING SCHR¨O DINGER EQUATIONS” presents a new framework, Self-consistent Gradient-like Eigen Decomposition (SCGLED), to solve the approximated Schrodinger equations with gradient-like eigen-decomposition. It regards F(V) as a special “online data generator”, thus allows gradient-like eigen decomposition methods in streaming k-PCA to approach the self-consistency of the equation in an iterative way similar to online learning. SCGLED is shown to be robust to initial guesses, free of quantum-mechanism-based heuristics designs. This paper demonstrates SCGLED can replace traditional heuristics-based initial guess methods with large performance advantages and is capable of finding highly precise solutions independently without any traditional iterative methods.", "summary_of_the_review": "In summary, I recommend its publication in ICLR after my comments are addressed.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a numerical approach to solving the multi-body Schrodinger equation.  Three reviews give low confidence scores and the one review with high confidence, and high score, is very brief and the reviewer appears to have a weak background in this area.  My feeling is that the ICLR reviewer pool does not contain reviewers who are really competent to review this paper.  There is a large literature in the physics community on this problem and the paper should be reviewed in an appropriate venue.  This is especially true for evaluating the empirical results. If the mathematical techniques are relevant to general machine learning, and the authors want to have an impact on machine learning community, then it should be possible to give empirical results on a problem commonly used to evaluate machine learning methods at machine learning venues. Whether or not this is important for physics should be judged by physicists.  In any case, the reviews are for the most part not enthusiastic.
Decision: Reject