Online PCA in Converging Self-consistent Field Equations
Authors: Xihan Li
Xiang Chen
Rasul Tutunov
Haitham Bou Ammar
Lei Wang
Jun Wang
Venue: NeurIPS 2023 poster (2023)
Keywords: Self-consistent Field Equation
Computational Science
Online PCA
TL;DR: We developed a new algorithm based on online PCA to converge Self-consistent Field Equations

Abstract: Self-consistent Field (SCF) equation is a type of nonlinear eigenvalue problem in which the matrix to be eigen-decomposed is a function of its own eigenvectors. It is of great significance in computational science for its connection to the Schrödinger equation. Traditional fixed-point iteration methods for solving such equations suffer from non-convergence issues. In this work, we present a novel perspective on such SCF equations as a principal component analysis (PCA) for non-stationary time series, in which a distribution and its own top principal components are mutually updated over time, and the equilibrium state of the model corresponds to the solution of the SCF equations. By the new perspective, online PCA techniques are able to engage in so as to enhance the convergence of the model towards the equilibrium state, acting as a new set of tools for converging the SCF equations. With several numerical adaptations, we then develop a new algorithm for converging the SCF equation, and demonstrated its high convergence capacity with experiments on both synthesized and real electronic structure scenarios.

Online PCA in Converging Self-consistent Field Equations
Xihan Li Xiang Chen', Rasul Tutunov Haitham Bou Ammar", Lei Wang', Jun Wang'
University College London Huaaei Noah's Ark Lab Huawei R&D U.K.
Institute of Physics, Chinese Academy of Sciences
{xihan 1i, j.ng.jnnn))) &cs. www.ucl.......o ac.uk
(xiangchen.ai, rasul. tutunov hai tham armar))huawei.c.ommmm wangleiGiphy ac. cn
Abstract
Self-consistent Field (SCF) equation is type of nonlinear eigenvalue problem which the matrix to be aigen-decomposee is function of its own eigenvectors. It is of great significance computational science for connection to the Schrödinger equation. Traditional fixed-point iteration methods for solving such equations suffer from non non-convergence issues. In this work, we present novel perspective such SCF equations as a principal component analysis (PCA) for non-stationary time series, which a distribution and its own top principal components are mutu- ally updated over time, and the equilibrium state of the model corresponds to the solution the SCF equations. By the new perspective, online PCA techniques are able engage in to enhance the convergence of the model towards the equi- librium state, acting as a new set of tools for converging the SCF equations. With several numerical adaptations, SCF equation, and demonstrated its high convergence capacity with experiments both synthesized and real electronic structure scenarios.
then develop new algorithm for converging the
Introduction
In this work, we are concerned with the convergence issue of solving the following type of nonlinear eigenvalue problem:
F(u)u Au
in which F(v) is given mapping from N N-dimensional unit vector to N N positive semi- definite matrix, is the largest eigenvalue of F(v), and is the normalized eigenvector of F(v) corresponding to A. The core challenge of this problem is the involvement of self-consistencyy, that is, the form of the eigenvalue equation (i.c., the matrix F(v) to decomposed) is defined by its final solution v, but the solution itself is not directly accessible unless the form of the equation is given, which is a paradox. Such type of nonlinear eigenvalue problem is especially concerned in computational science, since
variations of Equation (I) like Hartree-Fock equations 10.11 and Kohn-Sham equations [15] [9 are the foundation of electronic structure calculation by approximating the Schrödinger equation, in which F(v) corresponds to the Fock operator approximating the Hamiltonian operator of a quantum system, and the eigenvector corresponds to the coefficients of an Such equations are usually called "self-consistent
orbital wave function under certain basis 33 25
field (SCF) equations".
However, like many other types of nonlinear equations, existing numerical algorithms solve SCF equations with theoretical convergence guarantee, while many works are committed to improve the practical "successful rate" for the solution to be converged. To be more specific, the iterative
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

method to solve SCF equations is generally referred to self-consistent field (SCF) method [3, 23 whose basic idea is regard the SCF equation as a fixed-point equation f(e) so as to perform fixed-point iteration. For Equation ( we can rewrite it as Fig( F(r)) in which Hig(-) returns
the eigenvector of a matrix corresponding to its largest eigenvalue. Then the fixed-point iteration is performed by generating an initial solution to and performing 2/+1 = Fig( F (wz)) iteratively until convergence. Note that there is no theoretical guarantee that the above iteration step leads to converged solution In practice, vanilla SCF method also easily fails in real electronic structure calculation under Hartree-Fock or Kohn-Sham equations, acting as oscillating between two or more different states that are not solutions of the equations [4. To mitigate the convergence issue of SCF method, existing works mainly follow two directions: one is to generate better-quality initial solutions Vo in eemiiemiiiical way [4 35 22], another one is to mix F(U)) with those in previous iterations
F(ve-1), F(re-2).. ... to stabilize the iteration process 28
16 6].
In this work, we propose different direction to converge SCF equations. We have an insight that the essence of SCF equation a special type of eigen-decomposition where the matrix to be decomposed is not determined during the decomposition. In this way, it shares key similarity with principal component analysis (PCA) in a non-stationary time-series environment, as PCA is also rooted on iigendecoomposttinn and the covariance matrix of the data distribution is not determined as it may change over time. By transiting to the PCA perspective, it is possible to apply a series of online PCA techniques that are successfully developed to handle PCA in non-stationary time-series (or streaming) environments. They are able to adapt to new patterns in the data stream dynamically by incremental updates of the principal component. Connecting it with SCF equations, such an incremental feature is potentially helpful in mitigating the convergence issue, especially by preventing long-range oscillation between successive iterations. Motivated by these insights, we propose a dynamic PCA model with an auto-encoder structure, whose equilibrium state is the solution Equation (T. In this model, we view the eigenvector in the SCF equation as a principal component for a certain data distribution ((æ; E) in PCA, and interpret F(v) in the SCF equation as a "reconstruction function" that convert a vector to the covariance matrix Σ of certain distribution. Then utilize online learning techniques to lead the dynamic PCA model towards equilibrium state.
For some important real-world applications of SCF equations, such Hartree-Fock and Kohn-Shin equations for electronic struction calculation, we also show that our PCA-base model can be adapted to these applications, and proposed several numerical adaptations converge these more complicated equations. Particularly, we proposed an adaptive mechanism that allow the iteration to switch between online mode for convergency and regular mode for accuracy, which provides unlimited chances of trials to reach a converged trajectory, which is lacking in standard SCF methods when stuck in oscillation.
Experimental result on synthesis and real scenarios shows that our proposed approach largely reduce the occurrence of oscillation, leading significant improvement of convergence performance. In this way, our work expands the reach of online PCA methods into handling self-consistency. In summary, make the following contributions:
novel formulation of the SCF equation as finding the equilibrium state of a dynamic PCA
model for non-stationary time series.
A new application of online learning techniques to the proposed model so as to enhance its convergence to the equilibrium state and avoid oscillation, improving the convergence performance of SCF equations solving in a generic way.
A new algorithm based on online PCA with several numerical adaptations, which is capable of converging the SCF equation in real-world electronic structure calculation with high successful rate.
Extensive experiments on a synthetic problem and real datasets for electronic structure calculation, demonstrating the high capacity of the proposed algorithm in converging the SCF equation.
'To have an insight, consider that the mapping Eig(F(-)) R + R is generally not contraction mapping that the Banach fixed-point theorem [2] does not work here.

---

2
Related work
With the prosperity of deep learning and differentiable optimization in recent years, there works combining machine learning with computational science and electronic structure calculation, including deep-learning-aiee
wave function representation quantum Monte Carlo methods such as and neural representation of the exchange-correlate
functional However, these works stay within traditional formalism of the problem, focusing more on improving simulation accuracy towards physical reality by using neural networks better functional approximators. while our work's 's focus very different, stressing on machine learning oriented formalism of the SCF equation, which has not been explored in the literature to the best of our knowledge.
FermiNet (27), PauliNet [13] and (3]
in density functional theory 24
To converge SCF equations, existing works mainly follow two directions. One to generate better- quality initial solutions vo [14] 35 (22. However, these methods are generally semi-empirical as they require specific assumption of F, and leverage domain knowledge (e.g., quantum mechanism) for the initialization. another one is mix F(v)) with those in previous iterations F(v--1) 1); (()-2))
to stabilize the iteration process [28 21 16, 6). While these methods perform well on converging SCF equations efficiently, they can still be stuck in indefinite oscillating between non-solution states
without the chance of escaping from it.
Principal component analysis (PCA) is fundamental, well-studied tool used for data analysis and compression [7). The principal components, which are "representative directions" of the data distribution that preserve the data's variation, can be computed by aggnndecooppositoon of the data covariance matrix. However, when the data are formed as an online, non-stationary stream whose distribution (or more specifically, the covariance matrix) may shift over time, specialized online learning techniques are required to estimate the top-k principal components in a real-time manner, and dynamically adapt new patterns in the data stream. Starting from Hebb's rule in neuroscience focus on this direction named Online k-PCA.
[12] a series of works [26 34
3 Solving SCF Equations with Online PCA
To mitigate the convergence issue in the solving of the SCF equation (T. we connect it with PCA by proposing a new PCA model for non-stationary time series, showing that the solving of the SCF equation is equivalent to finding the equilibrium state of the proposed PCA model. this way, online learning techniques for PCA can be exploited to accelerate the convergence. We also propose some numerical adaptations so to solve real-world SCF equations in electronic structure computation.
3.1 The Connection between SCF Equations and PCA
Idcally
Σ>
Inpat Distribution
Recoustzucted Distribution
P(x;
Compresssoo (PCA) vin
Reconstruction
P(x:
F(v)
Principal Components
Figure 1: SCF equations finding distribution that is invariant before and after being processed by an auto-encoder structure involving PCA. An input distribution Par; Σ) is compressed yielding set of principal components Vi.n- Those are then used to produce a reconstructed distribution P(r:; E'), where ideally we would like Σ Σ'
To connect the SCF equation with PCA, notice that Equation also be stated as Big(F(())
or equivalently
Σ F(iig( E))
(3)
3

---

Time step r
Time step
Time step c+1
Distribution P(::
Compression (PCA)
Compression (PCA)
Compression (PCA)
Reconstruction
Reconstruction R1 + F(v))
I + P(P.-))
Principal Component
Figure A diagram of the dynamic PCA model, in which a distribution P(x; Σ) and its top principal component V1. are mutually updated over time. Importantly, we notice that we can regard the approach as applying PCA on non-stationary distributions that evolve over time. Given that such formalism is novel and is an outcome of SCFs, the machine learning literature is yet to devise effective solutions. In this work, we take the first steps in devising such algorithms and demonstrate successful applications in SCFs.
where Eig(.) is the eigenvector of matrix corresponding its largest eigenvalue, and v Eig(2) (.).
Here, if we consider the function D = (v()) as a "reconstruction function" that convert a vector the covariance matrix Σ of certain data distribution ((:; Σ), then the function e Eig(E) is equivalent to a "compression function" that finds the top principal component v of this distribution P(æ; Σ) with PCA, since the top principal component exactly the eigenvector of the distribution's covariance matrix corresponding to the largest eigenvalue. From this perspective, the solving of Equation 3 (and equivalently (2) can be seen as finding a distribution (ar; Σ) parameterized by the covariance matrix Σ that is invariant before and after being processed by the following two stages:
Compression (PCA): perform PCA the distribution P(r; Σ.) to obtain its top principal component v, which is the representative direction of the distribution. Reconstruction (F))) process v with the given reconstruction function Σ' F(v) so as to obtain the reconstructed distribution E'r
which is shown in Figure While its architecture seems similar to the autoencoder 220 the encoder (PCA) and decoder (reconstruction function F(r)) here are both fixed without any adjustable parameters. Instead of training the encoder and decoder, here we aim to find input distribution P(x; E) which remains invariant after been "encoded" and "decoded"".
3.2 A New PCA Model for Non-stationary Time Series
For solving of the SCF equation 2). traditionally we perform fixed-point iteration by letting the
fixed-point mapping (v) = Eig( F (w)) and performing Ut+1 = f(v)) iteratively until convergence. Here, by replacing f with the reconstruction and compression stages mentioned above, we obtain an equivalent form the fixed-point iteration, which presented as a dynamic model that distribution P(x; Σ) and its top principal component are mutually updated over time, shown in Figure 2 The evolution process of the dynamic PCA model is as follows:
Given initial top principal component Up reconstruction function F(v)
For t 1, 2, ... until converge (1|0; F2-1 c)
Reconstruct the distribution P(x; Σ,) by Σ, = F(v..1).
Perform PCA on X, ~ P(:; E,) and obtain the top principal component v <- PCA(X.).
In this model, the top principal component is updated by performing PCA on the current distribution, and the distribution is updated by performing reconstruction the current top principal component. As it is equivalent to the fixed-point iteration, the equilibrium state of the new dynamic model is also the fixed point of Equation 2 which corresponds the solution of Equation (I.
An important feature of the dynamic PCA model is that, can be regarded as applying PCA on non-stationary distribution over time, the distribution that is processed by PCA is subject change

---

RCA
Time Kop
Figure 3: Convergence comparison between vanilla fixed-point iteration and Online PCA given
F(v) t2 "i : we set the initial solution as [1/2, v The principal component at time step is set to be 122 v 3/2]" for both models. While the vanilla fixed-point iteration oscillates between [1/2, v3/2]" and [v3/2, 1/2] along with the time step, the dynamic PCA model proposed in Section 3.2 converges to the ground-truth solution [v2/2, 2 steadily after applying online PCA technique.
n2 11
over time in the evolving process. This new perspective enables a variety of PCA techniques for non-stationary environments be applied on the solving of the problem, whose details are discussed below.
3.3 Online PCA for Converging SCF Equations
As have discussed in the introduction, vanilla fixed-point iteration method has serious convergence issues. acting as oscillating between two or more different points, neither which are the solution
v2 v102 if we set the initial solution on as of Equation 2 For example, given F(v) 21 of
[L/2, /3/2] and perform fixed-point iteration = Fiv(F(())))). we will find it oscillating between 11/2, T and 1/2], along with the time step. Neither of them are close to the analytical solution [v2/2, 22/2]T. However, the new PCA-based perspective allows us to apply online learning techniques to mitigate this issue. In online PCA, principal components are usually updated in incremental manner so to adapt to new patterns in the data distribution. The incremental property is especially appealing to us, since the oscillation of fixed-point iteration behaves infinite "jumps" between different states, and incremental updates by adding soft restrictions to the difference of principal components between successive time steps. After applying online learning techniques, the evolution process of the dynamic PCA model is as follows:
reduce such jumps
Given initial top principal component Up, reconstruction function F(v)
For / = 1, 2,2,... ... until converge (I|v 1/ <
Reconstruct the distribution P(x; 2x) by E. F(vr-1).
Sample I; from P(x; )x)
Update by online PCA with Z₁
and an illustrative example is shown in Figure
3.4 A Case Study for the Convergence Behavior of Online PCA
For the iterative methods solving Equation (I) while the convergence analysis is generally intractable due to the arbitrariness of F(U) (usually nonlinear by involving vv' ") here provide case study for a specific form of Fw)) whose analytical ground truth solutions are available. While vanilla fixed-point iteration method doesn't work in this case, we can derive and visualize the convergence behavior of our proposed online PCA method in an analytical way. Here we let
F(v) = (Au) Avv" A
(4)
in which is an orthogonal matrix. Substitute Equation into T and we obtain
(Ave A ) Xu
(5)
The analytical solution of Equation (5) the normalized eigenvector of corresponding to the eigenvalue 1, since the largest eigenvalue of matrix Ave A is with corresponding eigenvector
5

---

A
V
V
R
A
(a) Updating online PCA
(b) Convergence dynamics (c) Convergence probability for 0 E (m)
w.r.l.
Figure 4: Convergence analysis of our proposed online PCA method for solving Equation 55) Note that vanilla fixed-point iteration can never converge in this case.
Ae (i.e., Eig(Avn TAT) Av), reducing Equation
standard eigenvalue problem Av.
However, The vanilla fixed-pon iteration method cannot converge when solving Equation (5). Note that the mapping each iteration will be reduced ut+1 Avt, which seems very similar the power method on A for finding the eigenvector with the largest eigenvalue, but with convergence ratio A = (i.e., cannot converge) since all the absolute value of A's eigenvalues are 1 as A is an orthogonal matrix.
Now turn to the convergence analysis of our proposed online PCA method for this case. For the sake of visualization, we further restrict to be three-dimensional as a rotation transformation matrix around the 2-axis with angle 8, that is
cos sin
sin 0
0
=
cos
0 1
0
0
so that the solution of Equation 55 will be v* = 00, O, =1] the two unit vectors on the z-axis. Then, we let the initial top principal component vp be a unit vector on the yz plane
[0, cos so. sin soo]T :P E [0, 2
where yPo is the initial angle between the vector and the ay plane.
We analytically derived the convergence behavior of the online PCA method described in Section 3.3 for solving Equation (5). whose detail is leaved the appendix. The result is as follows: for 0 E (0, ). the online PCA method is guaranteed to converge to the ground-truth solution on z-axis. However, for E : T), there is phase transition of convergence behavior w.r.t. the initial angle
Po. shown in Figure 4b, While the online PCA method stays converged for 4o > arccos / x1 (0), it will fail to converge otherwise. Assuming the initial vector is sampled uniformly on the unit sphere, the probability of convergence for the online PCA method w.r.t. the rotation angle is shown in Figure 4c
3.5 Numerical Adaptations for Converging Real-world SCF Equations
Now we investigate more complicated and applicative scenario in electronic structure calculation, which is usually known as Hartree-Fock equations or Kohn-Sham equations. The form of such SCF equations 33 is
=
(8)
where
F(V): an N x N real symmetric matrix to be decomposed, as given function of V. Is defined as follows:
F(V) H VVVV)
in which H is an N x real symmetric matrix and Uer(-) is an N x real symmetric matrix as function of 2VVT. Both of them are given in the equation.
S: an N N positive semi-definite matrix, which is a constant input in the problem.

---

A: A = diag A1. A&) a x k diagonal matrix containing the top-k smallest eigenvalues.
V: V [U..... 2 an N matrix containing & normalized column eigenvectors
corresponding to the top-k smallest eigenvalues.
Our proposed model can be adapted to handle it follows
1. While F()) is not guaranteed to be positive semi-definite, note that we can always make an "eigenvalue shift" towards F(v) by adding it with 0 (a is moderately large number) to make positive semi-definite, without changing its eigenvectors (i.e., the solution of Equation (8).
2. For S + I, Equation (I) becomes F(v)u AS which is generalized eigen- decooposooooiioonn problem 8 We can then perform standard orthogonalization technique that transforms the generalized eigenvalue problem into standard one [33]
3. To tackle the smallest eigenvalues instead of the largest ones, we can simply add a minus sign to F(r), that the order of its eigenvalues will be reversed without changing its eigenvectors.
4. For le > 1, our proposed model can be trivially extended to k > cases by retaining top principal components rather than one.
And we name the adapted model as "Online SCF". Then, we introduce some practical improvements that can further enhance the efficiency and convergence capability of our proposed model for solving Equation 8
Adaptive Update Interval: Instead of updating Σ in every iteration, we control the update interval of via parameter Ty to improve efficiency. Since only updated slightly each iteration step with small learning rate it may not be necessary to do a fresh computation of Σ F(V) in each time step, especially considering that the computation of Uer(.) in Equation ( can be costly. Moreover, we change Iy adaptively in the iteration to improve the efficiency by the following intuition: if there is significant change between M, and 11- our dynamic PCA model in an unstable state and we should apply online PCA method in more responsive way to improve the convergence by setting a smaller /5. However, if the gap is small, then our dynamic PCA model is very likely to be already close to stable state, in which case the precision of PCA result is more dominant for obtaining a highly precise solution (especially for electronic structure calculation requiring error 10 10) and larger Iy is more reasonable. In this way, set by to be inversely proportional to the difference between Σ,, and E,- as I/S = A in which A AE,, E,-1) 2 0 is a properly
scaled function evaluating the difference between successive E.
Adaptive PCA Mode Switching: Note that, the larger Iy: is, the more costly in time between two updates of Σ, and the closer the result is to regular PCA on P(x; E). Thus cut-off value Teut is set that the PCA model will switch from online to regular when 15, > T offt. avoiding exhausted iteration on a single Σ. Moreover, this brings the preciseness needed for the final stage of the iteration when the error is small and convergence may not be problem. Empirically, such mode switching will introduce temporary disturbance for a few iterations, so we also set a small "tabu tenure" D prohibiting switching back to the original PCA mode in T Taba steps. Such switching between online and regular PCA can be triggered multiple times. A high amount of switches indicates that the convergence of the iteration may be hard, so Teur-olf will increase by a small value Teur-off-ine after each switching to increase the proportion of online PCA to handle the convergence issue.
F Tisbu
Additionally, we also applied the direct inversion of the iterative subspace (DIIS) method, which is empirically effective in traditional methods for electronic structure calculation. The basic idea is to update Σ,, at iteration í linear combination of matrices in previous T iterations Eq-T, E,-1, whose detail can be found in (28) and the appendix. In the experiments involving electronic structure calculation, we apply DIIS on all tested methods.
Some other numerical adaptations, including the application of momentum and sample-free update, also leaved in the appendix. Summarizing all considerations above, we propose algorithm for solving Equation (8), named "Adaptiiee Online SCF", shown in Algorithm
"This term is borrowed from Tabu Search 9
7

---

Algorithm Adaptive Online SCF for solving Equation ( in electronic structure calculation
Input: H, S, U-((-) in Equation ( and ( learning rate difference evaluation function
(E,, E{-1) for convergence criteria and A2(E,; 24-1) for the computation of Iy, cut-off
threshold T cut-off and increment value T cut-offfine- tabu tenure Tubuu convergence threshold
Output: V* the solution of Equation
1: Initialize V/g R randomly.
2: 2o f
3: Find X satisfying X XTSX- I.
4: $o + -Online
5: f +-0,it0
6: while A1 Ex-1) > do
7:
V2 XV;
8:
24 f- DIIS(E--.. F(-1,F(V))
9:
2/ XT
10:
I. [1/A2(Ir: E
11:
if I3 I Teur-olr and not (st Regular and < Tubu) or (st = Online and < Tubuu then
12:
for t' 0, 1, ... I do
13:
Use online PCA to update V/' towards "covariance matrix" Ey with momentum. end for
14:
15:
8t c Online
16:
else
17:
Use regular PCA to compute "principal component" Vi from "covariance matrix" -b% 9; Regular
19:
end if
+ 1 s = St-1 else 0 Teur-olf + Teu if st St-1 else Teut-olt + Teut-otT-ine
21:
++1
23:
end while
V* XV
Experiments
4.1
Experimental Verification the Case Study
To verify the convergence analysis in (Section 3.4, we conduct Monte Carlo experiments to solve Equation (5) for rotation matrix A with different rotation angle 0. For each angle, we uniformly sample 1,000 initial vectors Vg the unit sphere, perform the online PCA method and traditional fixed-point iteration (vanilla SCF) starting from these vectors to solve Equation (5. and use the proportion of converged vectors approximation of convergence probability. To be aligned with the next section, we also included the DIIS method described in the appendix that is extensively used in solving Equation
The result shown in Figure While vanilla SCF fixed-point iteration) cannot converge on any instance, the convergence probability of online PCA is aligned with the analytical result shown in Figure 4c DIIS achieves around 40% convergence probability regardless of the rotation angle, which significantly better than vanilla SCF but still fall behind for 0 < 3x /4 compared with online PCA method. The result validates the convergence analysis in Section 3.4 showing the potentially high convergence capacity of online PCA method.
4.2 Convergence Capability Evaluation on Electron Structures
In this section, we perform extensive benchmarks on the QM9 dataset
diverse, large dataset 31 29
evaluate the capacity of converging Equation (8) the SCF equation the scenario of electronic structure calculation. The QM9 dataset contains the atomic coordinates of 133,885 molecules in total, which is huge size, thus we sampled 1% the dataset (1,338 molecules) a purely random manner for our evaluation. The effective potential matrix function Uan(.) ()) in Equation 9 is based on Hartree-Fock theory and Density Functional Theory (DFT) with B3LYP exchange-correlation functional, provided by PySCF [32.. We evaluated the methods the two theories separately.

---

0.76
0.8
(b) An example of convergence trajectories
(a) Convergence ratio
Figure 5: Performance evaluation solving Equation 5) whose analytical solutions are available. (a) While vanilla SCF (fixed-point iteration) cannot converge any instance, the convergence probability online PCA is aligned with the analytical result shown in Figure 4c| (b) An example of convergence trajectories, traditional SCF (fixed-point iteration) fails to converge (cycling around the solutions) while our Online PCA method converges to the solutions (red and blue arrow) smoothly.
Fortree-Fook
DFT with B3LYP
Methods
#Noonconverged
Average #(iterations)
#(Nonconverged
Average #(iterations)
molecules)
molecules)
Regular SCF
124
(9.27%)
25.49
407
(30.42%)
21.09
Full Online SCF
13
(0.97%)
584.68
217
(16.22%)
1835.24
Adaptive Online SCF
(0%)
42.97
(0%)
60.58
Table Results on 1,338 randomly sampled molecules in QM9 dataset. All methods are initialized with core Hamiltonian and accelerated by DIIS. Average #(iterations) is for converged molecules only.
Standard 6-31G basis set [5] are applied for the computation of all molecules. All tested methods are initialized with Hamiltonian and accelerated by DIIS. Three methods evaluated follows:
Regular SCF: The default method PySCF with DIIS enabled, similar to most of the
quantum chemistry software.
Full Online SCF::Algorithm T without adaptive mode switching mechanism. Online SCF
is applied throughout the whole iteration process, with a learning rate of 10-2. ". To avoid the explosion of update interval T3. when approaching to convergence, we simply set an upper
limit of 10,000 for It.
Adaptive Online SCF: AAlgorithm I iincluding the adaptive mode switching mechanism. Regular SCF is allow to kick in when the iteration process is close to convergence. Teurofff and
Teat-olFine are set to be 100 and 10 respectively. Tubu is set to be 10.
The result is shown in Table Compared with regular SCF approach, full online SCF method significantly reduced the number of nonconverged molecules both Hartree-Fock and DFT scenarios, demonstrating its high convergence capacity in solving Equation update rule online methods results comparatively low precision. This not only increases number of required iterations significantly, but also restricts it from achieving higher convergence capacity under strict convergence criteria. The adaptive mode switching mechanism successfully resolved the issue. By allowing regular SCF to kick in at later stage, with the flexibility return back to online mode when oscillation occurs, adaptive online SCF achieves converged solution for all test molecules, with moderate increase of average iteration number.
(8). However, the gradient-like
The behavior of adaptive online SCF is shown in Figure 6 While most of the molecules get converged with only one mode transition in Figure 6al there are also few "hard cases" like Figure 6b) that require multiple mode transitions between online and regular SCF. The detailed statistics is shown in Table 2 The capability of mode switching is essential for the convergence capacity, since it provides unlimited chances of trials reach a converged trajectory, which is lacking in regular SCF methods with only a few choices of starting point to select.

---

104
109
101
10
10
10
10
Outine 5037
Culise SED
Onlic SCT
10
10
Usloe SCF
Hegsla SCF
10
10
Regalr
Regakir
Repako
10
10
1.0
(a) Converge with one mode transition.
(b) Converge with multiple mode transitions.
Figure 6: Examples of converge curves for adaptive online SCF.
#(mode transition)
2
Hartree-Fock
1143
179
13
2
0
1
DFT with B3LYP
1025
189
60
27
18
6
6
3
Table 2: Mode transition statistics of adaptive online SCF on sampled QM9 dataset. While all molecules finally converged on both Hartree-Fock and DFT scenarios and most of the them only require one mode transition from online to regular, the distribution of mode transition for DFT is more long-tailed tailed (the 4 molecules with #(mode transition) > 8 have 12, 13, 15 and 32 transitions respectively).
Conclusion
In this work, we take the first steps in devising PCA-based algorithms for converging non-linear equations, and demonstrate successful applications in SCFs. This work contributes both the field of computational science and machine learning follows:
For computational science, this work presents new algorithm to converge SCF equations in electronic structure calculation with high successful rate, especially without any heuristics based on prior quantum mechanism knowledge bootstrap the solving stage.
For machine learning, this work explores brand new area of "self-consistent" eigenvalue problems, especially SCF equations, for online PCA methods such as Oja's algorithm and EigenGame, which are previously regarded specialized methods for A-PCA. While such methods can properly handle data stochasticity, this work shows that they are also capable of handling self-consiste leff-consistency, which leads to a potential of application in broader field of scientific computing.
References
[1] Zeyuan Allen-Zhu and Yuanzhi Li. First efficient convergence for streaming k-pea: A global, gap-free, and ncar-optimal rate. In 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS), pages 487-492, 2017.
[2] Stefan Banach. Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales. Fundamenta Mathematicae, 3(1):133-133) 1922.
[3] Thomas D. Barrett, Aleksei Malyshev, and A. I. Lvovsky. Autoregressive neural-network wavefunctions for ab initio quantum chemistry. Nature Machine Intelligence, 4(4):351-358, April 2022. Number: Publisher: Nature Publishing Group.
[4] Eric Cancès, Mireille Defranceschi, Werner Kutzelnigg, Claude Le Bris, and Yvon Maday. Computational quantum chemistry: A primer. Special Volume, Computational Chemistry, volume 10 of Handbook of Numerical Analysis, pages 3-270. Elsevier, 2003. ISSN: 1570-8659,
[5] R. Ditchfield, W. J. Hehre, and J. A. Pople. Self-consistent molecular-orbital methods. ix, an extended gaussian-type basis for molecular-orbital studies of organic molecules. The Journal of Chemical Physics, 54(2):724-728, 1971.
[6] Alejandro J. Garza and Gustavo F. Scuseria. Comparison of self-consistent field convergence acceleration techniques. The Journal of Chemical Physics, 137(5):054110 August 2012.
10

---

[7] Ian M. Gemp, Brian McWilliams, Claire Vernade, and Thore Graepel. Eigengame: PCA as a nash equilibrium. In 9th International Conference on Learning Representations, ICLR 2021. Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[8] Benyamin Ghojogh, Fakhri Karray, and Mark Crowley. Eigenvalue and Generalized Eigenvalue Problems: Tutorial, arXiv:1903. 1224 [cs, stat], March 2019. arXiv: 1903. 11240.
[9] Fred Glover and Manuel Laguna. Tabu Search, pages 2093 2093-2229. Springer US, Boston, MA, 1998.
[10] D. R. Hartree. The wave mechanics of an atom with a non-coulomb central field. part i. theory and methods. Mathematical Proceedings of the Cambridge Philosophical Society, 2411):89-110, 1928.
[11] Douglas Rayner Hartree and W. Hartree. Self-consistent field, with exchange, for beryllium. Proceedings the Royal Society of London. Series A Mathematical and Physical Sciences, 150(869):9-33, 1935.
[12] D. O. Hebb. The organization of behavior; neuropsychological theory. The organization of behavior; a neuropsychological theory. Wiley, Oxford, England, 1949. Pages: xix, 335.
[13] Jan Hermann, Zeno Schätzle, and Frank Noé. Deep-neural-networf solution of the electronic Schrödinger equation. Nature Chemistry, 12(10):891-88) -897, October 2020. Number: 10 Pub- lisher: Nature Publishing Group.
[14) Roald Hoffimann An extended hückel theory. hydrocarbons. The Journal of Chemical Physics, 39(6): 1963.
[15] P. Hohenberg and W. Kohn. Inhomogeneous electron gas. Phys. Rev., 136:B864 B871, Nov 1964.
[16] Xiangqian Hu and Weitao Yang. Accelerating self-consistent field convergence with the augmented Roothaan energy function. The Journal of Chemical Physics, 132(5):054109, February 2010. Publisher: American Institute Physics.
[17] Ian T. Jolliffe and Jorge Cadima. Principal component analysis: a review and recent de- velopments. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2065):20150222 April 2016. Publisher: Royal Society.
[18] M. F. Kasim and S. M. Vinko. Learning the exchange-correlatoon functional from nature with fully differentiable density functional theory. Phys. Rev. Lett., 127:126403 Sep 2021.
[19] W. Kohn and L. J. Sham. Self-consistent equations including exchange and correlation effects. Phys. Rev.,
Nov 1965.
[20] Mark Kramer. Nonlinear principal component analysis using autoassociative neural networks. AIChE Journal, 37(2):233-22 1991.
[21] Konstantin N. Kudin, Gustavo F. Scuseria, and Eric Cancès. A black-box self-consistent field convergence algorithm: One step closer. The Journal of Chemical Physics, 116(19):22558821, May 2002. Publisher: American Institute of Physics.
[22] Susi Lehtola. Assessment of Initial Guesses for Self-Consistent Field Calculations. Superposi- tion of Atomic Potentials: Simple yet Efficient. Journal of Chemical Theory and Computation, 15(3):1593-160, March 2019. Publisher: American Chemical Society.
[23] Susi Lehtola, Frank Blockhuys, and Christian Van Alsenoy. An overview of self-consistent field calculations within finite basis sets. Molecules, 25(5):1218, March 2020. arXiv: 1912.12029
[24] Li Li, Stephan Hoyer, Ryan Pederson, Ruoxi Sun, Ekin D. Cubuk, Patrick Riley, and Kieron Burke. Kohn-sham equations as regularizer: Building prior knowledge into machine-learned physics. Phys. Rev. Lett., 126:036401, Jan 2021.
[25] Richard M. Martin, Electronic Structure: Basic Theory and Practical Methods. Cambridge University Press, 2004.

---

[26] Erkki Oja. Simplified neuron model as principal component analyzer. Journal of mathematical biology, 15(3):267-273 1982.
[27] David Pfau, James S. Spencer, Alexander G. D. G. Matthews, and M. C. Foulkes. Ab initio solution of the many-electron schrödinger equation with deep neural networks. Phys. Rev. Research, 2:033429, Sep 2020.
[28] Péter Pulay. Convergence acceleration of iterative sequences. the case of sef iteration. Chemical Physics Letters, (3(2):393-399) 1980.
[29]
Raghunathan Ramakrishnan, Pavlo Dral, Matthias Rupp, and Anatole von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data, 1, 2014.
[30] C. C. J. Roothaan. New developments in molecular orbital theory. Rev. Mod, Phys., 23:69-89, Apr 1951.
[31] Lars Ruddigkeit, Ruud van Deursen, Lorenz C. Blum, and Jean-Louis Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database gdb- 17. Journal of Chemical Information and Modeling, (2(11):2(()))) 1):286442222222 2875, 2012. PMID: 23088335.
[32] Qiming Sun, Timothy C. Berkelbach, Nick S. Blunt, George H. Booth, Sheng Guo, Zhendong Li, Junzi Liu, James D. McClain, Elvira R. Sayfutyarova, Sandeep Sharma, Sebastian Wouters, and Garnet Kin-Lic Chan. Pysef: the python-based simulations chemistry framework. WIREs Computational Molecular Science, 8(1)=e1340, 2018.
[33] Attila Szabo and Neil Ostlund. Modern quantum chemistry: introduction to advanced electronic structure theory. Dover Publications, Inc., 1996.
[34] Cheng Tang. Exponentially convergent stochastic k-pea without variance reduction. In H. Wal- lach, H. Larochelle, A. Beygelzimer, F. "'Aché-Bucc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[35] JH Van Lenthe, R Zwaans, Huub JJ Van Dam, and MF Guest. Starting sef calculations by superposition of atomic densities. Journal computational chemistry, 27(8):226-93) 2006.
12


Reviews:
[{"review_id": "bgLdVgYEft", "reviewer": "Reviewer_h8kr", "summary": "This paper proposes a new method for solving self-consistent field equations - a form of nonlinear generalized eigenvalue problem in which the matrix being diagonalized is a function of the eigenvectors of the diagonalization. These equations are of great interest in quantum chemistry, and are typically solved via fixed-point iteration, but can struggle with the stability of the iterative process.\nThis paper proposes a connection to the Principal Component method (PCA) by viewing the function F(v) as a mapping from a vector to a data distribution, wherein the map F acts as a form of decoder/reconstruction function and PCA itself acts as an encoder/compression function. This formulation is entirely equivalent to the original problem, but allows the use of certain modified online/adaptive PCA methods to stabilize the iterative procedure. \n\nThe authors demonstrate that this method performs superior to vanilla SCF iterations in a specific, theoretically-tractable case study, then apply the method to the more difficult case of solving the Kohn-Sham equations in electronic structure theory. The authors test their method on the QM9 dataset, which contains a large number of molecules for the purposes of electronic structure calculations. They sample 1% of the dataset at random, then evaluate their method on each case in question and compare to the existing SCF interpretation from the PySCF package. They find that their method results in convergence for all molecules considered (compared to 70-90%), while requiring roughly 2-3x more iterations.", "questions": "Is convergence rate a primary limiting factor in Hartree-Fock/DFT calculations in practice? If so, how does this compare with the potential downsides of increased computational cost required by your method? \n\nAm I correct in assuming that the RMM-DIIS method is the one used as a baseline in the paper? If so, was there a reason other methods were omitted from comparisons? Is there a reason these other methods are not applicable to the problem under consideration?", "limitations": "It would be helpful if the authors gave a more complete presentation of the applicability of the proposed approach in the context of the quantum chemistry literature, and the significance of their results in that context.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The proposed method is interesting, and applies machine learning techniques to a foundational problem in quantum chemistry. Improving the accuracy or efficiency of Hartree-Fock/DFT calculations would be highly valuable to the quantum chemistry community, and is thus an interesting area of research.\nThe proposed method performs well on the theoretical case study, and the results in table 1 show a clear improvement in convergence rate over the existing baselines under discussion.", "weaknesses": "It would be helpful to have a better understanding of the significance of these results in the context of quantum chemistry, where they are most likely to be used. The results shown in Table 1 show that the existing baseline achieves a convergence rate of approximately 70-90% depending on the scenario, whereas the proposed approach achieves 100% convergence at a cost of approximately 2-3x the number of iterations. Within the field of quantum chemistry, is failure to converge a significant limitation, and is this tradeoff worth it? While I am not a quantum chemist, my understanding is that DFT calculations are already exceptionally computationally expensive, and significantly increasing the number of iterations required for convergence may be a very severe drawback.\n\nIn addition, it would be good to see a more thorough comparison. The experiments in this paper are only performed on a single dataset, and only compare to a single baseline. While I, again, am not a quantum chemist, my brief review of the literature revealed a number of existing methods seeing widespread use - including RMM-DIIS (is this the one used as a baseline in the paper?) as well as Davidson or Blocked Davidson iterations, and combination methods combining both RMM-DIIS and Blocked Davidson. Unless I am misunderstanding the applicability of these methods to the problem under consideration, it would be helpful to see a comparison to a broader range of baselines, as well as a wider range of datasets. As is, I feel like the comparisons in section 4 are too narrow to provide a compelling case for the proposed approach, but it is entirely possible that I am missing important context from the field of quantum chemistry. I am happy to revisit this if my understanding is incomplete.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive comment. The detailed responses regarding each concern are listed below.\n\nFor the trade-off between convergence and efficiency, note that such a trade-off characteristic can be adjusted in our proposed method by setting different $T_{\\text{cut-off}}$ in L230 of the paper. If a larger $T_{\\text{cut-off}}$ is set, then the portion of Online SCF will be more dominant, which leads to empirically better convergence with the cost of larger number of iterations. If a smaller $T_{\\text{cut-off}}$ is set, then the portion of Regular SCF will be more dominant, which leads to smaller number of iterations for converged molecules, while non-converged molecules are more possible to appear (if $T_{\\text{cut-off}}$ is set to be zero, the the proposed method is equal to Regular SCF). In this work, since we are more focused on the convergence side as the title suggests, we set a relatively large $T_{\\text{cut-off}}$ to boost the convergence performance. However, the parameter can be set differently to fit practical scenarios. If you are aware that the molecule of your interest may be challenging to get a converged solution, you may wish to set a larger $T_{\\text{cut-off}}$ to strengthen the convergence capability of the method, while otherwise you can set a smaller $T_{\\text{cut-off}}$ for efficiency. We will elaborate more about this feature of our method in the revised paper.\n\nFor the comparison, while multiple existing methods exists, most of them are some variations of DIIS technique. An incomplete list includes energy-DIIS, augmented-DIIS, LIST, GDIIS and RMM-DIIS you mentioned, which may behave more efficiently or with better convergency in specific scenarios, and can be good alternatives when standard DIIS fails. However, most of these variations are about including quantum-chemistry-specific information into the DIIS procedure (e.g., energy-DIIS minimizes the Hartree-Fock energy functional, and augmented-DIIS minimizes augmented Roothaan Hall energy function), and these ideas can also be included in our methods. To compare apples to apples, we should also develop corresponding variations of our method from “energy-Adaptive SCF”, “augmented-Adaptive SCF” to “RMS-Adaptive SCF”, which could be a bit too exhausted and quantum-chemistry oriented, and may not be of interest to a majority of NeurIPS audiences. We leave works about how these variations affect our proposed method and DIIS as a future work to be presented to quantum chemistry communities.\n\nFor the dataset in Sec 4.2, we select QM9 dataset mainly for its sufficient challenge to convergence. Since the focus of our work is on the convergence capability, we intend to set the experiment to be challenging enough to differentiate different methods. We actually did experiments on other datasets, however, preliminary results shows that small datasets like W4-17 [1] are not challenging enough for our experiment, as both our methods and the baseline converges very well. We will mention the result on other datasets in the revised paper.\n\n[1] Karton, Amir, Nitai Sylvetsky, and Jan M. L. Martin. “W4-17: A Diverse and High-Confidence Dataset of Atomization Energies for Benchmarking High-Level Electronic Structure Methods.” Journal of Computational Chemistry 38, no. 24 (2017): 2063–75."}, {"review_id": "5eFGnx8XGm", "reviewer": "Reviewer_EC9k", "summary": "In this work, the authors approach solving the Self-consistent Field (SCF) equation from a principal component analysis (PCA) for non-stationary time series perspective. They shows that, the equilibrium state of such an online PCA corresponds to the solution of the SCF equations. By doing so, this work is abled to achieve better convergence compared to the traditional fixed-point iteration methods for solving such equations.", "questions": "- Why only the first Eigen vector?", "limitations": "- This paper focuses on solving one important but rather niche problem.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- As mentioned in the paper, solving self-consistent Field (SCF) equation is of great significance in computational science for its connection to the Schrödinger equation. So proposing a novel approach, to overcome the non-convergence issues of the traditional fixed-point iteration methods for solving such equations, is important.\n- The authors also mentioned that, this is the first steps in devising PCA-based algorithms for converging non-linear equations. So further study in this direction can help solving other such relevant problems.", "weaknesses": "- Please edit line 181 in the manuscript. Some part of the line is omitted by figure 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive comment. \n\nWe will correct the layout error in line 181. The covered sentence is \"where $\\psi_0$ is the initial angle between the vector and the xy plane.\"\n\nThe reason to include only the first eigenvector/eigenvalue in eq (1) is for the simplicity of form, as the main issue of SCF equations -- \"self-consistency\" is remained. In a more complicated case shown in eq (8), we include top-k eigenvector/eigenvalues and discussed this extension in line 211 and 212."}, {"review_id": "eDaztB3dDA", "reviewer": "Reviewer_myiv", "summary": "The paper presents a new online-PCA based algorithm with some additional computational innovations to solve self-consistent systems. They add a mode-switching method and delayed calculation to improve convergence issues. The results are very good, but on a somewhat limited/niche dataset. ", "questions": "Is the methods that is being compared with in sec 4.2 the only other methods that can solve the SCF problems?\n\nAre there any other real-world problems other than electronic structures where this methodology can be used? Non-stationary time series? \n\nIn Figure 2. Is it possible to explain the intuition behind the role of F? It would help the paper to have an idea what the \"typical role\" of F in systems such as these is. (maybe this is not possible, but I still ask...)\n\nWhat is DIIS? I did not see the introduction to this and it seems like it suddenly popped up without a definition. ", "limitations": "The limitations are adequately addressed.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well presented, simple and shows very good results on a task that is seemingly impossible to solve with other methods. ", "weaknesses": "I am not an expert in the application area (electronic structures), but it seems like there could have been more extensive experiments to illustrate the benefits of the method. I don't think sec 4.1 gives a good enough picture of the benefits of the new method.\n\nAlso, the algorithm boxes on page 4 and 5 are slightly confusing and figure 4 is misplaced and is covering up some text it seems (at least in my printed version). ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive comment. The detailed responses regarding each concern are listed below.\n\nFor other methods, while multiple existing methods exists, most of them are some variations of the DIIS technique. An incomplete list includes energy-DIIS, augmented-DIIS, LIST, GDIIS and RMM-DIIS, which may behave more efficiently or with better convergency in specific scenarios, and can be good alternatives when standard DIIS fails. However, most of these variations are about including quantum-chemistry-specific information into the DIIS procedure (e.g., energy-DIIS minimizes the Hartree-Fock energy functional, and augmented-DIIS minimizes augmented Roothaan Hall energy function), and these ideas can also be included in our methods. To compare apples to apples, we should also develop corresponding variations of our method from “energy-Adaptive SCF”, “augmented-Adaptive SCF” to “RMS-Adaptive SCF” to make a fair comparison, which could be a bit too exhausted and quantum-chemistry oriented, and may not be of interest to a majority of NeurIPS audiences. We leave works about how these variations affect our proposed method and DIIS as a future work to be presented to quantum chemistry communities.\n\nWe will correct the misplacement of Figure 4 in the revised paper, sorry for the confusion. The covered sentence is \"where $\\psi_0$ is the initial angle between the vector and the xy plane.\"\n\nFor other problems, we think it may also model the crowd behavior in social science, in which the behavior of an individual will be influenced by the \"representative options\" of the crowd, and the representative options of the crowd will be influenced by averaging all the change of the individuals. Generally speaking, systems with \"mean-field\" characteristics may be benefitted from this methodology.\n\nFor the role of F, while F is the \"input\" of the problem in eq (1) which could be arbitrarily defined, a typical definition of F in the scenario of electronic structure is defined in eq (9) whose computational detail is shown in the line 41 and 51 in the appendix.\n\nFor DIIS, it is an convergence acceleration technique introduced in line 71 of the appendix. Sorry for the confusion and we will revise the main paper to include an introduction of DIIS."}, {"review_id": "For94ISoij", "reviewer": "Reviewer_uFr2", "summary": "This paper explores online PCA methods for a certain type of non-linear eigenvalue problem.", "questions": "Could you please expand the last paragraph of the conclusion section on how this is a contribution to machine learning? ", "limitations": "-", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This is a well-written paper on an interesting problem in computational science.", "weaknesses": "My main critique is that, as presented, the contribution of this paper appears not as much to machine learning or data science, rather to computational science. \n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the review. Since our work lies in the applications of online PCA methods to the otherwise unknown area and the specific self-consistent Eigen problem, our work contributes to the machine learning community by expanding the reach of online PCA methods. Before our work, online PCA methods are regarded as specialized methods to handle the stochasticity issue in online or streaming environment. Our work shows that they are also capable of handling self-consistency issues in solving an important class of nonlinear eigenvalue problem. Additionally, the concept of self-consistency is closely related to mean-field theory, which is an active topic in machine learning. Machine learning and physics are closely connected with mutual concepts such as mean-field and Boltzmann machine, which leads to a potential application of our work on a boarder range of machine learning applications involving mean-field characteristics.\n\nMoreover, according to the CfP of NeurIPS 2023, \n\n> NeurIPS 2023 is an interdisciplinary conference that brings together researchers in machine learning, neuroscience, statistics, optimization, computer vision, natural language processing, life sciences, natural sciences, social sciences, and other adjacent fields.\n\nand we believe that our work fits in the topics of \"Optimization\" and \"Machine learning for sciences\"."}]

Author rebuttal:

Meta-review: There is an overall consensus among reviewers that this is an interesting submission that should be accepted, which is also my evaluation.
Decision: Accept (poster)