Quadratic Quantum Variational Monte Carlo
Authors: Baiyu Su
qiang liu
Venue: NeurIPS 2024 poster (2024)
Keywords: AI for science
Machine learning for physics
Machine learning for chemistry
Quantum physics
Variational Monte Carlo
MCMC
Transformers
Primary area: machine_learning_for_physical_sciences
TL;DR: We proposed Q^2VMC, a method to enhance efficiency and accuracy of quantum variational Monte Carlo methods.

Abstract: This paper introduces the Quadratic Quantum Variational Monte Carlo (Q$^2$VMC) algorithm, an innovative algorithm in quantum chemistry that significantly enhances the efficiency and accuracy of solving the Schrödinger equation. Inspired by the discretization of imaginary-time Schrödinger evolution, Q$^2$VMC employs a novel quadratic update mechanism that integrates seamlessly with neural network-based ansatzes. Our extensive experiments showcase Q$^2$VMC's superior performance, achieving faster convergence and lower ground state energies in wavefunction optimization across various molecular systems, without additional computational cost. This study not only advances the field of computational quantum chemistry but also highlights the important role of discretized evolution in variational quantum algorithms, offering a scalable and robust framework for future quantum research.

Quadratic Quantum Variational Monte Carlo
Baiyu Su University of Texas Austin baiyusueutexas.cm edu
Qiang Liu University of Texas at Austin lqiangocs. utexas. edu
Abstract
This paper introduces the Quadratic Quantum Variational Monte Carlo (Q²VMC) algorithm, an innovative algorithm in quantum chemistry that significantly enhances the efficiency and accuracy of solving the Schrödinger equation. Inspired by the discretization of imaginary-time Schrödinger evolution, QCVMC employs novel quadratic update mechanism that integrates seamlessly with neural network-bart ansatzes. Our extensive experiments showcase VMC's superior performance, achieving faster convergence and lower ground state energies in wavefunction optimization across various molecular systems, without additional computational cost. This study not only advances the field of computational quantum chemistry but also highlights the important role of discretized evolution in variational quantum algorithms, offering a scalable and robust framework for future quantum research.
Introduction
Finding fast and accurate approaches to solving Schrödinger equations is a central challenge in quan- tum chemistry, with far-reaching implications for material science and pharmaccutical development. The ability to solve this equation precisely would unlock a plethora of properties inherent to the microscopic systems being studied. However, the task of deriving exact wavefunctions for even moderately sized molecules is notoriously difficult, with no analytical solutions in general cases.
The advent of deep learning has significantly advanced the field of quantum chemistry, particularly through enhancements the Quantum Variational Monte Carlo (QVMC) method [T33] Enhanced by neural network-based approaches, commonly referred to as neural ansatz, methods like PauliNet and FermiNet [5] have demonstrated remarkable success. These approaches often match or surpass the accuracy of traditional "gold standard" methods such as CCSD(T) [7] even for complex molecules 8 9. This rapid development has spurred a broad spectrum of research into more accurate and efficient neural ansatz models, significantly impacting ab-initio quantum chemistry [2] Recent reviews [3] provide comprehensive overviews of the advancements and diverse applications extending beyond molecular systems to areas like solid-state physics and electron gases 16]
Despite the accuracy and flexibility Quantum Variational Monte Carlo (QVMC), optimizing remains a challenging task, often requiring prolonged convergence times. Various methods have been developed to accelerate training, such as stochastic reconfiguration (SR) 17.19. Newton method 20.. adaptive imaginary-time evolution (21). and Wasserstein Quantum Monte Carlo (WQMC) [2 In our work, we enhance optimization efficiency by employing the perspective of imaginary- time time Schrödinger evolution [3 24), which naturally guides the wavefunction toward the ground state over an extended time horizon. According to McLachlan's 's variational principle, it can be shown that this continuous- cintuuuuu--mmeeeee process yields parametric updates analogous to those in standard QVMC with infinitesimal learning rates 25). However, while theoretically robust, implementing this evolution in practical settings is challenging with finite time steps. Traditional approaches approximate the updates within parametric space, but this method is limited by the non-convex nature of the objective and the unpredictability arising from complex theoretical properties. To overcome these challenges, we propose discretizing the evolution process itself, ensuring convergence to the ground state even
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Algorithm 1 QVMC Q
Require:
Molecule Hamiltonian H. neural ansatz (x(x) of wavefunction parameterized by
Require:
Initial weights lg an optimizer optimizer, and learning rate schedule {ne 1 while not converged do Draw sample
(x(); i= 1,...,,, NN from vig (x) via MCMC. Calculate local energy and loss:
Ex((x0)) = Vo. (x
Σ (x))) [(0.) - N
1
Update model weights via 8p+1 = optimizer((r. A0, F) (see Eq. 19), where
AO
Σ
c6i)
1 N
Σ
cli)
Vo log wo, (x
N
QVMC:
e0 = n (x
VMC
e = F2
2 F (x
end while
return the neural wavefunction vā (x)), and samples {x(()) J==1 si=1 ~0 (x)
with finite time steps. We then project the discretely evolved distribution back into parametric space, forming an update algorithm that iteratively refines the neural ansatz towards the ground state.
Diffusion Monte Carlo (DMC) 28 is a well known method in quantum chemistry that also employs ground state projection. Known for its promising results, DMC often surpasses the limitations of specific ansatz choices 32 However, as con-aaametric approach, DMC offers flexibility and computational efficiency but lacks the ability to provide explicit values of the wavefunction, which can be essential in applications. Additionally, DMC methods encounter the fixed-node approximation fixed trial wavefunction, which cannot
issue: their effectiveness depends heavily on the accuracy be improved during the computation. By contrast, our approach maintains parametric representation of the wavefunction that evolves continuously toward the ground state, effectively sidestepping the limitations posed by fixed-node constraints.
few previous works have similarly focused on projecting an evolved quantum state onto the parametric manifold of an ansatz, as explored in (3]. To the best of our knowledge, all existing approaches rely on conventional projection methods, specifically the quantum fidelity or the Fubini- Study metric. Although these metrics are widely used in physics, their mathematical properties are intricate and remain underexplored (34. Furthermore, none of these methods account for finite step size. In contrast, our approach takes advantage of the fact that wavefunction analysis is primarily conducted through the probability distribution (q x (x)(2) derived from it. Accordingly, we project probability distributions using the Bullback-Leibler divergence, chosen for its mathematical
simplicity and its ability to effeetively capture distributional differences of interest. The introduction of quadratic term naturally emerges from the squared nature of the wavefunction in the probability distribution, while the preconditioning by the Fisher information matrix arises naturally from the curvature of this projection.
Building on this framework, we introduce the Quadratic Quantum Variational Monte Carlo VMC), an innovative optimization mechanism that enhances the conventional QVMC by allowing finite-time updates without additional computational overhead, as detailed in Algorithm T This novel approach not only maintains theoretical equivalence with QVMC under infinitesimally small time steps but also demonstrably achieves twice the optimization speed significantly better accuracy within the same computational budget.

---

2 Results
In this section, we present a brief overview of the results achieved by the Quadratic Quantum Variational Monte Carlo (Q²VMC) method, demonstrating its enhanced efficiency and accuracy in wavefunction optimization. Our method achieves improvements convergence speed and energy accuracy across various molecular systems. Technical details about the relevant experiments is written in the experiments section.
Summary of key results We evaluated the performance of Q2VMC against traditional Quantum Variational Monte Carlo (QVMC) using state-of-the-arr attention based neural network ansatzes: Psiformer []] and LapNet [9]. A total of six different molecules with diverse sizes are tested, with number of electrons ranging from 30 to demonstrate robustness. Each one of the 12 possible combinations are optimized with the default settings as in their original papers where possible and with (our method) or without (baseline reproduce) the quadratic modification. Our findings indicate that Qa VMC not only accelerates the convergence process but also reduces the variance in batch energies, suggesting a more stable approach towards reaching the ground state. These enhancements highlighted as:
As demonstrate by the training curves in Figure 1 Q VMC shows 2x speed-up in optimization comparing with the baselines, achieving target energies in approximately half the iterations required by QVMC.
Faster Convergence:
Enhanced Accuracy:
The energy accuracies obtained are consistently superior to those achieved by conventional QVMC, as detailed in Table of energy accuracies This superior- ity is particularly pronounced in complex systems with a higher number of electrons, where the traditional methods struggle to maintain precision and stability.
Simple Integration and Hyperparameter Robustness: As shown in Algorithm 1 Q2C can be seamlessly incorporated into existing frameworks with only a single line of code change. This section presents results obtained with the original hyperparameters, highlight- ing that effective performance gains are achievable without additional tuning efforts. For completeness, Appendix C provides results from experiments where hyperparameters were adjusted specifically for Q²VMC, showing that these tuned settings achieve comparable or superior performance to the Psiformer (Large) model using the traditional QVMC method, despite the latter's 's use of a network approximately four times larger than the Psiformer (Small) employed here.
Table Energies for a set of molecules studied in Psiformer (8] and LapNet 19]. Reference energies are taken from the respective papers. In order to climinate any potential effeets from different evaluation strategies, we also report our reproduced baseline values in the appendix.
System (Flectrons)
Psiformer
Q*VMC+Psi
LapNet
Q'vMC+Lap
Li2 (6)
14.99486(1)
-14,99490(1)
-1.999485(1)
-1.999486(1)
NHg (10)
56.56367(2)
-56.56374(2)
-56.5635(())
-56.56370(2)
CO (14)
-113322416()
-113.32442(2)
-113.32417(4)
-113.32428(2)
CH₂ (18)
-95.86050(4)
-95.86073(2)
-95.86025(3)
-95.8605((2)
CgHçO (26)
-155.04656(7)
-155.04696(3)
-155.04563(6)
-155,04619(4)
C2Ho (30)
-155.946198)
-155.94665(4)
-155.94528(4)
-155.94618(4)
3 Background
3.1 Quantum Variational Monte Carlo (QVMC)
At the heart of quantum mechanics lies the wavefunction, which embodies all possible classical states of system. When first quantization considered, the wavefunction serves as a mapping from the states of particles to complex amplitudes. For instance, the state of a single electron can be represented by its position x E R3 and spin (t. Consequently, the wavefunction of an
N -electron system is mapping v (R3 x (1) + C, with the square of its magnitude, |
3

---

27910:
-00079
-0.000
-0000
21803
FM0IC TTACE
2001
0311
6.11
Ve- Crane
614
6.2
0.94
750 10001
1900
197300
29353
1000
W000 1EEW 1N000 1750C 2000
29303
1000
1500
3008
INOS (INE3 9A040
0EV35424)
-0.00
13803
5015
7506
DROC
Sig
6:5
0.22
324
325
6.2
021
500n
Figure 1: Optimization curves for different molecules
representing a probability density πw2 = | /C, where C. = The probability density Tg2 represents the likelihood of observing the quantum system in a specific state upon measurement. Note that when the normalization condition is not enforced, wavefunctions d are invariant under scalar multiplication, implying 1b an for any non-zero scalar a, where all such functions correspond to the identical normalized probability density тч. With an abuse of
k [ is the normalization constant.
notation, simply write T&a | when corresponding normalization clear from the context.
The behavior of non-relativistic quantum systems are dictated by the Schrödinger equation, which, in its limeeindependent form, poses an eigenfunction problem HP- = Evb. Here, H represents the Hermitian linear operator known as the Hamiltonian, and the eigenvalue E represents the energy associated with a specific eigenfunction. The Hamiltonian's 's structure is crucial, encapsulating the physical properties of the quantum system. In quantum chemistry, the Hamiltonian generally takes the form of:
1 -- Vxx).
=-
where Vi is the Laplacian coordinate space, and (xx represents a potential function dependent on the particle positions (e.g. configurations of the nucleus).
Quantum Variational Monte Carlo (QVMC) computational approach used to determine the ground state, corresponding to the lowest eigenvalue Ep. of the Schrödinger equation. In the studies

---

of QVMC, two simplifications are commonly employed. First, given that H is Hermitian, its eigen- functions can be considered real-valued, permitting a focus solely on real-valued wavefunctions. Second, the absence of spin variables d; in the Hamiltonian allows for simplification in modeling the system, permitting us fix the spins of electrons and shift our focus solely on their positional values. Hence, in QVMC, we utilize an unnormalized wavefunction ansatz the R -> IR, parameterized by The term neural ansatz denotes the representation of the by a neural network.
The quest for the ground state solution is guided by the Rayleigh-Ritz principle, which involves minimizing the loss function:
(()) = J (x) Hi (x)dx
= 18 100
1 (x) (oo()) 2 Eo.
J 10 (x)( dx
(x)
where (x, del v0 (x) represents the local energy. The gradient of this loss function is computed as
Vol(0) 10 [(F((x) EL) Vo log vo (x))
with EL = (E)))( denoting the average local energy. Minimizing & with gradient descent thus yields iterative process, where Markov Chain Monte Carlo sampling is employed extract samples from distribution |tbs| and the samples are used estimate the energy gradient, which then directs the parameter updates [35].
To improve the efficiency of optimization, the Stochastic Reconfiguration [36 17
Quantum Natural Gradient Descent 37 38 has commonly been adopted for QVMC updates. This method enhances convergence by preconditioning the gradient with an (approximation of) Fisher information matrix F(0) -1 related the quantum state v/2, which can be implemented efficiently using approximate natural gradient frameworks like KFAC [39]] Consequently, the practical parameter update step, considering a learning rate 7), is given as
A0QVMC
4 Quadratic Quantum Variational Monte Carlo
This section introduces methodology for updating the neural ansatz through the V2 approach. In Section 4.1, we present a discretized imaginary-time. Schrödinger evolution. This process operates within the non-parametric Hilbert space of wavefunctions, guiding the system progressively toward the ground state. Subsequently, section 4.2, we discuss how to project the evolved distributions back onto the parametric manifold of the neural ansatz by minimizing the Kullback-Leible divergence between the evolved and updated distributions, which forms the basis of the V2C algorithm.
4.1 Imaginary-Tiin Schrödinger Evolution
Consider Hilbert space equipped with the inner product (u, v) f wv. and spanned by orthonormal basis functions (x)):; Given Hermitian operator H. normalizing its eigenfunctions so that (6.. σb) = 1, results in a basis that embodies three essential attributes: 1) they are eigenfunctions of the Hamiltonian with associated eigen-energies E,, 2) normalized, and 3) mutually orthogonal for distinct indices. These attributes ensure that any function within our Hilbert space can be precisely represented as linear combinations of these orthonormal basis functions associated with the Hamiltonian. The energies of these eigenfunctions
conventionally ordered as Eg < E₁ < Eg < Thus, the primary objective of QVMC is to approximate do. with is associated to the lowest energy Ep. using parametric ansatz vig
The imaginary-tine Schrödinger equation emerges from the time-dependent Schrödinger equation by substituting t' with it. yielding:
(x,(x,t Øt
(x,
(5)
5

---

Considering an initial wavefunction at t 0 as (x, = 0) = Ei=g ag the imaginary-tima time
Schrödinger evolution has closed-form solution expressed in terms of these basis functions:
(x ()
tii
(xii(())
Σ
oge th (x(x).
1=0
Scaling the wavefunction by factor of e reveals the evolution's impact:
(x, t) = (x(x) +
aj e (e:()))) (x))))
(6)
a
Given Fg.- Fo >> O for all > O, as + oo, the wavefunction's 's projection onto any basis other than do diminishes zero. Hence, starting with any wavefunction that overlaps with do. the imaginary-time Schrödinger evolution consistently approximates the ground state as approaches infinity.
While in theory, the operator e e- as t + co can directly yield the ground state function, exact computation of this operator is impractical. One must discretize the time step to incrementally evolve the process. The evolution process can conveniently operate with discrete time in the following manner.
Definition 4.1 (Discretization). For given time step r, the Discretized Imaginary- Schrödinger Evolution, corresponding to a Hamiltonian H and its ground state energy Eg, is described by a series of functions (16(1) ) === such that:
1 - TH vo(n)
1 1 -F(
ch(n+1)
1 EEg (m))
1 TBo
where E/") Hustn) /1b(()).
This process is proven to converge to the ground state:
Theorem 4.2 (Convergence). Assuming (x)(0), (o)) * and | ll2 < oc, then 4b6n) weakly converges to Φo- up constant factor, as - DC.
Remark 4.3. This result differs from its continuous time counterpart as shown in equation 6 The evolution process in our approach does not require infinitesimal time step to converge, thus is insensitive to the size of the time step taken. Asymptotic convergence is guaranteed regardless of the time step size. Consequently, unlike previous methods, our approach does not necessitate the use of very small time steps, which can often impede effective convergence.
The evolution in the Hilbert space of wavefunctions may also be motivated from other perspectives, e.g gradient flow under Fisher-Rao metric (22) and the discrete evolution is similar to the quantum power method in the computational quantum literature in the limit that 7 - oo (see, for example, For more comprehensive theoretical analysis, we direct readers to these sources.
40
4.2
Parametric Projection of the Evolution Process
In practical applications, operating within the infinitediimensional space functions is not feasible. Instead, we utilize neural ansatz Vh parameterized by finite set of parameters 0 to approximate the underlying functional. This necessitates an iterative process: a) evolving the current vi following discrete evolution produce (1-7E,,) Ve, b) projecting the evolved function back into the parametric space to update model parameters, resulting in Va+An and c) updating associated MCMC data samples based on this projection before repeating the process with the updated neural ansatz.
While step a) is straightforward and efficiently implementable (as detailed in Appendix A), step b) requires suitable divergence metric for effective projection. In this study, minimize the Kullback- Leibler (KI.) divergence between the probability distribution induced by the evolved wavefunction and that represented by the updated neural ansatz within trust region [42
Proposition 4.4. Let h(A0) denote the Kl.-divereence between the evolved distribution (1 T EE Ex 6 (x)) and the updated distribution V6+
h(20) K.C [[1 [11- T EE 18
(8)

---

Given the size of trust region e, our objective of projection is
arg (h(A0) s.t. KC(V = 2/2).
(9)
A0
As + 0*, the optimal update direction approaches to
~0 lim -0 Frgg
(10)
g F-1g"
6-0
where 9 and are the gradient and Fisher information matrix respectively:
g = L log (x)] - (E [[ -EL()))) ( log (x)),
F E Vo log e E Vo log vil) (Vo log v
E
V log v0 +38)'
represents the Fisher information matrix associated with the distribution induced by the neural ansatz. All expectations here are taken with respect to the distribution (x))²
Proof. Refer to Appendix B.
Therefore, the update given by the projecting the evolution process, i.e. the Q²VMC update, is like:
ABQavMC a F-19
log log (x) - E (1- Ve log vi (x)])
Because the mean of the scaling factors E [11 Er. (x))² is subtracted from the update, the constant of does not matter. Therefore, the form of update derived solely from the perspective of discretizing imaginary- time Schrödinger evolution and projection match closely with the update of QVMC upon choosing the time step t = 11 except for the quadratic term of 172 E7, (x). Because the term EL (x) has already been computed in the QVMC and the quadratic term can be added on with no effort, our method has no relative computational overhead. Notably by taking the infinitesimal time step limit + 0 will exactly recover the QVMC update as the additional term decays with O(?2), thereby showing the consistency of our method in the small step size regime.
5 Experiments
This section details the experimental setup and evaluation strategy utilized to obtains the results shown above.
Methodology Overview Two recent cutting-edge, attention- based neural ansatzes, Psiformer g and LapNet (9) are tested in our evaluations. The architectural hyperparameters Table in Appendix. Note that g provides two possible model sizes, Psiformer Small and Psiformer Large with the latter roughly 4x size than the former, and experiments are all conducted with the former. To demonstrate the easy integration and robustness of our method, we adhered to all the original training hyperparameters from their publications (detailed in Appendix Table 5). Training curves of baseline and horizontal lines representing reference energies from respective papers are plotted to facilitate comparison and indicate successful reproduction of the claimed performance. The only modification in our QMV experiments pertains to the gradient coefficients, in accordance with the VMC update rule
delineated in
Convergence and Stability Figure presents the energy convergence trajectories for six molecules, demonstrating that VCC achieves both rapid and consistent convergence across range of systems. Specifically, we tested on Lig (6), NH; (10), CO (14), methylamine-CH, (18), ethanol-C2H,0 (26), and bicyclobutane-C,HH, (30), where the numbers parentheses denote electron counts.
7

---

Training and Evaluation Consistent with the methodologies as in the referenced studies, we optimize the models 200,000 training iterations for all molecular systems. Nevertheless, was observed that smaller molecules typically can reach convergence in fewer iterations e.g. Li2 while larger systems like bicyclobutane has clearly not converged yet within the duration. We encourage future benchmarking in this field to select the total iterations adaptively. We adopt similar numerical hacks as in [8] | to facilitate numerical stability. Importantly, the local energies are clipped so that values will be within the range P 5.0 of mean absolute deviation from its median. The coefficients cl0) are then further computed after clipping.
Following the training, additional evaluation conducted over 20,000 steps, employing MCMC to sample batches of data without updating the network parameters. The computed energies for the tested molecules, comparing against benchmark values, are tabulated in Table 1. For smaller molecules like Lig, the energy performance gains were marginal, highlighting the system-dependent aspect of convergence energy. However, our approach facilitated consistently faster convergence across the board. For larger molecules, which do not reach convergence within the allocated iterations, Q2V markedly improved energy performance.
5.1
Ablation Study
The update of VMC can be decomposed into two parts:
AfgavMc = A0QUMC + - [BE EE) v log vx(()
(11)
Therefore, one might suspect that the better performance of Q² VMC comes solely from its larger update magnitude ||0l] 3 |[ and we make better performance of QVMC by utilizing a larger learning rate. Note that the greater relation cannot be confirmed as the terms being added are not in the same direction. In this section, we confirm that the performance of QVMC can indeed be boosted by carefully tuning for a larger learning rate within a specific system. However, one cannot make QVMC perform better than Qa VMC solely by tuning the learning rate.
We primarily study the system of (H, as it large enough to allow different algorithms distinguish while not so large to allow objectives to converge within the 200k steps duration. All experiments in this section are done with the Psiformer model. We take a fine-grained tuning of the learning rate To of QVMC within the range of (0.01, 0.02, 0.05, 0.1, 0.2, 0.5]. The strategy of training and evaluation follows the experiments. The convergence energies are listed in Table 2 and the training curves can be found in Appendix C.
Table 2: Convergence energies of NH₃ trained with QVMC using different fjo-
77
0.01
0.02
0.05
Eoonvrge
-56.56327(3)
-56.56350(2)
-56.56366(2)
70
0.1
0.2
0.5
Feoonvrgg
-56.56372 (1) -56.56379(1)
Diverge
It is clear from the results that with larger learning rates for QVMC, one can yield better convergence energy values, while setting over-large values will make the training diverge, even if the gradient clipping in terms of the Fisher norms are enabled [43]. Among the trainings of different learning rates, the best performance is obtained from the one using learning rate of 170 0.2. We therefore use the matched learning rate for training with QCVMC to see if it can still do better than this. The convergence energy values are listed in Table
Table 3: Convergence energies of NH₃ trained with QCV using different 7/o-
0.05
0.2
Econvegg -56.56374(2)
-56.56384(1)
8

---

Our method is already performing well enough even with the default learning rate of 0.05 not tuned specifically for the system of NH3. Furthermore, since experiments have already shown that the performance of VMC is superior to any trials obtained from optimizing the objective using QVMC, there is no need to further tune the learning rate of Q2VMC for comparison.
Following the heuristics recommended by 10 we further ablated additional hyperparameters, including increased decay time, reduced learning rate, and reduced norm constraints. The results, summarized in Table 4, specify the modified hyperparameters alongside the achieved ground state energies upon convergence. As shown, none of these adjustments matched or exceeded the accuracy attained by Q²VMC.
Table 4: More experiments for ablation study: Computed ground state energies of NH₃ molecules with standard quantum Monte Carlo method. Tested with different (reduced) learning rates, (increased) learning rate decay times, and (increased) norm constraints.
Learning Rate
Decay Time
Norm Constraint
Energy
3e-3
-56553349(3)
le4
le-2
-6,56336(()
3e-2
Diverge
2e-2
3e-3
-56.56369(1)
3e4
lc-2
-56.56366(1)
3e-2
Diverge
3e-3
-56.56371(2)
le4
le-2
-56.56313(3)
3e-2
Diverge
5e-2
3e-3
-56.56374(1)
3e4
le-2
-56.56344(2)
3e-2
Diverge
5.2 Code and Computational Details
All models were implemented using the JAX framework [4]. which is available under the Apache-2.0 License. The architectures were adapted from public implementations of FermiNet [43 and LapNet (55. both of which are also distributed under the Apache-2.0 License. Modifications were made these architectures integrate the V²C algorithm. Natural gradient updates were based on KFAC-JAX [46). adhering to the same licensing terms. For the LapNet experiments, training was conducted on four Nvidia GeForce 3090 GPUs, utilizing standard single precision caleulations and double-precision for matrix multiplications, with training durations ranging from 5 to 90 clock hours depending on the size of the molecule. Similarly, Psiformer experiments were performed single precision four Nvidia V100 GPUs, with each run varying from 8 to 140 clock hours.
Conclusions
In this study, we introduced the Quadratic Quantum Variational Monte Carlo (Q'VMC), which optimizes neural ansatz in quantum variation Monte Carlo by evolving wavefunctions towards the ground state in non-parametric space, then projecting these onto the neural network's 's parametric manifold using KL divergence minimization. Our experiments demonstrate that Q2V not only strengthens the theoretical foundation but also significantly surpasses traditional QVMC updates in speed and accuracy.
Limitations and Future Works While VMC demonstrates clear advantages, it also faces several unresolved challenges, such as determining the optimal imaginary time step and quantifying the inaccuracies introduced by approximate projection methods. Due to current computational constraints, our experiments were limited to systems with to 30 electrons. Similarly, these limitations prevented us from conducting complete set experiments on other important quantum chemistry applications, such as relative energies 47 48 and excited states 59. In future work, we aim to extend

---

QCC to these domains to further assess its performance. Additionally, our method currently achieves only a constant factor speed-up, as observed the experiments, but the fundamental ((N*) scaling with the number of electrons remains bottleneck, restricting its application to very large systems. We hope to address this scaling issue to enable testing on larger molecules.
Broader Impacts Broader impacts of this work could influence computational chemistry, poten- tially reducing the reliance on physical experiments and accelerating the discovery of new drugs and environmentally friendly chemical processes, while adhering to stringent ethical standards.
7
Acknowledgement
The authors thank the four anonymous reviewers for their invaluable discussions and insightful feedback. Their suggestions regarding limitations and challenges were instrumental in shaping improvements to our paper and inspiring directions for future research. The research is conducted in Statistics & AI group UT Austin, which receives supports in part from NSF CAREER 1846421, SenSE2037267, Office of Navy Research, and NSF Al Institute for Foundations of Machine Learning (IFML).
References
[1] William Lauchlin McMillan. Ground state of liquid he Physical Review, 138(2A):A442, 1965.
[2] M Peter Nightingale and Cyrus J Umrigar. Quantum Monte Carlo methods in physics and chemistry. Number 525. Springer Science Business Media, 1998.
[3] Eric Neuscamman, CJ Umrigar, and Garnet Kin-Lic Chan. Optimizing large parameter sets in variational quantum monte carlo. Physical Review Condensed Matter and Materials Physics, 8544)00 2012.
[4] Jan Hermann, Zeno Schätzle, and Frank Noé. Deep-neural-networt solution of the electronic schrödinger equation. Nature Chemistry, 12(10):891889) 2020.
[5] David Pfau, James Spencer, Alexander GDG Matthews, and W Matthew C Foulkes. Ab initio solution of the many-electron schrödinger equation with deep neural networks. Physical Review Research, 2(3):033429, 2020.
[6] James S Spencer, David Pfau, Aleksandar Botev, and W Matthew C Foulkes. Better, faster fermionic neural networks. arXiv preprint arXiv:201 /, 025 2020.
[7] Anton V Sinitskiy and Vijay S Pande. Physical machine learning outperforms" human learning" in quantum chemistry. arXiv preprint arXiv: 1008,001,, 2019.
[8] Ingrid von Glehn, James Spencer, and David Pfau. A self-attention ansatz for ab-initio quantum chemistry, 2022.
[9] Ruichen Li, Haotian Ye, Du Jiang, Xuelan Wen, Chuwei Wang, Zhe Li, Xiang Li, Di He, Ji Chen, Weiluo Ren, et al. Forward laplacian: A new computational framework for neural network-based variational monte carlo. arXiv preprint arXiv:2307.0827 04 2023.
[10] Leon Gerard, Michael Scherbela, Philipp Marquetand, and Philipp Grohs. Gold-standard solutions to the schrödinger equation using deep learning: How much physics do we need? Advances Neurul Information Processing Systems, 35: 10282-10244 2022.
[11] Kyle Sprague and Stefanie Czischek. Variational monte carlo with large patched transformers. Communications Physics, 7(1):90, 2024.
[12] Michael Scherbela, Leon Gerard, and Philipp Grohs. Towards a transferable fermionic neural wavefunction for molecules. Nature Communications, 15(1):120, 2024.
10

---

[13] Jan Hermann, James Spencer, Kenny Choo, Antonio Mezzacapo. W Matthew C Foulkes, David Pfau, Giuseppe Carleo, and Frank Noé. Ab initio quantum chemistry with neural-network wavefunctions. Nature Reviews Chemistry, 7(10):692-709, 2023.
[14] Gabriel Pescia, Jannes Nys, Jane Kim, Alessandro Lovato, and Giuseppe Carleo, Message- passing neural quantum states for the homogeneous electron gas. Physical Review B, 11003):035108, 2024.
[15] Gino Cassella, Halvard Sutterud, Sam Azadi, ND Drummond, David Pfau, James S Spencer, and W Matthew C Foulkes. Discovering quantum phase transitions with fermionic neural networks. Physical Review Letters, 130(3):03440,, 2023.
[16] Xiang Li, Zhe Li, and Ji Chen. Ab initio calculation of real solids via neural network ansatz. Nature Communications, 13(1):7895, 2022.
[17] Sandro Sorella. Generalized lanczos algorithm for variational quantum monte carlo. Physical Review B, 64(2):02512, 2001.
[18]
Sandro Sorella. Wave function optimization in the variational monte carlo method. Physical Review B Condensed Matter and Materials Physics, 71 (1(2))224110) 103, 2005.
[19] Filippo Vicentini, Damian Hofmann, Attila Szabó, Dian Wu, Christopher Roth, Clemens Giu- liani, Gabriel Pescia, Jannes Nys, Vladimir Barga--Calderón Nikita Astrakhantsev, et al. Netket 3: Machine learning toolbox for many-body quantum systems. SciPost Physics Codebases, page 2022.
[20] Julien Toulouse and Cyrus Umrigar. Optimization of quantum monte carlo wave functions by energy minimization. The Journal of chemical physics, 126(8), 2007.
[21] Eimantas Ledinauskas and Egidijus Anisimovas. Scalable imaginary time evolution with neural network quantum states. SciPost Physics, 15(6):229, 2023.
[22] Kirill Neklyudov, Jannes Nys, Luca Thiede, Juan Carrasquilla, Qiang Liu, Max Welling, and Alireza Makhzani. Wasserstein quantum monte carlo: A novel approach for solving the quantum many-body schr)." " odinger equation. arXiv preprint arXiv:2307.07050, 2023.
[23] Patricia Amara, Hsu, and John E Straub. Global energy minimum searches using an approxi- mate solution of the imaginary time schrödinger equation. The Journal of Physical Chemistry, 97(25):6715 6121,, 1993.
[24] Sam McArdle, Tyson Jones, Suguru Endo, Ying Li, Simon C Benjamin, and Xiao Yuan. Varia- tional ansatz-based quantum simulation of imaginary time evolution. npj Quantum Information, 5(1):5, 2019.
[25] Xiao Yuan, Suguru Endo, Qi Zhao, Ying Li, and Simon C Benjamin. Theory of variational quantum simulation. Quantum, 3:191, 2019.
[26] Monte Carlo. Diffusion quantum monte carlo. Computers in Physics, 4:662, 1990.
[27] Ioan Kosztin, Byron Faber, and Klaus Schulten. Introduction to the diffusion monte carlo method. arXiv preprint physics/9702023, 1997. 1997.
[28] Paul RC Kent, Abdulgani Annaberdiyev, Anouar Benali, M Chandler Bennett, Edgar Josué Landinez Borda, Peter Doak, Hongxia Hao, Kenneth D Jordan, Jaron T Krogel, Ilkka Kylänpää, al. Qmepack: Advances in the development, efficiency, and application of auxiliary field and real-space variational and diffusion quantum monte carlo. The Journal of chemical physics, 152(17), 2020.
[29] CJ Umrigar, MP Nightingale, and KJ Runge. A diffusion monte carlo algorithm with very small time-step errors. The Journal of chemical physics, 99(4):2855-2800 1993.
[30] RJ Needs, MD Towler, ND Drummond, P López Ríos, and JR Trail. Variational and diffusion quantum monte carlo calculations with the casino code. The Journal of chemical physics, 152(15), 2020.
11

---

[31] Weiluo Ren, Weizhong Fu, Xiaojie Wu, and Ji Chen. Towards the ground state of molecules via diffusion monte carlo on neural networks. Nature Communicationss 14(1):1860, 2023.
[32] Kousuke Nakano, Sandro Sorella, Dario Alfe, and Andrea Zen. Beyond single-reference fixed-node approximation in ab initio diffusion monte carlo. arXiv preprini arXiv:2402.01458, 2024.
[33] Alessandro Sinibaldi, Clemens Giuliani, Giuseppe Carleo, and Filippo Vicentini. Unbiasing time-dependent variational monte carlo by projected quantum evolution. Quantum, 7:1131,
2023.
[34] Ran Cheng. Quantum geometric tensor cubbiii-study metric) simple quantum system: A pedagogical introduction. arXiv preprint arXiv:1012.13337 2010.
[35] William MC Foulkes, Lubos Mitas, RJ Needs, and Guna Rajagopal. Quantum monte carlo simulations of solids. Reviews of Modern Physics, 73(1):33, 2001. 2001.
[36] Sandro Sorella. Green function monte carlo with stochastic reconfiguration. Physical review letters, 80(20):4558, 1998.
[37] Shun-Ichi Amari. Natural gradient works efficiently in learning. Neural computation, 10(2):251- 276, 1998.
[38] James Stokes, Josh Izaac, Nathan Killoran, and Giuseppe Carleo. Quantum natural gradient. Quantum, 4:269, 2020.
[39] James Martens and Roger Grosse. Optimizing neural networks with cooncckr-ffactored approx- imate curvature. In International conference on machine learning, pages 2408-2417. PMLR, 2015.
[40] Federico Becca and Sandro Sorella. Quantum Monte Carlo Approaches for Correlated Systems. Cambridge University Press, 2017.
[41] Clemens Giuliani, Filippo Vicentini, Riccardo Rossi, and Giuseppe Carleo. Learning ground states of gapped quantum hamiltonians with kernel methods. arXiv preprint arXiv:2303.08900 2023.
[42] James Martens. New insights and perspectives on the natural gradient method. The Journal of Machine Learning Research, 21(():5776-5851, 2020.
[43] David Pfau James S. Spencer and FermiNet Contributors. FermiNet. http::/github..com com/ deepmind//ermmintt 2020.
[44] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs. http://githn:
com com/ hom/google/jaxx 2018.
[45] Ruichen Li, Du Jiang, Haotian Ye, and Weiluo Ren. bytedanceeLLpplet 2024.
Lapnet. https://github.oom
[46] Aleksandar Botev and James Martens.
KFAC-JAX.
https://github. com/
pooglee-deemmin//ffaajjaaay 2022.
[47] Michael Scherbela, Leon Gerard, and Philipp Grohs. Variational monte carlo a budget fine- tuning pre-trained neural wavefunctions. Advances in Neural Information Processing Systems, 36:23902-23920, 2023.
[48] Weizhong Fu, Weiluo Ren, and Ji Chen. Variance extrapolation method for neural- network variational monte carlo. Machine Learning: Science and Technology. 5(1):015016, 2024.
[49] Michael Entwistle, Zeno Schätzle, Paolo A Erdman, Jan Hermann, and Frank Noé. Electronic excited states in deep variational monte carlo. Nature Communications, 14(1):274, 2023.
12

---

[50] David Pfau, Simon Axelrod, Halvard Sutterud, Ingrid von Glehn, and James S Spencer. Accurate computation of quantum excited states with neural networks. Science. 385(6711)xeadn001 2024.
[51] Jinde Liu, Xilong Dou, Xi He, Chen Yang, and Gang Jiang. Calculating many excited states of the multidimensional time-nndepende schrödinger equation using a neural network. Physical Review A, 108(3):032803, 2023. 2023.
13

---

A Details of Computing Local Energies and Gradients
Hamiltonians for Molecules The characteristics of chemical system are encapsulated by its Hamiltonian, which in quantum chemistry typically involves specifying the positions and charges of the atomic nuclei. Under the Born- Bom--Opeenheimer approximation, nuclei are treated as classical particles with fixed positions, simplifying the Hamiltonian to:
1
Z1 X; X1
Z,Z3 T>J X
--
(12)
X,
i>3
Here, V represents the Laplacian operator for the ith electron, and Z, and X, denote the charges and fixed coordinates of the nuclei, respectively. While potential energies are derived straightforwardly. the Laplacian components required for kinetic energy calculations can be efficiently computed using the standard library of JAX [44] or the more advanced Forward Laplacian technique [9].
Computation of Local Energies is often more stable to parameterize the logarithm of wavefunction, represented as fx(x) = log (x), rather than the wavefunction itself. The local energies can be expressed entirely terms of the log wavefunction as follows:
Ex,0(x) ( log (x) log (x)] V(x)
(13)
Here, the gradients and Laplacians caleulated with respect the positions of all particles, for i = 1.... and across all spatial dimensions j = 2, 3. This setup facilitates efficient computation of both the gradient of the log wavefunction with respect to the model parameters V. log ex(x) and in
combined, the overall updates in both the QVMC and VMC methods, as in equations and ??.
B Derivations of various results presented in the paper
Theorem B.1. Assuming (0)10), (o) + 0 and [| || < co, then v6(00) weakly converges to do, to constant factor, as Co.
Proof. Expressing the initial wavefunction in its spectral decomposition form, 4610) = =, a;di the discretized evolution can be written as:
(5(m) Σ as
E
Φi
(14)
Eu
Given that - (E,) - - < 1 for > 0 (negative energies), all coefficients a,"), i diminish
n approaches infinity, while ag sequence weakly converges to ando by definition.
(n) remains constant for all Therefore, the
Proposition B.2. Let h(A0) denote the KL-divergence between the evolved distribution (1 (6) and the updated distribution v/
h(20) = K.C - (EI-)TExx))) 2
(15)
Given the size of trust region e, objective of projection is
A arg (amiin(((AM) (h(20) s.t. KC(V S 2
(16)
20
As + 0+, the optimal update direction approaches to
Ag = lim 1
F F-1
(17)
c-
g FFF-1 F-
where g and are the gradient and Fisher information matrix respectively:
g EVV v3
[L[(1 (((1-1EE(x))))
' [(1 T (x))
14

---

F F log oi E Va log vā]) (Velog - [V log vb)
represents the Fisher information matrix associated with the distribution induced by the neural ansatz. All expectations here are taken with respect to the distribution (x)=
Proof. Following the results from Section 6 of [42]. we establish that for well-defined objective h(0), the optimal update within trust region is given by:
1 F F lim arg min { (MA0) s.t. KL(V)+40llV 9 F- c
(18)
A9
Therefore, the proof left with computing the gradient of the objective and the Fisher information matrix associated with the distribution lool?.
Let Og represent the fixed part of the parameters, and, with slight abuse of notation, let denotes the variables under optimization. We avoid explicit dependency on Og in the notation of local energies (x) for clarity. The objective function h(()) is expressed as:
h(0) = KL [11 TE vi (x) (x)
- / J(1- TEL (x))) (x)dx
(E-TE,,(x)) 2x) 2 (x)
(1 7 TE [x) (x)) (x)
oo (x) ʃ 1 (x)dx dx
log
log
[(1 7 Ex (x)dx
The gradient then computed as:
0h
0e 0=0
(1 T (x)) (x)
2 (x)
log
dx
00
[(1 7 E (x)
(x)dx
J& Ooto o (x)
/
(1 - (x) (x)
1/2
1o00 (x)
log
(x)dx dx
[[1 (x) (x)dx 00
J&
= E V log (x)) - [E [11 T (x))
E [(1 (x) log (x)
The Fisher information matrix, normalized for the distribution, is:
2 J (x)
F= = E
V log J 2े dx
Ve log
E (Vo log 2 E V log (1) VVolog v3 - E (Vo (8) v
C
Additional Experiment Results and Hyperparameters
Table 4 details the network architecture hyperparameters for the neural ansatzes Psiformer and LapNet used in our experiments. Table 5 outlines the training hyperparameters employed for pretraining and optimizing these ansatzes. Table presents the results of our efforts reproduce the baseline energy values, comparing them with the energies reported in the reference papers. These results affirm the consistency of our implementations with those described the original publications.
15

---

Table Neural Network Architecture Hyperparameters
Parameter
Psiformer
LapNet
Determinants
16
Network layers
4
4
Attention heads
4
4
Attention dims
64
64
MIPP hidden dims
256
256
Table 6: Optimization Hyperparameters
Parameter
Value
Training
Optimizer
KFAC
Training iterations
2e5
Batch size
4096
Learning rate at iteration & Initial learning rate Iro
Iro/ + t/to)
0.05
Learning rate decay steps to
le4
Local energy clipping
5.0
Pretraining
Optimizer
LAMB
Pretraining iterations
2e4
Learning rate
le-3
MCMC
Decorrelation steps
30
KFAC
Norm constraint
le-3
Damping
le-3
Momentum
0
Decay factor of covariance moving average
0.95
Table 7: Energies of reproduced values based on our own experiments comparing the reported baseline values as in g and 9
System
Psiformer
Psi Reproduced
LapNet
Lap Reproduced
Lig -14.99486(1)
-14.99488(1)
-14.99485(1)
-14.99486(1)
NH;
-56.56367(2)
-56.56366(2)
-56.56359(2)
-56.563612))
CO
-113.32416(4)
-113.32429(2)
-113.32417(4)
-113.32416(3
CH,NH2
-95.86050(4)
.95.86051(3)
.95.86025(3)
-95.86026(3)
C₂HØO
-155.04656(7)
-155.04667(3)
-155.04563(6)
-155.04561(7)
C,Ho
-155.94619(8)
-155.94618(8)
-155.94528(4)
-155.94535(3)
16

---

Table Energies for molecules tested with Psiformer. The table includes benchmarking values from 88. including both small and large models, where the latter has approximately four times the number parameters as the former. We present the original results from our paper, obtained using
the hyperparameters from [8] (not tuned), well results obtained with tuned hyperparameters. Both sets results are derived from the Psiformer (Small), The results with tuned hyperparameters
for the small model match exceed the accuracies of the benchmarking large model.
System
Psiformer (Small)
Psiformer (Large)
Q2VMC (original)
QRVMC (tuned)
Lig
14.99486(1)
-1..99485(2)
14.99490(1)
14.99949255)
NH3
-56.56367(2)
-56.56381
-56.56374(2)
-56.56386(2)
CO
-113.32416(4)
-113.32466(3)
-113.32442(2)
-113.32469()
2H
-95.86050(4)
-95.86096(3)
-95.86073(2)
-95.86094(3)
C₂H₂O
-155.04656(7)
-155.04759(6)
-155.04696(3)
-155.04740(5)
C₃Ho
-155.94619(8)
-155.94836(7)
-155.94665(4)
-155.9481 (14998111(5))
0313:3
AP-1100)
0.0
061
0,0000
007
0.05-'' VMC
-2.0022
Veronce
-200944
-2.0006
-2.0008
Figure 2: Energy training curves with differ- ent learning rates in ablation studies.
Figure 3: Variance training curves with dif- ferent learning rates in ablation studies.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: All claims made in the abstract and introduction either be found in the Section of main paper, where we introduce out methodology: or Sections and 4 of the main paper where we present our experiment results and relevant technical details.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discussed the limitations of this work in last section of the main paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers penalize honesty concerning limitations.
will be specifically instructed
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
18

---

Answer: [Yes]
Justification: Theoretical results are Theorem 4.2 and Proposition 4.4. Assumptions of both results are provided in the context and proof is in Appendix Section B.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Technical details required to reproduce the main experimental results have been included in Section 5 of the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
19

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
Code has been attached with the submission of this paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis..o cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https /nips
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Details relevant to this study have been included in Section 5 of experimental details. Other choices, e.g hyperparameters of KFAC, are defaults that can found the attched code.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The common sense of error bars does not apply to the study of quantum variational Monte Carlo methods. We have included the variance curves for each set of energy (loss) curves, which should work as error bars in other studies.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
20

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Information is included in Section 5 of the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: httppp::///neuiiie www./psu...../. htcppp.ccppppppe cccppullic/thhicccuueeeeel...e.e
Answer: [Yes]
Justification: Ethics
The research conducted in the paper conforms with the NeurIPS Code of
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: These have been discussed in the last paragraph of the main paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
21

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The models and algorithms presented in the paper pose no risk of misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Citations have been made throughout the paper where appropriate. Licenses have been mentioned in the code and computational details of the main paper.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
22

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, batasets.papersstt paperswi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The submission includes documented zip file of the code used perform the experiments presented in the paper.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
23

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "TovKtV5TJG", "reviewer": "Reviewer_yQzZ", "summary": "This paper centers on quantum chemistry, specifically targeting the ground state of molecular systems. Unlike previous methods that apply approximate natural gradient techniques to the wavefunction, Q2VMC executes natural gradient optimization on the distribution. Experimental results demonstrate that Q2VMC significantly enhances energy performance.", "questions": "See Weakness above.", "limitations": "The authors addressed some of the limitations of the proposed algorithm in the conclusion.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper introduces Q2VMC, which can be easily implemented.\n2. Faster convergence and enhanced accuracy are clearly demonstrated in the experiments.", "weaknesses": "1. You mention Wasserstein quantum monte carlo in your paper. I wonder whether your method can be added on top of WQMC. If so, additional experiments would be great. If not, I would like to see the comparison between WQMC and Q2VMC.\n2. Typo: line 122, \"wavefunctios\" instead of \"wavefunctions\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Weaknesses and Questions\n\n1. We thank the reviewer for pointing out the typographical error. The typo \"wavefunctios\" has been corrected to \"wavefunctions,\" and the entire paper has been carefully proofread to address any other typos.\n\n2. Regarding the integration of our proposed methods with Wasserstein Quantum Monte Carlo (WQMC), the answer is yes. Our proposed algorithm, Q^2VMC, is a versatile parametric projection method that can be combined with any gradient flow, including WQMC. This versatility means that if better gradient flows are developed in the future, they can also be integrated with our method.\n\nTo address the reviewer's request for additional experiments, we have included new results in Table 2 of the overall rebuttal PDF. We compared our method against the standard WQMC baseline using three atoms as in the original paper, and we also included an additional molecule NH$_3$. It is important to note that the baseline WQMC ground state energy values are derived from our own implementation. These values are more accurate than those reported in the original paper because we adhered to a total of 200k training steps, as opposed to the 10k/20k steps used in their experiments, to maintain consistency and avoid confusion. Our results consistently show improvements over the baseline performance, demonstrating the efficacy of our method.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KfOCrhrx8x", "reviewer": "Reviewer_2zvx", "summary": "In this paper, the authors propose the Quadratic Quantum Variational Monte Carlo (Q$^2$VMC) algorithm to enhance the optimization process of Quantum Variational Monte Carlo (QVMC). Unlike the standard QVMC, Q$^2$VMC employs an improved projection method to guide the update direction of the ansatz. The authors conduct experiments on various systems and perform an ablation study to demonstrate the effectiveness of their method.", "questions": "While the authors focus on improved projection step to derive a better optimization algorithm, the reviewer is curious about if more significant improvements could be achieved by choosing a better target function.", "limitations": "The systems studied in the paper are relatively small, so the performance of the proposed method for larger systems remains unknown. Considering the substantial computational resources required for large-scale experiments, the reviewer does not request the authors to provide concrete numerical results on larger systems during rebuttal. However, the reviewer kindly suggests that the authors provide some evidence to imply the effectiveness of their method on larger systems, e.g., using a smaller network or fewer training step to study a large system.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Designing a better optimization algorithm is useful for most research in this field. \n2. Different from some existing improved QVMC algorithms (e.g., WQMC[1]), the method proposed in this paper does not introduce any additional computational burden, making it easy to integrate with existing VMC packages.\n\n[1] Kirill Neklyudov, Jannes Nys, Luca Thiede, Juan Carrasquilla, Qiang Liu, Max Welling, and Alireza Makhzani. Wasserstein quantum monte carlo: A novel approach for solving the quantum many-body schrödinger equation. arXiv preprint arXiv:2307.07050, 2023.", "weaknesses": "1. The primary concern of the reviewer is the significance of the proposed method. Although the authors provide numerous numerical results, the energy differences between Q$^2$VMC and QVMC are usually within chemical accuracy. As shown in the ablation study section, such small energy differences may influenced by the choice of hyperparameters. While the authors try to demonstrate that under most initial learning rate choices, the performance of Q$^2$VMC is better than QVMC, some important hyperparameters have not been considered for now. For example, Ref.[1] increases the norm constraint of KFAC optimizer and decreases the learning rate decay rate when lowering the initial learning rate. The reviewer suggests a more detailed ablation study to demonstrate the significance of the proposed method.\n\n2. While the proposed update method can be derived from the change of the metric used in the projection step, it is difficult to argue that there is a large difference between the original VMC projection step with the proposed step in Q$^2$VMC. From reviewer's perspective, the original VMC projection, which uses standard inner product in Hilbert space as the metric, may be more reasonable for the quantum systems. The reviewer suggests that the authors should provide a more detailed theoretical analysis about the proposed method.\n\n[1]Leon Gerard, Michael Scherbela, Philipp Marquetand, and Philipp Grohs. Gold-standard solutions to the schrödinger equation using deep learning: How much physics do we need? Advances in Neural Information Processing Systems, 35:10282–10294, 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed comments and suggestions.\n\n## Weaknesses\n\n1. Regarding the improvement in speed and convergence provided by our method, our primary goal was to highlight that researchers can easily integrate the Q^2VMC method into their implementations without hyperparameter tuning, achieving significant speed-ups. We regret that this point was not effectively communicated. Consequently, the hyperparameters used to evaluate our method in the paper may have been suboptimal. We have now fine-tuned the hyperparameters for our algorithm, resulting in much better accuracy in the empirical experiments. Please refer to Table 1 in the global rebuttal PDF. Additionally, thanks to the suggestions from the reviewers, we have tuned some extra optimization hyperparameters of the baseline, including learning rate decay and norm constraint, to demonstrate the consistency of our improvements. Please see Table 3 in the global rebuttal PDF.\n\n2. We believe that the close relationship between our method, Q^2VMC, and the original VMC is a strength rather than a weakness. This relationship enables efficient and easy integration of our method with standard VMC framework, improving results at no additional computational cost. The reason for projecting probability measures using KL divergence instead of the original wavefunction by standard inner product (or fidelity if normalized) is based on the observation that in quantum chemistry, densities, which depend solely on the probability measure $ p \\propto |\\psi|^2 $, are of primary interest. In contrast, the wavefunctions of fermionic systems can be positive in some regions and negative in others, complicating the analysis. Thus, it is more natural, intuitive, and easier to use the neural network to define the probability measures directly. We kindly ask the reviewer what specific theoretical justifications they expect.\n\n## Questions\n\nWe appreciate the reviewer's question about choosing a better target function. However, we would like to request further clarification on what is meant by this. As detailed in Section 4 of our paper, the proposed optimization loop of the algorithm involves first evolving by a discretized gradient flow, then projecting the evolved distribution onto the parametric distribution of the neural network, and finally updating the MCMC samples. Our work focuses partly on the discretization but mainly on the projection step, the second step in this loop. If the reviewer is referring to designing more efficient gradient flows, which is the first step, this is beyond the scope of this work. A better gradient flow can certainly enhance efficiency, but for any novel gradient flow, our proposed algorithm of projection can still be integrated into the loop. For example, in the PDF of the global rebuttal, we demonstrated consistent improvement of our method integrated with the novel flow of Wasserstein gradient flows [1].\n\n## Limitations\n\nWe thank the reviewer for raising the concern about the scalability of our method to larger systems. Scaling up the experiments to test our algorithm on even larger molecules is certainly our goal and the hope of many researchers. However, despite our improvements, the time and computational complexity of QMC remain notoriously high. In our paper, we have tested molecular systems with up to 30 electrons (bicyclobutane), which is already considered quite large among most contemporary works. Following the reviewer's suggestion, we attempted to test our method on a larger system, benzene, which has 42 electrons. Due to resource constraints, we have only completed 160k out of the total 200k optimization steps with Psiformer (Small) by the rebuttal submission time, requiring over 1200 V100 GPU hours. The current inferred energy, averaged over the latest 5000 steps of training with our method, is -232.2412(2), compared to the benchmarking value of -232.2400(1) trained for the full duration. We appreciate the reviewer's suggestion of studying larger systems with smaller networks and fewer training steps. However, scientifically, this would require re-evaluating all baseline systems on the smaller network to reveal the scaling impact. Due to time constraints, we could not complete the full benchmark building yet during the rebuttal, but will definitely add an additional table using a smaller network or fewer training steps to study smaller to larger systems in the camera-ready version once testing is completed.\n\n[1] Kirill Neklyudov, Jannes Nys, Luca Thiede, Juan Carrasquilla, Qiang Liu, Max Welling, and Alireza Makhzani. Wasserstein quantum monte carlo: A novel approach for solving the quantum many-body schrödinger equation. arXiv preprint arXiv:2307.07050, 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "988wR7DJTQ", "reviewer": "Reviewer_77wD", "summary": "This paper proposed a new QMC method that utilizes the imaginary time evolution of the Schrodinger equation. Unlike Diffusion Monte Carlo (DMC) which uses Langevin dynamics to simulate the dynamic of the imaginary time evolution, this paper suggests a way to perform the update in discrete steps and then project back to the parametric space. Interestingly, under the KL-divergence metric, the projection is equivalent to the regular VMC algorithm with an additional term.", "questions": "1. We know that DMC needs importance sampling to achieve good performance. Why is it not the case here?\n2. Do you have an intuitive explanation for why Q^2VMC can perform the ground-state projection as in DMC with only a minor modification to the standard VMC algorithm?\n3. How exactly does your method sidestep the sign problem?", "limitations": "The central issue of scaling of the ab initio method is still not addressed. This is evident as the systems tested are small in terms of the number of electrons contained.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. the author shows that under discretization of neural network parameterization and KL-divergence projection, imaginary time evolution has a similar update rule as the regular VMC. The simplicity of the modification makes the algorithm very easy to adapt.\n2. The mathematical derivation for the key step (Appendix B) is clear and easy to follow.", "weaknesses": "1. DMC should be cited, and the difference between the proposed method and DMC should be highlighted, as early as in the abstract. Also, a comparison with DMC in terms of convergence and speed should be provided. If the author can address this point I'll consider rasing the score.\n2. the improvement in both speed and convergence is a bit marginal. \n3. The presentation flow is a bit strange, which makes the paper harder to understand than necessary. For example, the imaginary time evolution should be in the background section. \n4. There are some typos as well. For example, in line 252 the two sides of the inequality are the same. I'd suggest the authors to proofread their paper further.\n5. There could be more experiments on larger systems to further demonstrate the improvement provided by the new method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed comments and suggestions.\n\n## Weaknesses:\n\n1. We appreciate the reviewer's insight regarding the relation of our work with the widely known Diffusion Monte Carlo (DMC) method. In summary, our method is closely related to DMC in terms of using the Imaginary Time Schrödinger Equation to evolve to the ground state. The key difference is that our method always maintains a parametric representation of the evolved wavefunction throughout the process, as defined by the neural network, and updates the parameters guided by KL divergence projection to track the evolved distribution of the particles (replicas). We have updated our abstract and introduction to emphasize this important relationship, and in the camera-ready version of the paper, we will discuss much more contemporary works in the field of DMC.\n\n2. Regarding the improvement in speed and convergence provided by our method, our primary goal was to highlight that researchers can easily integrate the Q^2VMC method into their implementations without hyperparameter tuning and achieve significant speed-ups. We regret that this point was not effectively communicated. Consequently, the hyperparameters used to evaluate our method in the paper may have been suboptimal. We have now fine-tuned the hyperparameters for our algorithm, resulting in much better accuracy in the empirical experiments. Please refer to Table 1 in the global rebuttal PDF. Additionally, given the current benchmarks' accuracy, achieving even better results within this framework is increasingly difficult. Our tuned algorithm matches the benchmark performance of Psiformer (Large) with Psiformer (Small), where the former has four times the parameters of the latter.\n\n3. We appreciate the reviewer's suggestions regarding the presentation of our paper. While the evolution is background information, we placed it in the following section to intuitively introduce the discretized evolution and its projection after discussing the continuous imaginary time evolution. However, we will make additional efforts to clarify this part in the camera-ready version.\n\n4. Thank you for pointing out the typographical errors. We have meticulously proofread the paper and will further improve its clarity and readability.\n\n5. We aim to scale up our experiments to test the performance of our method on larger systems. However, despite our improvements, the time and computational complexity of QMC remain notoriously high. In our paper, we have tested molecular systems with up to 30 electrons (bicyclobutane), which is considered quite large among most contemporary works. Following the reviewer's suggestion, we attempted to test our method on a larger system, benzene. Due to resource constraints, we have completed 160k out of the total 200k optimization steps with Psiformer (Small) by the rebuttal submission time, which has already required more than 1200 V100 GPU hours. The current inferred energy with our method is -232.2412(2), compared to the benchmarking value of -232.2400(1) trained for the full duration. Full evaluations of more large systems are unfortunately beyond the scope of this work.\n\n## Questions:\n\n1. & 2. The intuition behind our method lies in the gradient expression:\n$$g=\\mathbb E_{|\\psi_\\theta|^2} \\left[(1 - \\tau E_L(\\textbf{x}; \\theta))^2  \\nabla_\\theta \\log \\psi^2_\\theta (\\textbf{x})\\right]$$\nThis expectation can be viewed as performing an importance sampling step to sample from the evolved distribution $(1 - \\tau E_L(\\textbf{x}; \\theta))^2|\\psi_\\theta|^2$ towards the ground state, with the current MCMC samples follow the distribution of $|\\psi_\\theta|^2$. The term $\\nabla_\\theta \\log \\psi^2_\\theta (\\textbf{x})$ represents the gradient of the log-distribution used to update the network parameters to match this importance-sampled distribution. A more rigorous derivation minimizes the KL divergence between the evolved distribution and the current parameter distribution, leading to an expression with only a minor modification to the standard VMC algorithm.\n\n3. The sign problem can refer to various but closely related issues in fermionic systems. In DMC specifically, particles are restricted to their initial nodal pockets during the diffusion process and thus previous works rely on the accurate predictions of the nodal surface (e.g., [1]). In our Q^2VMC algorithm, we maintain a parametric representation of the wavefunction while evolving towards the ground state, allowing the particles to evolve with MCMC updates as in standard VMC. Thus, the sign problem should not complicate our algorithm.\n\n[1] Ren, Weiluo, et al. \"Towards the ground state of molecules via diffusion Monte Carlo on neural networks.\" Nature Communications 14.1 (2023): 1860.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KoybkSe25n", "reviewer": "Reviewer_L9eG", "summary": "This work introduces a new gradient formulation for Variational Quantum Monte Carlo based on the discretized imaginary time evolution of the Schrödinger equation. In their empirical evaluation, their Q²VMC algorithm consistently converges faster and to lower energies than the traditional VMC objective at no additional cost.", "questions": "* \"Remark 4.3:  [...] Asymptotic convergence is guaranteed regardless of the time step size.\" - Does this mean one can pick the learning rate arbitrarily or very large? This does not seem supported by the results in Table 2. What happens for large lr?\n* The author writes that the quantum infidelity in Equation (11) is not well understood. Could the author elaborate on what the concrete issue is there? It looks like the 1 - (cosine similarity)² between the two wave functions, which has previously also been used for instance in enforcing orthogonality between wave functions to obtain excited states [1].\n* How does the objective influence relative energies (typically much more important than absolute energies)?\n* Does the Q²VMC objective also work for excited states?\n\n[1] M. Entwistle, Z. Schätzle, P. A. Erdman, J. Hermann, and F. Noé, “Electronic excited states in deep variational Monte Carlo,”\n\n\nOverall, I do not recommend acceptance due to the issues outlined in its current form. While I believe the contribution to be sufficient, the discussion of related work is lacking. If the authors address my concerns, I am willing to increase my score.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "The paper presents an interesting derivation of a new gradient for the classical objective in VMC. The objective is well-motivated, and the derivation is sound. Further, the empirical results support the theoretical claims of the paper.", "weaknesses": "The paper struggles to distinguish itself from previous works and lacks sufficient discussion of the surrounding literature.\n* For someone unfamiliar with QMC (which is likely at NeurIPS), the paper may seem hard to understand due to missing steps in derivations.\n* The convergence result of the imaginary time evolution is well-known and the basis for diffusion quantum Monte Carlo (DMC) and stochastic reconfiguration in traditional VMC.\n* A proper discussion of related work is missing—for instance, the connection to Wasserstein QMC [1], diffusion Monte Carlo [2] or stochastic reconfiguration [3].\n* \"However, one cannot make QVMC perform better than Q²VMC solely by tuning the learning rate.\" is limited to the evaluation of this work.\n* The notation is unclear at places, e.g., in Eq. (7) it would help to write the explicit dependence on (x) there as E_0 is not a function of (x) but psi and E_L are.\n* While the empirical results are consistent, the improvement generally lives in the region of <1mE_h.\n\n[1] K. Neklyudov et al., “Wasserstein Quantum Monte Carlo: A Novel Approach for Solving the Quantum Many-Body Schrödinger Equation.”\n\n[2] C. J. Umrigar, M. P. Nightingale, and K. J. Runge, “A diffusion Monte Carlo algorithm with very small time‐step errors”\n\n[3] S. Sorella, “Generalized Lanczos algorithm for variational quantum Monte Carlo”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Weaknesses\n1. We appreciate the reviewer’s detailed feedback and suggestions for improving our paper. We acknowledge the challenge to balance the introduction of quantum background concepts for machine learning audiences, and the strict page limit. For the camera-ready version, we have provided a more comprehensive introduction to the background while make explanation of the convergence of the (discrete) evolution more concise.\n\n2. We acknowledge the reviewer’s concern and will include a thorough discussion of Diffusion Monte Carlo (DMC) and stochastic reconfiguration, and more articles will be cited. In particular, our method is closely related to DMC as constantly evolving towards the ground state with Imaginary Time Schrodinger Equation, but keeps a parametric representation; and Q^2VMC is related to stochastic reconfiguration (SR) in terms of how Fisher information matrix naturally arises in the projection. We will also discuss how our method can be seamlessly integrated with Wasserstein QMC(WQMC), providing results showing that our methods can also improve the performance of WQMC. Please find the global rebuttal for experiment details. \n\n3. We have conducted more evaluations and ablation studies to demonstrate the consistent improvements offered by our algorithm. For example, as suggested by other reviewers, we have included combined tuning of learning rate decay and norm constraint of KFAC.\n\n4. To address the presentation issues, we have meticulously proofread the paper and will further improve the clarity and readability in the camera-ready version.\n\n5. Regarding the empirical results and performance of our method, our primary goal was to emphasize that researchers can easily integrate the Q^2VMC method into any of their own implementations without any hyperparameter tuning and achieve significant speed-ups. We regret that this point was not effectively communicated. Consequently, the specific hyperparameters to evaluate our method in the paper can be suboptimal. We have now fine-tuned the hyperparameters for our algorithm, resulting in much better accuracy in the empirical experiments. Please find the global rebuttal for details. Additionally, given the accuracy of the current benckmarks, it will be increasingly difficult to make results better within the framework. Given the size of the system and the absolute values of the \"exact\" energies in the benchmark, it may not be possible to improve some of the accuracy beyond 1mE_h.\n\n## Questions\n\n1. In Section 4.1 and Remark 4.3, we have clearly stated that our discussion is limited to the nonparametric space of functions. The ability to take a large time step in the Hilbert space unfortunately does not imply the same in the parametric space of neural networks, which depends on the intricate loss landscape and stochastic variance of gradient estimation. Consequently, training a neural network with very large learning rates can lead to divergence, as expected. Greater parametric updates are certainly possible, by e.g. making an inner loop to update parameters after a large step in Hilbert space. However, our primary experiments found this to hinder performance evaluated by optimization time.\n\n2. Quantum infidelity measures the difference between functions, allowing for projections. Details on the definition and usage of quantum infidelity in projections can be found in [1]. The issue with these measurements is that infidelity values are defined on wavefunctions, which can be positive or negative in different regions, complicating the evaluations. In quantum chemistry, we are primarily interested in densities, which depend solely on the probability measure $p\\propto|\\psi|^2 $. Thus, it is much more natural to work directly with the divergence between probability measures, avoiding complications from wavefunctions. This framework is more flexible and allows us to use the KL divergence or any other f-divergences for projection, as in our proposed algorithm.\n\n3. We agree that relative energies are crucial. However, such computations are known to require high computational complexity. Therefore, evaluating an algorithm on ground state energies is a widely adopted practice for assessing performance. Following the reviewer’s suggestion, we conducted experiments on the ionization potentials of several atoms using the LapNet network and were able to finish the testings with atom V within the rebuttal period. The results are: the atom's energy is -943.8785(3), the ion's energy is -943.6417(2), and the ionization potential is 0.2368(4), compared to the benchmark value of 0.2361(2) [2] and experiment value of  0.23733 [4].\n\n4. We appreciate the reviewer’s suggestion regarding excited states. Our algorithm can, in principle, be used for computing excited states. For example, combined with recent work [3], our algorithm can be adapted with a simple modification of the gradient of the objective function as defined in eq(9). We have added a separate paragraph to discuss some recent works on excited states computation in the related works section. However, thorough justifications and complete experiments to test the performance for excited states are beyond the scope of this paper. \n\n[1] Giuliani, Clemens, et al. \"Learning ground states of gapped quantum Hamiltonians with Kernel Methods.\" *Quantum* 7 (2023): 1096.\n[2] Li, R., Ye, H., Jiang, D. et al. A computational framework for neural network-based variational Monte Carlo with Forward Laplacian. Nat Mach Intell 6, 209–219 (2024).\n[3] Pfau, David, et al. \"Natural quantum Monte Carlo computation of excited states.\" *arXiv preprint arXiv:2308.16848* (2023).\n[4] Balabanov, Nikolai B., and Kirk A. Peterson. \"Basis set limit electronic excitation energies, ionization potentials, and electron affinities for the 3d transition metal atoms: Coupled cluster and multireference methods.\" The Journal of chemical physics 125.7 (2006).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## Global Rebuttal

We thank the reviewers for their thorough and insightful comments. We have carefully considered all feedback and made substantial revisions to address the concerns raised. This global rebuttal is intended to introduce the additional data provided in the accompanying one-page PDF, which contains three tables to further demonstrate the effectiveness and robustness of our proposed Q^2VMC method.

### Table 1: **Benchmarking and Hyperparameter Tuning Results**

This table included the energy values for a set of molecules tested with the Psiformer model, both in its small and large configurations. The benchmarking values from [1] are included for comparison. We present results using the original hyperparameters from [1] and our results with tuned hyperparameters, both with the Psiformer (Small) model.

- The table shows that the results with tuned hyperparameters for the small model with out method match or exceed the accuracies of the benchmarking large model.
- This addresses the reviewer's concerns about the significance of improvements and the potential impact of hyperparameter tuning on performance.

### Table 2: **Integration with Wasserstein Quantum Monte Carlo (WQMC)**

This table presents the ground state energies for a set of four molecules, optimized using the Wasserstein Quantum Monte Carlo (WQMC) method [2] and combined with our Q^2VMC method.

- The results demonstrate that our method consistently improves upon the WQMC baseline, validating the robustness and efficiency of our approach when integrated with other existing gradient flows.

### Table 3: **Additional Ablation Study**

This table provides results of some extra experiments with additional tuned hyperparameters conducted as part of the ablation study, as suggested by the reviewers. It shows the computed ground state energies of NH_3 molecules using the standard quantum Monte Carlo method, tested with different (reduced) learning rates, (increased) learning rate decay times, and (increased) update norm constraints.

- It addresses concerns about the potential influence of hyperparameter choices on the baselines and demonstrates that one cannot make standard VMC more accurate than our proposed Q^2VMC algorithm by only tuning the hyperparameters. 

### Conclusions

We believe these additional results comprehensively address most of the concerns of reviewers, demonstrating the significance, robustness, and ease of integration of our Q^2VMC method. We have made clarifications and provided additional insights in the revised manuscript to ensure a better understanding of our contributions. We are confident that our revisions and the new data will meet the reviewers' expectations and illustrate the value of our work.

We thank the reviewers again for their constructive feedback and hope our responses satisfactorily address their concerns.

Meta-review: This paper introduces a novel algorithm for quantum variational MC based on minimising the KL divergence of the wavefunction probability before and after a projection step towards the ground state. The method is a small modification of the standard VMC algorithm and the paper presents benchmarks to show its benefits. The rebuttal addressed some weaknesses in terms of hyperparameters and presentation as well as well as comments on the relation to DMC and WQMC.
Decision: Accept (poster)