Gold-standard solutions to the Schrödinger equation using deep learning: How much physics do we need?
Authors: Leon Gerard
Michael Scherbela
Philipp Marquetand
Philipp Grohs
Venue: NeurIPS 2022 Accept (2022)
Keywords: Computational Physics
Machine Learning for Science
Quantum Monte Carlo
Fermionic Neural Networks

Abstract: Finding accurate solutions to the Schrödinger equation is the key unsolved challenge of computational chemistry. Given its importance for the development of new chemical compounds, decades of research have been dedicated to this problem, but due to the large dimensionality even the best available methods do not yet reach the desired accuracy.
Recently the combination of deep learning with Monte Carlo methods has emerged as a promising way to obtain highly accurate energies and moderate scaling of computational cost. In this paper we significantly contribute towards this goal by introducing a novel deep-learning architecture that achieves 40-70% lower energy error at 6x lower computational cost compared to previous approaches. Using our method we establish a new benchmark by calculating the most accurate variational ground state energies ever published for a number of different atoms and molecules.
We systematically break down and measure our improvements, focusing in particular on the effect of increasing physical prior knowledge.
We surprisingly find that increasing the prior knowledge given to the architecture can actually decrease accuracy.

Gold-standard solutions to the Schrödinger equation using deep learning: How much physics do we need?
Leon Gerard Michael Scherbela Philipp Marquetand and Philipp Grohs
RResearch Network Data Science @ Uni Vienna, Kolingasse 14-16, A-1090 Vienna, Austria
Faculty of Mathematics, University of Vienna, Oskar-Morgenstemn -latz 1, A-1090 Vienna, Austria & Institute of Theoretical Chemistry, Faculty Chemistry, University of Vienna, Währinger Straße 17, 1090 Vienna, Austria Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of
Sciences, Altenbergerstrasse 69, 4040 Linz, Austria
'leon cennngerrddd serrar@@nivii.aee univie.ac.al
"These authors contributed equally
Abstract
Finding accurate solutions the Schrödinger equation is the key unsolved chal- lenge of computational chemistry. Given its importance for the development of new chemical compounds, decades research have been dedicated this problem, but due to the large dimensionality even the best available methods do not yet reach the desired accuracy. Recently the combination of deep learning with Monte Carlo methods has emerged as promising way to obtain highly accurate energies and moderate scaling computational cost. In this paper we significantly contribute towards this goal by introducing novel deep-learning architecture that achieves 40-70% lower energy error at lower computational cost compared to previous approaches. Using our method we establish new benchmark by calculating the most accurate variational ground state energies ever published for number of different atoms and molecules. We systematically break down and measure improvements, focusing in particular on the effect increasing physical prior knowledge. We surprisingly find that increasing the prior knowledge given to the architecture can actually decrease accuracy.
Introduction
The challenge of the Schrödinger Equation Accurately predicting properties of molecules and materials is of utmost importance for many applications, including the development of new materials or pharmaceuticals. In principle, any property of any molecule can be calculated from wave- function, which is obtained by solving the Schrödinger equation. In practice, computing accurate wavefunctions and corresponding energies computationally extremely difficult for two reasons: First, the wavefunction is high-dimensional function, depending on all coordinates of all electrons, subjecting most methods to the curse dimensionality. Second, the required level accuracy is extremely high. While total energies of small molecules are typically hundreds of Hartrees. the chemically relevant energy differences are on the order of I milli-Hartee as depicted in Fig. []] Decades of research have produced a plethora of methods, which all require trade-off between accuracy and computational cost: On one end of the spectrum are approximate methods such as Hartree-Fock (HF) Density Functional Theory (DFT), which was awarded the Nobel prize in 1998. These methods can treat thousands of particles but can often only crudely approximate chem- ical properties. On the other end of the spectrum "gold-standard" methods such FCI (Full Configuration Interaction) or CCSD(T) (Coupled Clusters Singles Doubles (Perturbative Triples))
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

which yield energies that often closely agree with experiments, but can only treat up to 100 particles. Despite all these efforts. even for small molecules there do currently not exist highly accurate energy calculations. A 2020 benchmark of ttat-offthhe-rr methods for the benzene molecule found spread of mHa across different methods [T] - as matter of fact, our results show that the absolute energies calculated in [1] are off by at least 600 mHa, due to the small basis set used in their calculations. An important characteristic of a method is the type of approximation being made: Hartree-Fock or
Total energy
Correlation energy 511 mHa, 0.5%
109.50 505 Ha
100%
Required accuracy 1- mHa
0.002 %
Figure I: Conceptual visualization of the required accuracies on the example of an N2 molecule: For chemical applications, total energies must achieve accuracies of -...999998
FCI "variational", meaning that their predicted energies least upper-bound the ground-truth energy. Since a lower energy is always guaranteed to be a closer approximation of the true energy. this makes assessment of these methods frrwad.-forward. In contrast, CCSD(T) DFT do not have any guarantees on the accuracy of their results. The approximations resulting from such methods, while working well for many common systems, often fail for chemically challenging situations such as breaking of chemical bonds (2 3
Deep-learning-bane -ased variational Monte Carlo Combining deep learning and Monte Carlo meth- ods has recently emerged as a promising new approach for solving the Schrödinger equation [5 These methods offer high accuracy, moderate scaling of computational cost with system size and obey the variational principle. Within few years deep-learning-bate based methods have managed to outperform conventional high-accuracy methods for many different molecules, potentially defining new gold-standard for high-accuracy solutions. In the Born-Oppene Sern-Oppenheimer approximation a molecule, consisting of Nrme nuclei and net electrons, is fully described by its Hamiltonian in atomic units
ZiZs
Zr
H = -- Σ
Σ 139
Σ 1>J
Σ
r. -
r;
R1 Ry
Here RG Zr, I E {1,.... Tmuc) denote the coordinates and charges of the nuclei, (ri,..., ,......, )na) E RRaxn denotes the set of nel Cartesian electron coordinates differentiated between H4 spin-up and ny spin-down electrons. We define the inter-particle vectors T-j = r, ry and P.J = R. All properties of the molecule depend on the waveffunctoon ((r), which must fulfill the antisymmetry constraint: (PPr -r(r) for any permutation of two electrons with the same spin
[6]. The wavefunction &b can be found as the solution to the Schrödinger equation H = Egya with the ground-star energy and smallest eigenvalue Eg. By the Rayleigh-Riz, principle (7). the ground-state energy and the corresponding wavefunction can be found through minimization of the loss
H (()) the (r)
(
2 E,
for trial wavefunction the parameterized by parameters 0. The trial function the represented by neural network and typically has the form
vh (r) Σ det [Ad (r) (r:)) k.i=I.....m
(2)
d=1
with Ad. 3x + R. 0 determinant is taken over a Nel X ne matrix, with row-indices running over orbitals and column- indices i running over electrons. The determinant enforces antisymmetry. S are envelope functions enforcing the boundary condition lim - (r(r)) 0, and Age are neural networks. The local Hy energy & can be evaluated using automatic differentiation and the loss can be minimized by gradient based methods. The computation of the expectation value in eq. ]over the high-dimensional space R3xxa is done using Monte Carlo integration by sampling electron coordinates distributed according to 33 using the Metropolis- Hastings (8) algorithm. A thorough discussion of seep-learning-bare variational Monte Carlo (DL-VMC) can be found in [9]
R R, (1:4), i (1...., k E (1,...,me) Each

---

Related work Two major neural network architectures and their extensions have emerged through- literature: PauliNet [9] and FermiNet [1O]. PauliNet puts emphasis on maximizing physical prior knowledge, by focusing on the the envelope function. They use the output of CASSCF (Complete Active Space Self Consistent Field, a sophisticated conventional quantum-chemistry method) as f and use relatively small (~ 100k weights) neural network for A. FermiNet the other hand uses simple exponential function as envelope S2 and uses a large (~ 700k weights) neural network for A. Both approaches have been applied with great success to many different systems and properties, such as energies individual molecules 10 9]. ionization energies (TO), potential energy surfaces [TT
[2]. forces [1]] excited states [13]. model-systems for solids [14 15 and actual solids [16]. Several
approaches have been proposed to increase accuracy or decrease computational cost, most notably
architecture simplifications [17), alternative antisymmetric layers [18). effective core potentials [19]
and Diffusion Monte Carlo (DMC) [20 21 FermiNet commonly reaches lower (i.e. more accurate)
energies than PauliNet(TO), but PauliNet has been observed converge faster [12]. It has been proposed [9] that combining the embedding of FermiNet and the physical prior knowledge of PauliNet
could lead to superior architecture.
Our contribution In this work we present the counter-intuitive observation that the opposite approach might be more fruitful. By combining a PauliNet-like neural network embedding with the envelopes of FermiNet and adding several improvements the embedding, input features, and initialization of parameter (Sec. 2). we obtain the currently best neural network architecture for the numerical solution of the electronic Schrödinger equation. Combining our new architecture with VMC we establish new benchmark by calculating the most accurate variational ground state energies ever published for number of different atoms and molecules both when comparing to deep-learning- based methods, as well as when comparing to classical methods (Sec. 3 Across systems we reduce energy errors by 40-100% and achieve these results with 3-4x fewer optimization epochs compared to FermiNet. In Sec. 4 we systematically break down which changes cause these improvements. We hypothesize that including too much physical prior knowledge can actually hinder optimization and thus deteriorate accuracy we provide ample experimental evidence in Sec. 5
Improved approach
Similar FermiNet, our architecture expresses Ag, as linear combination of high high- dibeemmieniin electron embeddings h,, and the envelopes 22 as a sum of exponential functions
Bue
(r) Wdde h
Saaa (r:)
exp(-w Pul)) Thl
das
dey
dos dos trainable parameters and enforce w/z where Wedn; Tat wkr embeddings h by first transforming the inputs R₁, r; into feature vectors
dog
2 0. We compute these
h
Pall.P.r
2tr = | P.s
gh = -rag
where [:] denotes the concatenation operation and then applying T. iterations of embedding network (Fig. 2). The local difference vectors P.r are obtained by applying rotation matrices onto P.r as described in Sec. 2.2
2.1
Convolutional layers in embedding
Our embedding network uses four residual neural network streams (Fig. 2b); A primary one- electron stream that embeds single electron, and three auxiliary streams modelling the two-particle- interactions (electrons with same spins, electrons with different spin, and electron-ion).
h
[+1
A (f!) h
I+1 g.
A (g'g) g'y
I+1
= A (v'x) + o/
(4)
Here denotes the embedding iteration, A' denote fully connected neural networks, and gly- o/y denote clectron-electron- and electron-nncdeus-ecmmddii We use oıj 'same" for same-spin

---

Feadsforuand networks
Features
Changes milaive FermiNer
Adotion
Meen over elestrons
Concatenation
SchWai--ke convotution
π
Sanune
A
Local features
Iteration
Symmetric features
Symmettrcc Foatures
Mas
Iteration
A
A
A
Aima
Bi Jauan
Heration
Shut
Hy th onl
-8 nol
Ssame
Figure 2: Our architecture: High-level overview b) One single embedding iteration c) Sub-block of assembling symmetric features
pairs of electrons and dij 'diff" for pairs of electrons with different spin. Similar to FermiNet, in each iteration assemble the input f the primary stream from the auxiliary streams (Fig. 2
Σ hj 74 j=1++4
Lel
Lonc
(5)
Inspired by the success of SchNet sum of element- wise multiplication two-particle streams:
22 and the efficiency of the PauliNet embedding, we use the
()), effectively forming a convolution, to aggregate the auxiliary
The
Nion
Σ
B', (g/))
C ass (h))
Σ B (1)))
C (Zemb)
(6)
Eq. and form the core the architecture and are the key difference between FermiNet, PauliNet and our architecture. The PauliNet architecture emphasizes two-particle interactions and essentially only uses convolutions as input features: S - [s!et features, PauliNet also limits its effective depth by making convolutional kernels B functions of the electron-electron distances |rişl instead of the nigh-dimensional embedded representations 91z- The FermiNet architecture on the other hand emphasizes the one-electron stream and only uses
4.m"c). In addition not using the h; input
sums over 9sj input features, essentially corresponding to B' Id, C(.) 1. Furthermore FermiNet does not contain an explicit stream for dlectron-nucledd interactions. Since our architecture adequately models both the one-electron -enetroonemmeddiing well as the particle-interaeeee expect our architecture to more expressive than either predecessor, as demonstrated Sec.
Local, invariant input features
The first stage of any VMC wavefunction model is typically the computation of suitable input features from the raw electron coordinates and nuclear coordinates { (R,}, ). While the subsequent embedding stage could in principle take the raw coordinates, appropriate features allow to explicitly enforce symmetries and improve the model's transferability and accuracy. Input features should have three properties: First, they should be sufficiently expressive encode any physical wavefunction. Second, the features should be invariant under geometric transformations, Third, the features should primarily depend on the local environment of a particle, i.c. similar local geometries should generate similar local features, mostly independent of changes to the geometry far from the particle question. Published architectures have so far not been able to address all three points: PauliNet (10) uses only distances as input features, making them invariant and local, but not sufficiently expressive, as demonstrated by [12). FermiNet (10) raw distances and differences, making the inputs expressive and local, but not invariant under rotations. PESNet [2] proposes a global coordinate system along the principle axes of molecule, making the inputs invariant and sufficiently expressive, but not local.
propose using local coordinate systems centered on every nucleus and evaluating the electron- nuclei differences in these local coordinate systems. Effectively this amounts to applying rotation

---

a.) Ethene
b.) Hypothetical chain of atoms
Global
coordinates (PESNet)
Local
coordinates (this work)
Figure 3: Visualization of resulting coordinate systems for 2 example molecules: a) Ethene b) A
hypothetical bent chain of atoms.
matrix U, the raw alectron-nucleus differences: = p.,== UJP As opposed to [22 where U is constant for all nuclei, in our approach U. can be different for every nucleus. These electron- nucleus differences P., are invariant under rotations, contain all the information contained raw Cartesian coordinates and depend primarily on the local environment of an atom. To compute the 3x3 rotation matrices J. we first run single Hartree-Food calculation using minimal basis to obtain density matrix D. For each atom J we choose the 3x3 block of the density matrix which corresponds to the p-orbitals of the atom and compute its eigenvectors U, Dig(D)). To make our coordinate system unique we sort the eigenvectors of D, by their corresponding eigenvalues. If the eigenvalues are degenerate, we pick the rotation the eigenvectors that maximizes the overlap with the coordinate-axes of the previous nucleus. Fig. 3 shows the resulting coordinate systems spanned by U,. Note that the local coordinate system generally depicts more physically meaningful directions such as "along the chain". We find that these local coordinates slightly increase the accuracy for single geometries, but more importantly we expect the wavefunction to generalize better across different molecules geometries. This should improve the accuracy of approaches that attempt to learn wavefunctions for multiple geometries at once 12
2.3 Initialization of orbital envelope weights
When considering a single atom, the entries of the wavefunction determinant have essentially the form
(r) exp (-wi (prl))
In [10]. the exponential envelope was purely motivated by the boundary condition that the orbitals must decay to zero, and initialization with wa 1 proposed. However, when comparing this ansatz to analytical solutions, an interesting parallel can be found: Analytical solutions to the Schrödinger equation for atoms with single electron the only systems that have analytical solutions are of the form
7 (-2 71
(p,r)) exp
|
where (prr))) product of a Laguerre polynomial and spherical harmonic, and ng E N is known as the principal quantum number. This suggests Wk ~ 2/ns, which we also find when analyzing the weights of fully trained wavefunction. When initializing with w4 = 2/n, instead of
1, we observe faster convergence, lower final energies, and lower variance of the energies (Sec. The effect is most notable for molecules containing nuclei with large Z, where 2/nz >> 1.
2.4 Improved hyperparameters
Beyond the improved neural network architecture we fine-tuned the hyperparameters to reduce the number optimization steps required for convergence. Starting from the hyperparameters proposed by [10], we increased the norm constrain by 3x for the second-order optimizer KFAC (23 24].
decreased learning rate by 0.5x, and decreased learning rate decay time by 0.4x. observe that these changes stabilize the optimization and enable usage of 50% fewer Monte Carlo walkers, which results ~2x faster optimization and reduced memory allocation. complete set hyperparameters can be found in appendix B.
5

---

Results of improved approach
We evaluated the accuracy of our approach by comparing our computed energies against the most accurate references available in the literature. Fig. 4 compares energies against variational methods - for which lower energies are guaranteed to be more accurate - as well as non-variational high-accuracy methods. We find that across many different systems (small and large atoms, molecules at equilibrium geometry, molecules in transition states), our approach yields substantially lower and thus more accurate energies than previous variational results. Across all tested systems, outperform almost all existing variational methods, both haap-learninn-basee methods as well as classical ones. When comparing to high-accuracy FermiNet VMC calculations, we not only reach substantially lower energies. but also do so using 3-4x fewer training steps, with each step being 40% faster (cf. appendix C). Comparing a concurrently published Diffusion Monte Carlo approach. which used 10x more computational resources, we achieve similar or better accuracy for molecules like N₂ and cyclobutadiene and slightly lower accuracy for benzene. variaaaition methods (e.g. CCSD(T)) yield slightly lower energies than our calculations for some molecules, but since those methods do not provide upper bounds uncertainty guarantees they not provide ground-truth. For many applications not only absolute energies are important, but energy differences between
Best previcusly published
steps
steps
0.3
0.8
39 14)
0.2
CI
HgO
0.0
67
0.1
Ne
-0011
Ar
3.6
N
-139
Fe
0.0
-1.4
3.3
d-4.0
-155
1.2 10
1.4 1H1
190 MI
NH
Glycine
1.8
3.7
0.0
1.8
S
1.6
CO
1.3
C,H+
20
0.3
1.2
-2.1
10
0.5
1.0
-100
E E
mHa
E Eir mHa
E E.vl mHa
E, / mHa
Figure 4: Energies relative the previously known best estimate, (lower is better). Blue bars depict best published variational energies, footnotes mark the method: a: FermiNet VMC (1O] [7].
Conventional DMC 25 26 27 FermiNet DMC (21), d: MRCI-F12. A table of absolute energies
and methods for E can be found in appendix A. Note that E is not necessary variational and thus
may underestimate the true energy.
different molecules or geometries of interest, for example determine the energy required to break a chemical bond. particularly difficult challenge the dissociation of the N₂ molecule, i.e. the energy of an N₂ molecule different bond lengths (inset Fig. 5). Even methods that are generally regarded as highly accurate, such as CCSD(T), predict energies that deviate far from experimental data at bond-lengths from 2.5 bohr. Fig. S depicts this deviation between experimental and computed energy for our method and the best available reference calculations. We find that our results are closer to the experimental absolute energies than all previous work, and are similar to concurrently published FermiNet-DMC results which require 50x0 more computational resources. When comparing relative energies, our approach outperforms all other deep-learning based methods and CCSD(T), and is only beaten by the highly specialized r r12-MR-ACPP ACPF method (28). Similar to absolute energies, we also find that our relative energies converge substantially faster than for other deep-lcarning based methods, with relative energies being almost fully converged after 50k epochs.

---

Energy error for N2 dissociation curve
Relative errors
-1092
CCSDIT)
25.3
15
4
FermiNat block-dat
opochs
10.2
FermiNet dense-dat 50k cpochs
5.4
FermiNet dense det 100k epochs
10
PermiNet
3.2
r12 222MM FE-MM-ACC ACPF
x 2.1
This work
50K epochs
This
3.2
bond length / bohr
A.max AE
Figure 5: Comparison of energy error Eeexpeimmnt for the dissociation of the N2 molecule across various methods. Errorbars corresponds to the standard deviation wrt. two different seeds. Errorbars for our work are too small to be visible (~ 0.1 mHa). Results for Exxperiment can be found in (2]
FermiNet block-det & CCSD(T) in [10,, FermiNet DMC in [1] and r12-MR-ACPF in 28
Ablation study
To investigate which specific changes lead to the observed improvements in accuracy, we start from the improved FermiNet architecture proposed in [17) and incrementally add improvements in the following order: First, we use dense Nel net determinants introduced by the FermiNet authors (10
17 in their GitHub repository and described in (88 instead of block -oock-diagonal determinants. This generalization increases computational cost and parameter count (ef. appendix C) but has been found to better describe the wavefunction's 's nodal surface and thus increase expressiveness of the ansatz. Second, we change hyperparameters as described in Sec. [.4 which increases throughput by 2x. Third, we augment the electron embedding using our new SchNet-like neural network architecture described in Sec. 2.1 This leads moderate increase parameter count and computational cost. Fourth, we switch to local, invariant input features as described in Sec. 2.2 and remove the electron- electron difference vectors rú as inputs. Lastly we switch to initializing w kl 2/m, as described in Sec. 2.3 resulting in proposed final method. We note that the accuracy gains of these changes not fully independent of each other and the relative attribution depends on the order in which they applied: Earlier changes will generally generate larger energy improvements compared to later changes. At each step we compute total energies for three different molecules: ethene, N2 at the challenging bond-length of 4.0 bohr, and the K-atom. Fig. depppct the energy of our implementation of FermiNet, and the energy change caused by each subsequent improvement. Each experiment was repeated two times with different RNG seeds (appendix D), the errorbars depict the spread in energy. Overall we find that all our changes combined yield a ~3-20x improvement in the energy error. For ethene, the dominant contribution (3.7 mHa) comes from improved hyperparameters, which lead to the results being mostly converged after 50k epochs vs. the original settings which require 200k epochs for convergence. Using lower learning rate in combination with a larger gradient-normm-constriin ensures that more optimization steps are taken according to curvature Architectural
estimated by KFAC and fewer steps are clipped by the mundient-norm-comee improvements (embedding and input features) lower the energy eтror by additional 1.4 mHa. Because our embedding is a strict generalization of both FermiNet and PauliNet, our ansatz is more expressive and can therefore reach lower energies than previous ansätze. For N₂ it has already been observed that a single dense determinant can outperform models with multiple block-diagonal determinants 18]. We find the corresponding result that 32 dense determinants substantially lower the energy relative to an ansatz. with 32 block- diigonaa determinants. Comparing N₂ to ethene, we observe larger contributions from our architectural improvements and smaller contributions from improved hyperparameters. For the K atom, the overall gains are largest, totalling 60mHa, with substantial accuracy gains from all improvements. Since K has much larger nuclear charge (Z=19) than the constituents of ethene (Z=1,6) and N₂ (Z=7), also the physics-inspired initialization of the envelope parameters yields a substantial contribution. This improved initialization leads to a better initial guess for the wavefunction, which not only reduces the number of required optimization steps, but also leads to more accurate initial sampling.
7

---

Ethene
N2. d 4.0 bohr
K atom
FermiNet
8.0
63.0
isotropic
Dense Determinant
Hyper- paramne arrameterr
.2
SchNet-like embedding
Local input- teatures
Envelope initialization
Improvements by earlier work mmpommeents This work
This work
[Pasu 2020] 200k epochs
[Ren 2022] 400k epochs
40
60
E Erer / mHa
E - Erer / mHa
E - ErEr mHa
Figure 6: Breakdown of accuracy improvements for three different molecules, each trained for 50k epochs. The black dashed line depicts the best published variational result and as reference we use the best available estimates: CCSD(T) [10] for ethene, experimental data for N2 (29), and our own VMC calculations after 100k epochs for the K atom.
Incorporating prior knowledge
To further understand the effect of incorporating prior knowledge into neural network architectures for physical problems as the electronic Schrödinger equation, we examined two distinct ways of increasing prior information our model: First, by including built-in approximate physical model, analogous to PauliNet. Second, by increasing the number of pre-training steps more closely match a reference wavefunction before starting the optimization.
Explicitly include CASSCF PauliNet maximizes the physical prior information by computing the envelopes Ω with CASSCF, a sophisticated conventional method, and explicitly enforcing the Kato cusp conditions (30). Starting from our proposed architecture, we modified our approach step-by-step until we arrived at PauliNet-like architecture. Fig. 77h shows the energies of an NH3 molecule trained for 50k epochs each step.
CASSCF vs. exp. envelope
CASSCF vs. exp. envelope
Pretraining
published
CASSCF
CASSCF
This
0.7
2003 epocht
vartationall (Pau 2000).
envellpe
5dk opt. upechs
ost. epochs
determinant
0.2
Energy rel to reference mine
20
Best sublshed varaaknnak:
referencee mike
Hartroe Fock 20k opt. epochs
Paulitet emb. HP
1.2
Exp. envelape Paulket
CAsSCr envelope
1.r.-.. R.
1.0
feature
0.5
Paulieet
13.2
0.0
5
15
20
30
40
0.2
20 100
Energy rel, to reference / mHa
Epochs /1k
Protraining epochs 1k
Figure 7: Effect of prior knowledge for NHg: a) Change in accuracy after 50k optimization epochs when transitioning from our approach to PauliNet-like architecture, by increasing prior knowledge. b) Energy error for 3 architectures over number of training epochs. c) Mean and standard deviation of the energy error as a function of pre- mreitianning steps. Pre-training used CASSCF or HF as reference, and the energy was evaluated after 20k or 50k subsequent variational optimization epochs.

---

First, we switch from dense determinants to block-diagone determinants as used by the original PauliNet, leading to small loss accuracy. Second, we exchange our embedding for the PauliNet-like embedding using the hyperparameters proposed in (TT), leading to substantial loss in accuracy, presumably caused by a loss in expressiveness. Next, we replace the simple exponential envelopes by the more physically inspired CASSCF
causing large loss in accuracy. We then remove the vector ri Ry as input feature (keeping only its absolute value) as done in the original PauliNet architecture This again deteriorates accuracy, presumably due to enforcing rotational invariance which is too restrictive of a symmetry class as pointed out by [12]. Lastly we replace the electron- electron distances |r,yl (which are not smooth at ray 0 and thus lead to cusps) by smooth, cuspless radial basis functions as input feature and add an explicit term to enforce the deectonn-eecctron cusp condition. Since the CASSCF-envelopee are designed to fulfill the electron-ion cusp condition, this change leads to an enforced cusp condition, slightly improving the energy accuracy. The largest loss in accuracy throughout these changes is caused by introducing the CASSCCF-nvelopes,, suggesting that they introduce a strong bias of the wavefunction that cannot be easily overcome during training. Fig. 7 shows that architectures using exponential envelopes converge lower absolute energies compared to the CASSCF-based PauliNet and outperform PauliNet already after 5500 epochs.
Increase pre-training accuracy Before starting the unsupervised variational optimization of the wavefunction, we run a short supervised pre-training of the wavefunction to roughly match a given reference wavefunction. This is computationally inexpensive because it only requires evaluation of di and a back- proppaaopagttion step to update the neural network weights / but not the second derivative of the Hamiltonian. If the reference method yields a decent approximation to the true wavefunction, this pre-training significantly speeds-up optimization and avoids unstable parameter regimes (1O). To incorporate more prior knowledge, one could either use more sophisticated reference method (e.g. CASSCF instead of HF) or increase the number of pre- vre-training steps. In Fig. 7 we pre-trained the wavefunction with block diagonal determinant for the NH3 molecule using a CASSCF and Hartree-Fock reference. We increased pre-training iteration steps and evaluated the energy after subsequent 20k and 50k variational optimization epochs, each run was repeated with five different seeds. Increasing the number of pre-training steps initially increases accuracy since it provides a better starting point for the subsequent variational optimization but when increasing pre-training beyond 20k steps, accuracy deteriorates for both methods. Surprisingly, we observe a drop in accuracy when using CASSCF as a reference method compared the simpler Hartree-Fock method. This effect is even more pronounced when increasing the number of pre-training steps. It suggests that excessive pre-trale pre-training introduces bias that is hard to overcome during variational optimization, similarly to a built-in reference method.
6 Discussion and Limitations
Discussion We find that our approach yields substantially more accurate absolute energies than all previous work- both classical as well as deep-learning-basee basedd and that we reach these accuracies 4-6x faster than the next best method (FermiNet). Especially for larger systems, such as 4th row atoms or the amino acid glycine, we outperform conventional "gooddstanddrr" methods like MRCI-F12(Q) by 100 mHa. This corroborates the fact that deep-learning based methods are emerging as a new gold-standar in computational chemistry and showcases the immense potential of machine- baaning-based methods in the natural sciences. A concurrent work (2I) was able to achieve similar accuracies by applying Diffusion Monte Carlo (DMC) on top of FermiNet VMC calculation, highlighting the potential of deep-learning Monte Carlo methods. However, (2T) required 10x more computational resources and their VMC results - already by themselves 8x more expensive then calculations are consistently inferior to our results. This showcases promising route towards further improvements by using our substantially cheaper and more accurate VMC results as a starting point for a DMC calculation.
Regarding the question of how much physies to include in the model, we find varying results. For exact physical constraints, such as symmetries or the cusp conditions, inclusion in the model generally appears to be helpful. However for prior knowledge from existing approximate solutions (such as CASSCF) the situation is more subtle. On the one hand, soft physical guidance such as short supervised pre-training or physics-inspired weight initialization accelerates optimization. On the other hand, we show empirically that increasing physical prior knowledge, e.g. by incorporating

---

CASSCF or extensive supervised pre- trainini, does not necessarily increase accuracy. but can in fact introduce detrimental biases that are hard overcome during wavefunction optimization.
Limitations and outlook Despite the proposed improvements and favorable scaling of the method, computation of energies for large molecules still takes days of GPU-time on current hardware. While the same holds true for conventional high-accuracy approaches, substantial speed-ups are still required to make DL-VMC more accessible for practitioners. Additionally, when increasing the nuclear charges, the wavefunction becomes increasingly localised, which leads reduction average Monte Carlo stepsize and potentially correlated samples. We circumvent this effect for 4th row atoms by increasing the number of intermediate Monte Carlo steps, but further research into Monte Carlo sampling methods [31, 32) is required to fully address this issue. Despite our improvements for the accuracy of energy differences between different molecules or geometries, DL- VMC still outperformed by other, computationally cheaper methods some cases, Initial research into the regularity of the wavefunction across different molecules [1I 12 provides promising route to improvements. We note passing that thanks to the local coordinate input features, our architecture fulfills the required rotational invariance required for these approaches.
7
Code availability
The code alongside detailed documentation is available as part of the DeepErwin code package on the Python Package Index (PyPI) and github (https://githnn) comm///ouunniieee/eeerrrn under the MIT license.
8 Acknowledgements
We gratefully acknowledge financial support from the following grants: Austrian Science Fund FWF Project I 3403 (P.G.). WWTF-ICT19-04) (L.G.). The computational results have been achieved using the Vienna Scientific Cluster (VSC). The funders had no role in study design, data collection and analysis, decision to publish preparation of the manuscript. Additionally, thank Nicholas Gao for providing his results and data and Rafael Reisenhofer for providing valuable feedback to the manuscript.
10

---

References
[1] Janus J. Eriksen et al. "The Ground State Electronic Energy of Benzene". In: The Journal of Physical Chemistry Letters 11.20 (Oct. 2020). Publisher: American Chemical Society, pp. 8922-8929. DOI: 10. 1021 acs jpclett Oc02621 URL: https //doi. org/
1021/acs.jpclese 0c02621 (visited on 04/21/2022).
[2] Narbe Mardirossian and Martin
"Thirty years of density functional theory in computational chemistry: an overview and extensive assessment of 200 density functionals", In:
Molecular Physics 1 115. 19 (2017), pp. 2315-2372. DOL: 10. 1080 1080/00268976 00w.0006669778 2017 13336444
eprint: https //doi org/ 10. 1080 0000000026966666 2017 1333644
URL: https: //doi
org/10. org/10.1080/0000 000.000000000089766666 2017 1333644
[3] Aron J. Cohen, Paula Mori-Sánchez, and Weitao Yang. "Challenges for Density Functional Theory". In: Chemical Reviews 112.1 (Jan. 2012). Publisher: American Chemical Society,
pp. 289-320. ISSN: 0009-2665..
DOL: 10. 1021/ cr200107z.
URL:
1021/cr200107>
[4] Jiequn Han, Linfeng Zhang, and Weinan E. "Solving many electron Schrödinger equation using deep neural networks" In: J. Comput, Phys. 399 (Dec. 2019), 108929, ISSN: 0021-
9991. DO1: 10. 1016/016jj.. jop 2019. 108929
URL: http:: //xx. doi. 1016//00.00061.... jcp
2019.108922
[5] Giuseppe Carleo and Matthias Troyer. "Solving the quantum many-body problem with artificial neural networks". In: Science 355.6325 (Feb. 2017). Publisher: American Association for the
Advancement of Science, pp. 602 606. DOL: 10. 1126 /scienceieeeeeee aag2302 URL: https
//www.science sww.sciiienecee. www.oo///oooo http/doi/1.... /126/scienceeeeeec
science. aag2302 (visited on 05/09/2022).
[6] Wolfgang Pauli. Nobel lecture. Dec. 1946.
[7] Walter Ritz. "Über eine neue Methode zur Lösung gewisser Variattionsproblemme der mathe- matischen Physik." In: 1909. 135 (1909), pp. 1-61. DOT: :oi: 10151
1051:/ w9w.e.....11....... 1909. 135.
URL: https://dois org 10. 1515/ 111p1///crl1.110
135.
[8] W. K. Hastings. "Monte Carlo sampling methods using Markov chains and their applications".
In: Biomerrika 57.1 (Apr. 1970), 97- ISSN: 0006-3444, Dor; 10. www.://///iimtt/...o.o 97 eprint: https:ttp.... http:::://caaaacccaca.///
com www.////wo.tttttttoooooooooeaaee
pdf URL:: http:/ttts/:/ooo..ee htt///oo..org/../. http://0..0033/bmmm////////11111
[9] Jan Hermann, Zeno Schätzle, and Frank Noé. -eeee-neural- solution of the electronic
Schrödinger equation".. In: Nat. Chem. 12.10 (Oct. 2020), 891-897, ISSN: 1755-4349, DOI:
10. 00.108//415511 012-16557-02005444-- URL: httpp::/ddae org. 54100////1157-000000 /ttl://////////////////////////0
y
[10] David Pfau et al. "Ab initio solution of the many-electron Schrödinger equation with deep neural networks". In: Phys. Rev. Res. 2 (3 Sept. 2020), p. 033429. DOl: 10. 1103/
PhysRevResearch 2 033429
URL: https: link aps org / doi / 10 1103
PhysRevResearch. 2. 033429
[11] Michael Scherbela et Solving the electronic Schrüdinger equation for multiple nuclear geometries with weight-sharing deep neural networks. 2021. DOT: 10. 48550/ ARXIV 2105
08351 URL: https::////// / /ttp/v/.ivvvvvv////v//ea//aa.aa. wwrrxivvoorg... org/abs/////ooooooooooooooe 2205 0ttp//215..0005111
[12] Nicholas Gao and Stephan Günnemann. Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions. 2021. DO1: 10. 48550 ARXIV 2110. 05064 URL: https
7/arxiv... orgg abs//110. 05064
[13]
Mike Entwistle et al. Electronic excited states in deep variational Monte Carlo. 2022. DOI:
10. 48840855/ARR ARXII/ 2203.222222 09472 URL: ptttps://ariis org/ ab//abb/22200 09472
[14] Max Wilson et al. Wave function Ansatz (but Periodic) Networks and the Homogeneous Electron Gas. 2022. DO1: 10. 48550/
ARXIV 2202. 04622
URL: https: / //axiiv.... org/
abs/2202. 04622
[15] G. Cassella et al. Discovering Quantum Phase Transitions with Fermionic Neural Networks.
DOI: 10. 48550/ ARXIV. 2202. 05183 URL: hhttss///aaxiii org/ al//abbs/2222 05183 [16] Xiang Li, Zhe Li, and Chen. "Ab initio calculation real solids via neural network ansatz".
(cond-mat, physicx:physics/ (Mar. 2022), arXiv: 2203. 15472. DOI: 10. 48550/550/40 ARXIV. 2203. 15472
In: arXiv:2203.1544
URL: http: // /arxivxiivv. wwtpxvvvvorr//eoo wwwgg/a///////./ /20.. 200.2222..17722 15472 (visited on
04/20/2022).

---

[17] James S. Spencer et al. Better, Faster Fermionic Neural Networks. 2020. DOI: 10. 48550/
ARXIV 2011. 07125 URL: URL: htttt:tttt///tatti org. abs///as...1. 07125
[18] Jeffmin Lin, Gil Goldshlager, and Lin Lin. "Explicitly antisymmetrized neural network layers for variational Monte Carlo simulation". In: www.i::/I22......... 02.0349 [physics/ (Dec. 2021). arXiv: 2112.03491. DOI: 10. 48550/ 08550/ARXIV, WRW.V.2212.0...... 03491 03491 (visited on 03/25/2022).
URL:
org/ abs/ 2112
[19]
Xiang Li et al. "Fermionic neural network with effective core potential". In: Physical Re- view Research 4.1 (Jan. 2022). Publisher: American Physical Society, 013021. DOI: 10
1103 Su03/PhysRewResearee
4. 013021 URL: https ://lnnk//in aps org /og /10 1103/
PhysRevResearch ch 013021 (visited on 04/26/2022).
[20]
Max Wilson al. Simulations state-of-the-an fermionic neural network wave functions with diffusson Monte Carlo. 2021. DO1: 10. [physics chem-phh
2103. 12570
arXiv: 2103, 12570
[21]
Weiluo Ren, Weizhong Fu, and Ji Chen. Towards the ground state of molecules via diffusion Monte Carlo on neural networks. 2022. DO1: 10. 48550/ ARXIV 2204. 13903
URL: https
// //arxiv. seriiv.rg/// abs 22044. 13903
[22] Kristof Schütt et al. "SchNet: A continuous-fille convolutional neural network for modeling quantum interactions". In: Advances in Neural Information Processing Systems. Ed. by I. Guyon et al. Vol. 30, Curran Associates, Inc., 2017, URL: http://tttt//p neurips cc/paper/2015 /t/00///ffi. /ilf////// 30354468886113332444838633333333
Paper -df [23] James Martens and Roger Grosse. "Optimizing neural networks with kronecker-factored approximate curvature". In: International conference on machine learning. PMLR. 2015, pp. 2408-2417
[24] Aleksandar Botev and James Martens. KFAC-JAX. Version 0.0.1. 2022. URT. com/ Steepmind/kffc-1 jax
[25]
P. Seth, P. López Ríos, and R. Needs. "Quantum Monte Carlo study of the first-row atoms and ions". In: The Journal of Chemical Physics I 134.8 (2011), p. 084105. DOL: 10 1063/1
3554625 eprint: htttss://dii. org/ 10. 1063/ 3554625
URL: htttt:://tti....e/e
3063/1.3554621
[26] Norbert Nemec, Michael D. Towler, and R. J. Needs. "Benchmark all-electron ab initio quantum Monte Carlo calculations for small molecules", In: The Journal of Chemical Physics 132.3 (Jan. 2010). Publisher: American Institute of Physics, 0341 11. ISSN: 0021-9606. DOL: 10. 1063/1 3288054 URL: http: tttt:://aassciaationcooo org doi /ttp////wwo///////////////ooooet 1063/1 3288054 (visited on 04/12/2022).
[27] Bryan K. Clark al. "Computing the energy of water molecule using multideterminants:
A simple, efficient algorithm". In: The Journal of Chemical Physics 135.24 (Dec. 201 I).
Publisher: American Institute of Physics, p. 244105. ISSN: 0021 9606. DOI: 10 1063/1
3665391 URL:
3665339 (visited on
05/31/2022).
[28] Robert J. Gdanitz. " "Accurately solving the electronic Schrödinger equation of atoms and molecules using explicitly correlated (r12-)MR the ground state potential energy curve of
N2". In: Chemical Physics Letters 283.5 (1998), pp. 253-261. ISSN: 0009-2614. DOI: hhtpss
0 00009-000000000000 (614 //doi 0rggiiorg//1000 1016
URL:
[29] Robert Le Roy, Yiye Huang, and Calvin Jary. "An accurate analytic potential function for ground-state N2 from direct-potential-fi analysis of spectroscopic data". In: The Journal
of Chemical Physics 125,16 16 (2006), p. 164310. DO1: 10 1063/1 2354502 eprint: https
//doi aocoi.orrg/00 10. 1063/1001111 2354502 URL: /httpt::/// :/tt:s://doi..oo org/ 1063/1 2354502
[30] Tosio Kato. "On the eigenfunctions of many-particle systems in quantum mechanics". In:
Commun. Pure Appl. Math. 10.2 (1957), 151 nor: 10 1002/ cpa. 31601.00000011111 eprint:
wiley com doi /ttp////////////////////////e/// wwwwww wtt. aws/aaas.1000./ 1002/ http:////p..3310102222022.222220 31601002011
1002 cpa 31601002011 11111000......11111000.00.11
URL.:
[31] J. Umrigar, M. P. Nightingale, and K. Runge. "A diffusion Monte Carlo algorithm with
very small time-step errors". In: The Journal of Chemical Physics 99.4 (1993), pp. 2865-
2890. DOI: 10. 10w3010011111 465195 eprint: https: / //o/// org 1663/1.661111.....1 URL.:
httpps///
12

---

[32]
J. Umrigar. "Accelerated Metropolis method". In: Phys. Rev. Lett. 71 (3 July 1993), pp. 408 411. DOL: 10. 1103/ 1110/PhysReetieee PhysRevLet........e. 71 408 URL: https /t////inn aps org/ doi / 10
1103/PhysRevLett.777 71.488
13


Reviews:
[{"review_id": "jbUaHN_lp07", "reviewer": "Reviewer_fBuC", "summary": "The authors present and analyze a list of improvements that significantly increase accuracy and reduce the computational cost of variational Monte Carlo methods based on deep learning.", "questions": "- The units used in the equation before Eq. (1) should be clarified.\n- The (probably well-known) Rayleigh-Ritz principle mentioned in L57 warrants a citation for the interested reader.\n- How is $H \\psi(r)$ evaluated?\n- In Eq. (2), $k$ seems to be, next to $i$, an index for an electron. In the previous equations, you use $j$. Why?\n- The concrete form of Eq. (1) is not obvious. I assume that $\\lambda$ is actually a matrix of shape $n_{el} \\times n_{el}$?\n- Below Eq. (3), you have another $k$. Is that related to the $k$ from before?\n- I didn't fully understand the challenges you (and the community, I assume) are facing in Section 2.2. How are these requirements different from, let's say, an ML force field.\n- Do you mention anywhere the total computational cost of each method?", "limitations": "The authors discuss the limitations of their work.", "rating": 7, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n\n- Overall, the paper is very well-written and clear, even for someone with little domain expertise.\n- The introduction is helpful and allows non-experts to onboard (to the extent one can reasonably expect).\n- The key contributions are laid out.\n- Differences and similarities to previous methods (especially PauliNet) are exhibited.\n- Thoughtful experimental design.\n- Impressive experimental results.\n\n# Weaknesses\n\n- A (minor) weakness is the lack of immediate relevance of the model improvements to the broader ML community.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "c1MbEXRO0Qm", "reviewer": "Reviewer_NQXY", "summary": "This paper proposes a deep-learning architecture to solve ground state of many-electron systems. The proposed model combines PauliNet-like neural network and envelope function in FermiNet, with additional improvements on embedding, input feature and parameter initialization of previous methods. Experimental results show that the proposed method can reduce errors by 40-70% with 4-8x lower computational cost. This paper also establish a new benchmark of several deep learning based and classical methods over a number of different atoms and molecules. Authors also research into the reason for improvements and find out that including too much physical prior knowledge can deteriorate the accuracy.", "questions": "Please see weaknesses.", "limitations": "Limitation of this work is well addressed.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n- *originality:* Related works are adequately cited and it is clear to see the difference from these works. This work is a novel combination of PauliNet and FermiNet along with additional improvements.  \n- *quality:* This work provides solid experiments to prove the accuracy and efficiency improvements. Also, this work systematically breaks down which changes cause the improvements, and gives insights that including physical prior knowledge can hinder the optimization. \n- *clarity*: This paper is overall well written and easy to follow. The method and results are clearly presented.\n- *significance:* This work achieves the best results for the numerical solution of the electronic Schrodinger equation and establishes a new benchmark for current most accurate methods on a number of molecules and atoms. The proposed method is 4-8x faster than FermiNet in terms of optimization.  \n\n**Weaknesses:**\nThis paper demonstrates great experimental results, but lacking the necessary theoretical explanation to address the accuracy improvement under reduced computational cost. Section 4 discussed in detail the improvements obtained by each individual change and their combined effects, but the discussion is still restricted within the experimental point of view. Adding theoretical explanation on why the proposed method can obtain accuracy better than the classical as well as existing deep learning method can be beneficial. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "VtjKOqcQOFf", "reviewer": "Reviewer_n1mQ", "summary": "This paper focuses on empirical improvements of variational quantum Monte Carlo estimation of molecular ground state energy. In particular, the authors focus on evaluating the various design choices in the FermiNet and the PauliNet, including dense determinant, hyper parameter tuning, the choice of envelope function and the pretraining strategies. On top of these, the authors propose to use a SchNet-like embedding blocks and an input feature transformation to make the input feature to make the feature to be invariant, local and expressive. \n\nThe authors empirically evaluate all these design choices on molecules with up to ~40 electrons and obtain a model which improves upon the existing methods both in terms of accuracy and speed. The authors also conduct ablation studies by removing each design choice one by one. \n\nFrom the experiments, although the factors are not totally independent, it seems the most improvements come from the use of dense determinant, hyper parameters tuning, and the SchNet-like embedding. The proposed feature transformation only has minor effects.", "questions": "* Some results in Figure 4 are lower than the reference energy, especially for larger molecules where the differences are significant (>100mHa) compared to the chemical accuracy (~2mHa). To which extent are we certain that the QMC results are correct? Although the variational principe could grantee the variational energy to be upper bound but doesn't it assume the sampling to be accurate in the first place? For example, in the Figure 3 of the FermiNet paper [Pfau et al. 2020], during VMC training, the energy could overshot when the MCMC step size is not large enough.\n\n* Under which setting the claimed \"40-70% lower energy error at 8x lower computational cost\" is obtained? As shown in Table 5, the runtime per epoch improves from ~6s to 3.6s. How is the convergence time defined?\n\n[Pfau et al. 2020] Ab initio solution of the many-electron Schrödinger equation with deep neural networks. Physical Review Research, 2020. \n\n------\n**After rebuttal:**\n\nThe authors' response clarifies the concern about the uncertainty and the speedup. Overall I think the proposed method is well motivated and results are convincing. Hence I will increase my score from 5 to 7 and will recommend acceptance of this paper.", "limitations": "The authors state the limitations adequately.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength:** \n\nThe paper is well written and easy to follow. The experimental results are good and the ablation studies are extensive and well presented. The effects of different design choices can be easily identified. Since PauliNet and FermiNet differ in many aspects while both give high quality quantum Monte Carlo results, it is interesting to compare and clarify the effects of these differences. Making improvements by fusing these two methods is well motivated.\n\n**Weakness:** \n* Since the paper essentially integrates FermiNet and PauliNet, the novelty of proposed methods are very limited. In particular, the proposed coordinate transform only has sight effects. \n* Although the overall improvement and speedup are clear, for some claims it is difficult to find reference in the main paper. E.g., \"40-70% lower energy error at 8x lower computational cost\" as stated in the abstract. It would be clearer to collect these comparisons in one place.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9EgffXp72yK", "reviewer": "Reviewer_g9ta", "summary": "An improved method to approximately solve the Schroedinger equation is described, which combines ideas from the PauliNet and FermiNet papers. A variety of ablation studies are performed. good performance is achieved.", "questions": "Would a study of the transition states of the butadiene system (as in the paulinet paper) be instructive?\n\n\n", "limitations": "yes", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- good results\n- thorough study and recombination of approaches of prior works, without reinventing the wheel\n- ablation studies\n- no grandiose claims\n- good discussion of limitations\n\nWeaknesses:\n- I found the the description of the method quite unclear, and some notation was not very clearly defined where it appeared in the text \n- one could argue the paper is \"only\" a recombination of prior works, but if we take this argument, AlexNet would have needed to be rejected as well.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept