Neural Pfaffians: Solving Many Many-Electron Schrödinger Equations
Authors: Nicholas Gao
Stephan Günnemann
Venue: NeurIPS 2024 oral (2024)
Keywords: Machine Learning for Science
Pfaffian
Neural Network
Molecules
Electrons
Computational Physics
Computational Chemistry
Quantum Chemistry
Quantum Monte Carlo
Variational Monte Carlo
Neural Quantum States
Wave Function
Primary area: machine_learning_for_physical_sciences
TL;DR: We propose a new Pfaffian-based parametrization for generalized neural electronic wave functions.

Abstract: Neural wave functions accomplished unprecedented accuracies in approximating the ground state of many-electron systems, though at a high computational cost. Recent works proposed amortizing the cost by learning generalized wave functions across different structures and compounds instead of solving each problem independently. Enforcing the permutation antisymmetry of electrons in such generalized neural wave functions remained challenging as existing methods require discrete orbital selection via non-learnable hand-crafted algorithms. This work tackles the problem by defining overparametrized, fully learnable neural wave functions suitable for generalization across molecules. We achieve this by relying on Pfaffians rather than Slater determinants. The Pfaffian allows us to enforce the antisymmetry on arbitrary electronic systems without any constraint on electronic spin configurations or molecular structure. Our empirical evaluation finds that a single neural Pfaffian calculates the ground state and ionization energies with chemical accuracy across various systems. On the TinyMol dataset, we outperform the `gold-standard' CCSD(T) CBS reference energies by 1.9m$E_h$ and reduce energy errors compared to previous generalized neural wave functions by up to an order of magnitude.

Neural Pfaffians: Solving Many Many-Electron Schrödinger Equations
Nicholas Gao, Stephan Günnemann
[n. [n.gao,ss guennemann @tum. de
Department of Computer Science Munich Data Science Institute Technical University of Munich
Abstract
Neural wave functions accomplished unprecedented accuracies in approximating the ground state of many-electron systems, though high computational cost. Recent works proposed amortizing the cost by learning generalized wave functions across different structures and compounds instead of solving each problem indepen- dently. Enforcing the permutation antisymmetry of electrons in such generalized neural wave functions remained challenging existing methods require discrete orbital selection via non-learnable hand-crafted algorithms. This work tackles the problem by defining overparametrized, fully learnable neural wave functions suit- able for generalization across molecules. We achieve this by relying on Pfaffians rather than Slater determinants. The Pfaffian allows us enforce the antisymmetry on arbitrary electronic systems without any constraint on electronic spin configu- rations or molecular structure. Our empirical evaluation finds that single neural Pfaffian calculates the ground state and ionization energies with chemical accuracy across various systems. On the TinyMol dataset, we outperform the 'gold-stane gold-standard' CCSD(T) CBS reference energies by 1.9 m En and reduce energy errors compared to previous generalized neural wave functions by up to an order of magnitude.
Introduction
Solving the electronic Schrödinger equation is the heart of computational chemistry and drug discovery. Its solution provides a molecule's 's or material's electronic structure and energy (Zhang et al. 2023). While the exact solution is infeasible, neural networks have recently shown unprecedentedly accurate approximations (Hermann et al. 2023). These neural networks approximate the system's 's the lowest energy state, by minimizing the energy (V)) A ( where H is the Hamiltonian operator, a mathematical description of the system. While such neural wave functions are highly accurate, training has proven computationally intensive.
ground-state wave function V/ Mo
Gao & Günnemann (022)) have shown that training a generalized neural wave function on large class of systems amortizes the cost. However, their approach is limited to different geometric arrangements of the same molecule. Subsequent works eliminated this limitation by introducing hand-crafter algorithms (Gao & Günnemann 2023a) or heavily relying on classical Hartree- Fock calculations (Scherbela et al., (023). Both impose strict, non-learnable mathematical constraints and prior assumptions that may not always hold, limiting their generalization and accuracies. Hand- crafted algorithms only work for limited set of molecules, in particular organic molecules near equilibrium, while the reliance on Hartree-Food empirically results in degraded accuracies.
In this work, propose the Neural Pfaffian (NeurPf) overcome these limitations. As suggested by its name, NeurPf uses Pfaffians to define a superset of the previously used Slater determinants enforce the fermionic antisymmetry. The Pfaffian lifts the constraint on the number of molecular orbitals from Slater determinants (Szbo & Ostlund 2012). enabling overparametrized wave functions
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

with simpler and more accurate generalization. Compared to Globe (Gao & Günnemann 2023a). the absence hand-crafted algorithms enables the modeling of non-equilibrium ionized, or excited systems. By being fully learnable without fixed Hartree-Fook calculations like TAO (Scherbela et al. 2024). NeurPf achieves significantly lower variational energies. Our empirical results show that NeurPf can learn all second-row elements' ground-state, ionization, and electron affinity potentials with a single wave function. Further, we demonstrate that NeurPI's accuracy surpasses Globe on the challenging nitrogen dimer with seven times fewer parameters while not suffering from performance degradations when adding structures to the training set. On the TinyMol dataset, NeurPf surpasses the highly accurate reference CCSD(T) CBS energies the small structures by 1.9 m, and reduces errors compared to TAO by factors 10 and 6 on the small and large structures, respectively.
2 Quantum chemistry
Quantum chemistry aims to solve the iine-independent Schrödinger equation (Foules et 2001
H 1 E )
where V RRN4x3 N333 + the electronic wave function for N. spin-up and N, spin-down electrons, A is the Hamiltonian operator, and E is the system's energy. To ease notation, if not necessary, we omit spins in v/ and treat it w/ RNxx3 + E where N₂ Ng, The Hamiltonian Á for molecular systems, which we are concerned with in this work, is given by
N₂
1
N.
N
Zm
N,
7m 7m
H
H
|.
71
|1:
Rm
Rm
R.
m2m
with P. E I being the ith electron's position, and R'm E R Zm E E being the with nucleus' position and charge. The wave function describes the behavior of electrons in the system defined by the Hamiltonian H. As the square of the wave function 2r2 is proportional to the probability density p(F) x (i) of finding the electrons at positions F E R No its integral must finite:
V (F) < oo.
(3)
Further, as electrons are indistinguishable half-spin fermionic particles, the wave function must be antisymmetric under any same-spin electron permutation т:
V (F")
sgnn(T))sg((())
(4)
To enforce this constraint, the wave function is typically defined as so-called Slater determinant of N₂ Nj integrable so-called orbital functions Φ; IR3 IR:
V Stater (F) = det (r.) det [0))+)] = det **("t) det (+(F+).
(5)
Note that for the determinant to exist, one needs exactly Nf up and N₂ down orbitals of and 0j.
In linear algebra, Eq. (T) is an eigenvalue problem, where look for the eigenfunction Vo with the lowest eigenvalue Eg. In Variational Monte Carlo (VMC), this is solved by applying the variational principle, which states that the energy of any trial wave function v upper bounds Eg:
V) I (
Ep
(6)
1 (F)
By plugging in the probability distribution from Eq.
we can rewrite Eq. (6) as
En < Bp(F)
(F) Ep(r) [F)(
with (F) = (V being the so-called local energy. The right-hand side of Eq. 7 is
known as the variational energy. As Eq. (7) does not require V to be an analytic function, we can approximate the energy of any valid wave function V with samples drawn from p(F). If we pick

---

parametrized family of wave functions Ve we can optimize the parameters & to minimize the VMC energy by following the gradient of the variational energy
Ve p = Ep(r) [E (T) Ep(r) [EL [F))) Vo (F))
(8)
where we approximate all expectations by Monte Carlo sampling (Ceperley et al. 1977).
Neural wave functions typically keep the functional form of Eq_ (5) but replace the orbitals o, with learned many-electron orbitals di IR x RNxx3 + R (Hermann et al. 2023). These many-electron
orbitals deN are implemented as different readouts of the same permutation-equivarian neural
network. Multiplying each orbital by an envelope function X: TR3 + IR that decays exponentially to zero at large distances enforces the finite integral requirement in Eq.
Generalized wave functions solve the more general problem where the nucleus positions R and charges Z are not fixed. Since the Hamiltonian H Rz depends the molecular structure (R, Z), does the corresponding ground state wave function VR.z- Note that we still work in the Oorn-Oppenheimer approximation, i.e., we treat the nuclei as classical point charges (Zhang al. 2023). ). Given a dataset of molecular structures D {(R,, _..... the total energy (1 n, HR. 1 (.z.) is minimized to approximate the ground state for each structure. Typi- 2(,D)ED
( R.z
cally, the dependence on R. Z is implemented by using meta network that takes R. Z as inputs and outputs the parameters of the electronic wave function (Gao Günnemann 2022).
3 Related work
While attempts to enforce the fermionic antisymmetry in neural wave functions in less than O(N," 3) operations promise faster runtime than Slater determinants, the accuracy of these methods is lim- ited (Han et al., 2019 Acevedo et al., 2020) Richter- et al.] (023). PFau et al. (2020) and Hermann al. (2200) established Slater determinants for neural wave functions by demonstrating chemical accuracy on small molecules. Note, Eq. (5) may also be written via a block-diagonal matrix, i... (T) det (diag( 4)). Spencer et al. 2020 's implementation further increased accuracies
by parametrizing the off diagonals that were implicitly set to O before, with additional orbitals क:
Stater (T) = det = det
Several works confirmed the improved empirical accuracy of this approach (Gerard et al., 2022
Lin et al. 2021 Ren al.] 20233 Gao & Günnemann 2023b
2024). While later works refined Wilson et al. 2021 2023), the use of Slater determinants mostly remained a constant, with two notable exceptions: Firstly, Lou et al.)
the architecture to increase accuracy (von Glehn et al.
2023:
(2023) use AGP wave functions (Casula & Sorella 2003
Casula et al., 2004) to formulate the
wave function as (F) = det($*) ) dd+( det( + 4.7" This avoids picking exactly N
orbitals as & and 4+ may be non-square but fails to generalize Eq. (9), we empirically verify the impact of this limitation in App. Secondly, Kiim et al. (2023) introduced the combination of neural networks and Pfaffians, who demonstrated performance on the ultra-cold Fermi gas. Though universal in theory. their parametrization yields no trivial adaption to molecular systems. In classical quantum chemistry, [Bajdich al.) (2006) 2008) reported promising early results with Pfaffians in single-structure calculations for small molecules. In this work, we generalize Eq. p to Pfaffian wave functions that permit pretraining with Hartree-Fock calculations and generalization across molecules.
Generalized wave functions. Scherbela et al. (002)) started this research with a weight-sharing scheme between wave functions. These still had be reoptimized for each structure. Later, Gao & Günnemann 20022 2023b) proposed PESNet, a generalized wave function for energy surfaces allowing joint training without reoptimization. Subsequent works extended PESNet to different compounds where the main challenge is parametrizing exactly N₁ + N₂ orbitals, such that the orbital matrix in Eq. (9) stays square. The problem of finding these orbitals was formulated into a discrete orbital selection problem. |aa & Günnemann (2023a)'s 's hand-crafter algorithm accomplishes this by selecting orbitals via a greedy nearest neighbor search. contrast, (Scherbela et al.) (2024 2023) use the lowest eigenvalues of the Fock matrix as selection criteria. Both introduce non-learnable constraints, limiting generalization or sacrificing accuracy. NeurPf avoids the selection problem by introducing overparametrization when enforcing the exchange antisymmetry.
3

---

4 Neural Pfaffian
Previous generalized wave functions build on Slater wave functions and attempt to adjust the orbitals é; to the molecule. Slater determinants were chosen due to their previously demonstrated high accuracy. However, they require exactly N₁ + N₁ orbitals. While the nuclei allow inferring the total number of electrons N₂ of any stable, singlet state system, the spin distribution into N, and Nj orbitals per atom not readily available. Previous works implement this via a discrete selection of orbitals via non-learnable prior assumptions and constraints on the wave function; see Sec.
Here, present the Neural Pfaffian (NeurPf), superset of Slater wave functions that preserves accuracy while relaxing the orbital number constraint. By not enforcing an exact number of orbitals, NeurPf is overparametrized with No 2 [ax Ni orbitals. avoiding discrete selections and making it natural choice for generalized wave functions. Importantly, NeurPf can be pretrained with Hartree-Fock, which accounts for 99% of the total energy (Szabo & Ostlund 2012). We introduce NeurPf in four steps: (1) We introduce the Pfaffian and use it to define a superset of Slater wave functions.
envelopes that additionally accelerate convergence. (3) We introduce new pretraining scheme for matching Pfaffian and Slater wave functions. (4) We discuss combining our developments to build a generalized wave function.
(2) We present memory-efficien
4.1 Pfaffian wave function
The Pfaffian of a skew-symmetric 2n x 2n matrix A, i.e., A == -A", is defined as
Pf(A) n"n!
Σ sgn(r) II A.(21-1),((2)
(10)
TES2n
where S2n is the symmetric group of 2n elements. One may consider it a square root of the determinant of A since Pf(A)² = det(.A). An important property of the Pfaffian is Pf(BAB") det(B))f(( for any invertible matrix B and skew-symmetric matrix A. In the context of neural wave functions, this means that if A is an along both dimensions permutation equivariant function of the electron positions F, AFr(F)) PrA(F) PT the Pfaffian of A is a valid wave function that fulfills the antisymmetry requirement from Eq.
L (T(F)) = PF(A(F(F)) = PI(P::
(P))P((A(T) = sign(T)
(11)
To compute the Pfaffann without evaluating the 2n! terms in Eq. TT we implement the Pfaffian via tridiagonalization with the Householder transformation in Wimmer (0012).
There are various ways to construct (Bajdich al. 2006 2008 Kim al., 2023 Here, we
introduce a superset of Slater wave functions, enabling high accuracy on molecular systems. If A is
skew-symmetrie matrix, so is BAB" for any arbitrary matrix B. Thus, we can construct Prattan as
Praflian (F) = Pf(AAr)
Pf ((x(())xx(xx(()))
(12)
where Apr Nox. is learnable skew-symmetric matrix and बे. N.x3 NoN.. is permutation equivariant function like in Eq. (9). This construction elevates the need for having exactly / orbitals as in Slater determinants. We may now overparametrize the wave function with No 2 max{NN (Nr, Nij orbitals, allowing for a more flexible and simpler implementation without needing discrete orbital selection. By choosing Øer 4. it is straightforward superset of the Slater determinant wave function in Eq. (9). Note that, like in Eq. (9), we parametrize two sets of orbital functions &pr and Ppr and change their order for spin-down electrons to not enforce the exchange antisymmetry between different- spin electrons. As the normalizer Pf (Apr) is constant, we drop it going forward. As it is common in quantum chemistry (Szabo & Ostlund [0012] Hermann al. 2020 use linear combinations of wave functions increase expressiveness:
that Eq. [12 is
T Praftian
Σ
Ck L Prafliam, & (F).
(13)
We visually compare the schematic of the Slater determinant and Pfaffian wave functions in Fig.
In App. A discuss how to handle odd numbers of electrons such that &p has even

---

P
Σ
Inpui
Dibitab
$ 1i7
$..
learnablee
+
Amisymmetioo
Apr
fixed
+
@-1
$1.7
$1.77
Concstenate
Concataiste
+
+
+
d
> Xn
Ding Projection
Orddhag Projection
Clomaatton
Diag Projection
Offiiigg Prejection
Teno
NN
NN
Envelopa
Envelope
Electrons
Floctrons
(a) Slater wave function
(b) Neural Pfaffian
Fig. 1: Schematic of the Slater determinant (1a) and our NeurPf 1b). Where the Slater for- mulation requires exactly Ne orbital functions, the Pfaffian formulation works for any number No 2 NN, of orbital functions, indicated by the rectangular orbital blocks.
dimensions. Like previous work (Pfau et al., 2020 we parametrize the orbital functions d; as a product of permutation equivariant neural network h R x RN3 + IR and an envelope function x R + R:
- ((F wkk Wki T/os
N4
(14)
Nr-N, R being a scalar depending on with wki R. being learnable weight vector, and T/. the spin state of the system, i.c., the difference between the number of up and down electrons. The envelope function X ensures that the integral of the squared wave function finite. For h, we use Moon from Gao & Günnemann 2023a) thanks to its size consistency.
4.2 Memory-efficient envelopes
To satisfy the finite integral requirement on the square V in Eq. (3). the orbitals e are multiplied by an envelope function X R - IR that exponentially decays to zero at large distances. We do not split spins here and work with N. = N₂ + N₁ to simplify the discussion, but, in practice, we would split the envelopes into two sets, one for Opr and one for Ppp. The envelope function is typically a sum of exponentials centered the nuclei (Spencer al. 2020). In Einstein's 's summation notation, the envelope function can be written as
(FojF)))
Khmi
exp( Kemi ksm
Fbz
RSm DoRkmj
(15)
N.
No N.
where Nb denotes the batch size. Empirically, we found the tensor the right side containing many redundant entries. Further, due to the nonlinearity of the exponential function, one cannot implement the envelope in simple matrix contraction but has materialize the full five-dimensional tensor. NeurPf amplifies this problem as N. Z N₂ whereas Slater determinants constraint N, N.
We use a single set of exponentials and nucleus. This reduces the number of envelopes per electron from N₂ x Nr, N, to N₂ Nanv. where Nenv N₂ Neny/muc /ue is the number of envelope functions. In general, we pick N Nenv/mut
nucleus instead of having one for each combination of orbital
5

---

such that Nenv = No. These atomic envelopes are linearly recombined into molecular envelopes, effectively enlarging π to N₁ x N, Neny tensor. Thanks to these rearrangements, we avoid constructing a five-dimensional tensor. Instead, we define the envelopes as
(ba(F
Tkni
Rm 1)
(16)
Neav x No
Nr x X N.
Concurrently, |fau et al. (024) presented similar bottleneck envelopes. However, found ours to converge faster and not yield numerical instabilities. We discuss this further in App. and
4.3 Pretraining Plaffian wave functions
Pretraining is essential in training neural wave functions and has frequently been observed to critically affect final energies (Gao & Günnemann, 2023a; von Glehn et al. [023 Gerard et al. 2022). The pretraining aims to find orbital functions close to the ground state to stabilize the optimization. Traditionally, this done by matching the orbitals of the neural wave function to the orbitals of a baseline wave function, typically Hartree-Fock wave function TH det(Prr:). by solving
min | Pur
(17)
0
for the neural network parameters (Plau al. 2020). Since our Pfaffian has N, orbitals while Hartree-Fock has No, we cannot directly apply this to our Pfaffian wave function. Further, as we predict orbitals per nucleus, our arbitrary orbital order may not align with Hartree-Fock.
We propose two alternative pretraining schemes for neural Pfaffian wave functions: one based on matching single-clectron orbitals and one based on matching geminals, effectively two-electron orbitals. We need to expand the Hartree-Fock orbitals a to No orbitals to match the single-electron orbitals directly. We construct 4 by padding the extra No - N₂ orbitals with zeros. It can easily be verified that the wave function иFr- P
1 Pf( Pf( Pirr), is equivalent to the original Hartree-Fock wave function, i.e., Vir Vn det (fir) for any invertible skew-symmetric Aur- Further, note that the multiplication of Pur with any matrix T E SO(N.) from the special orthogonal group does not change WHE-P Thus, it suffices to match the single electron orbitals of der and * up to rotation T E SO(N))) yielding the following optimization problem:
min min 0 TESO(N.)
|| |ppr- T12
(18)
We solve this alternatingly for T and 0. To match the geminals Apr and Pur Aur we have account for the fact that the choice of Aur is arbitrary long it is skew-symmetric and invertible. Again, we solve this optimization problem alternatingly by solving for AμP E S = {A E SO(N)) } and O:
min min | A
4HP A
(19)
While both formulations share the same minimizer, combining both yields the most stable results. We hypothesize that this is because the single-electron orbitals are more stable than the geminals and thus provide better starting point for the optimization. In contrast, the latter provides a closer formulation of the neural network orbitals. Thus, we pretrain our neural Pfaffian wave functions by solving the optimization problem
min (a min 0 T + 3 min 1 (a TESO(N.)
P D Frr
(20)
0
Agr8
with weights a, 3 E [0, 1]. To optimize over the special orthogonal group SO(N.), we use the Cayley transform (Galleer 2013). App. further details the procedure.
4.4 Generalizing over systems
We now focus on generalizing the construction of our Pfaffian wave function for different systems. We accomplish the generalization similar to PFSNet (Giao & Günnemann] 2022) by introducing a second neural network, the MetaGNN M (IR3 x N. N.. -> θ that acts upon the molecular structure, i.e., nuclei positions and charges, and parametrizes the electronic wave function P'ratian Ne33 x + R for the system of interest. As architecture for the wave function and MetaGNN, we use the same architecture as in Gao et al. (2023a) with the exception being that we replace the Slater determinant with the Pfaffian as described in Sec. and minor tweaks highlighted in App. D.4

---

Pfaffian. To represent wave functions of different systems within a single NeurPf, we need to adapt the orbitals 6er and antisymmetrizer Apr from Eq. 122 to the molecule. In doing so, we must ensure No 2 max(N,, N.
Otherwise, $pAApr is singular, and the wave function is zero. One may solve this by picking No large enough that No max(NV,, N2} for all molecules in the dataset. However, this is computationally expensive, does not reuse known orbitals in the problem, and simply moves the problem even larger systems. Instead, we grow the number of orbitals No with the system size by defining Norb/muc orbitals per nucleus, as depicted in Fig. This allows us to transfer orbitals from smaller systems to larger systems. We only need to ensure that Norb/nue is larger than half the maximum number of electrons in period, e.g.. for the first period Norb/nue 2 1, for the second period Norb/nue babb/nuc" 2 5.
Moon
MetaGNN
Orbilals
+
Pp
Fig. 2: Orbital parametrization per nucleus. O O/O in- dicate electrons and nuclei, respectively.
The projection W from Eq. (14) and the envelope decays a are parametrized by node embeddings, while the envelope weights т and the antisymmetrizer Apr are derived from edge embeddings. We predict Norb/nuc Ne matrix per nucleus for W and a Neav/nuc vector per nucleus for σr. For the edge parameters π and Apr. we predict a Nenw/nuu /nue x Norb/nue and Norb/nuc X Norb/nue matrix per edge, respectively. These are concatenated into the Nenv x No and No x No matrices π and Apr. The latter is antisymmetrized to get Apr = 2 (Apr - ApT. We parametrize the spin-dependent scalars yj as node outputs for a fixed number spin configurations No. Because the change spin configuration does not grow with system size, N₂ is fixed. We generate two sets of these parameters, on for 4pr and on for Ppr. App. provides definitions for the wave function, the MetaGNN, and the parametrization.
Pretraining. Previous work like like/Gao& & Günnemann (2023a) needed to canonicalize the Hartree-Fock solutions for different systems before pretraining to ensure that the orbitals fit the neural network. Alternatively, SScherblaeerbeaaaeaa et al. (2023) relied on traditional quantum chemistry methods like Foster & Boys (960)'s 's localization to canonicalize their orbitals in conjunction with sign equivariant neural networks. In contrast, we ensure that the transformed Hartree-Food orbitals are similar across structures as we optimize T E SO(N)) and Aμr S for each structure separately, which simultaneously also accounts for arbitrary rotations the orbitals produced by Hartree-Fock.
Limitations. While our Pfaffian-based generalized wave function significantly improves accuracy on organic chemistry, we leave the transfer to periodic systems for future work (Kosmala et al. 2023). Further, due to the lac of low-level hardware/softwart support for the Pfaffian and the increased number of orbitals No 2 max{N; Nr, Ni). our Pfaffian is slower than comparably-sizer Slater determinant. While we solve the issue of enforcing the fermionic antisymmetry, our neural wave functions are still unaware of any symmetries of the wave function itself. These are challenging to describe and largely unknown, but their integration may improve generalization performance (Schütt jtt al. 2018). Finally, classical single-structee calculations, NeurPf may not improve accuracies. App.1 discusses the broader impact of our work.
5 Experiments
In the following, we evaluate NeurPf on several atomic and molecular systems by comparing it to Globe (Gao & Günnemann 2023a) and TAO (Scherbela et al., 2024). Concretely, investigate the following: (1) Second-row elements and their ionization potentials and electron affinities. Globe cannot compute these due to its restriction to singlet state systems. (2) The challenging nitrogen potential energy surface where Globe significantly degraded performance when enlarging their training set with additional molecules. (3) The TinyMol dataset (Scherbela al. (2024) evaluate NeurPI's generalization capabilities across biochemical molecules. In interpreting the following results, one should mind the variational principle, i.e., lower energies are better for neural wave functions. Further, 1 kcal mol =1.6 6 m F, is the typical threshold for chemical accuracy.
Like previous work, optimize the neural wave function using the VMC framework from Sec. 2 We precondition the gradient with the Spring optimizer (Goldshlager et al. 2024). App. E details the setup further. App. and
experiment on extensity and additional ablations.
Atomic systems and spin configurations. We evaluate NeurPf on second-row elements and their ionization potentials and electron affinities. These systems are particularly interesting as they represent
7

---

Be
B
C
Ne
Ground (EL)
10
10
Ion. (Er)
10.
10
10-1
Alt. (Ea)
Unphysical System
Unphysical System
TermiNos chemical Aaaiaalee. Avc.
Our
10F
10P
10P
10F
10
10a
10P10P
10? 10
10
10a
102
10
109102
102
10 102
Fig. 3: Ground state, electron affinity, and ionization potential errors of second-row elements during training. A single NeurPf has been trained on all systems jointly while references (Ptau al. 2020) were calculated separately for each system. Energies are averaged over the last 10% of steps.
a wide range of spin configurations. We cannot use Globe on such systems because they differ from the singlet state assumption. Instead, compare our results to the single-structure calculations from Pfau et al. 22200's ' FermiNet and the exact results from Chakravorty et al. (1933); Klopper et al. (20T0). In App. G we repeat this experiment for metals.
Fig. 3 displays the ground state energy, clectron affinity, and ionization potential errors of NeurPr during training compared to the reference energies from Pfau et al. (2020); Chakravorty al. (1993);
Klopper et al. 2010). is apparent that NeurPf reaches chemical accuracy relative to the exact results while only training single neural network for all systems. While separately optimized FermiNets may achieve lower errors, Pfau et al. (2020) trained 21 neural networks for 200k steps each compared to single NeurPf trained for 200k steps. i.c., 21 times fewer steps and samples. Whereas Gao & Günnemann (023a);; Scherbela et al. (2023) focus on singlet state systems or stable biochemical molecules, NeurPf demonstrates that generalized wave function need not be restricted to such simple systems and can even generalize to a wide range of electronic configurations.
Effect of uncorrelated data. Next, we evalu- ate NeurPf on the nitrogen potential energy sur- face, traditionally challenging system due to its high electron correlation effects (Lyakh et al. 2012). This is particularly interesting Gao & Günnemann (2023)) observed a significant accu- racy degradation when reformulating their wave function to generalize over different systems. In particular, they found that training only the nitrogen dimer leads to significantly lower errors than training with an ethene-augmented dataset, indicating an accuracy penalty in generalization.
our
Enarge arron (mdbl))
(Ethene)
Globe (Ethene)
FermiNei
PESNet
Experinment
Distance
Fig. Potential energy surface of nitrogen. Ener- We replicate their setup and compare the perfor- gies are relative to Le Roy et al. (2006).
mance of NeurPf trained on the nitrogen energy surface with and without additional ethene structures. Like Gao & Günnemann (002aa), the nitrogen structures are taken from Pfau al. (2020) and the ethene structures from Scherbela et al. (2022) As additional references, we plot Gao & Günnemann
(2022)* PESNet and Fu al. (2023)* FermiNet results.
Fig. shows the error potential energy surface relative to the experimental results from Le Roy et al. (2006). NeurPf reduces the average error on the energy surface from Globe's 2.7 mFF,, 2 m when
training solely on nitrogen structures. When adding the ethene structures, Globe's error increases 5.3 mži while NeurPI's error stays constant
m m lower error than the Globe without the augmented dataset. These results indicate NeurPt's strong capabilities in approximating ground states while allowing for generalization across different systems without a significant loss in accuracy.

---

TinyMol dataset. Finally, we look at learn- ing a generalized wave function over different molecules and structures. We use the Mon dataset (Scherbela et af., 2024), consisting of a small and large dataset. The dataset includes 'gold- standard' CCSD(T) CBS energies. The small set consists of 3 molecules with 2 heavy atoms, while the large set covers 4 molecules with 3 heavy atoms. For each molecule, 10 structures are provided. Here, we compare again both Globe (+Moon) and TAO to NeurPf. All models are di- rectly trained on the small and large test sets.
Small molecules
10a
H
C=0
10a
E Eecsorn cass (ur 4
H C=N
10a
10t
10"
Fig. 5 shows the mean energy difference to CTSD(T) at different stages of the training. We refer to App. K for per molecule error attribu- tion. It is apparent that NeurPf yields lower errors than the TAO and Globe after at least 500 steps. On the small structures, NeurPf even matches the CCSD(T) baseline after 16k steps and achieves 1.9 En lower energies after 32k steps. Since VMC methods are variational, i.e., lower energies are always better, NeurPf is more accurate than the CCSD(T) CBS reference. Compared TAO and Globe, NeurPf reports 5.9 F and 11 11.3 F/E, lower energies, respectively. On the large struc- tures, we observe a similar pattern where we find NeurPf having 25 times smaller error than TAO during the early stages of training and reaching E lower energies after 32k steps a 6 times lower error compared to the CCSD(T) base- line. Note that since the CCSD(T) (CBS) energies are neither exact nor variational, the true error to the ground state is unknown. Still, we provide additional numbers for NeurPf trained for 128k steps in App. K There, we find NeurPf yielding 4.4 E, lower energies on the large structures. These results show that generalized wave func- tion can achieve high accuracy various molecu- lar structures without pretraining when not relying on hand-craftee algorithms or Hartree-Fock calculations. For additional experiments, refer the reader to App. D where we first pretrain TAO and NeurPf on a separate training set and, then, finetune on the small and large test sets and App. M for comparison of joint and separate optimization.
Shaded region:
Improvement over CCSD(T) CBS
Large molocules
10
11
10*
Eocapyn cas (m.F))
NH
C
NI C=
-=C-0
10
10a
our
CCSD(T)
TAO
Globe
10a
250
500
1k
2k
4k
8k
16k
32k
Training Step
Fig. 5: Convergence of mean energy difference on the TinyMol dataset from [Scherbela et al. (2024). The y-axis is linear < and logarithmic Due to the variational principle, NeurPf is better than the reference CCSD(T) on the small molecules.
6 Conclusion
In this work, we established a new way of parametrizing neural network wave functions for general- ization across molecules via overparametrizatioo with Pfaffians. Our Neural Pfaffian is more accurate, simpler to implement, fully learnable, and applicable to any molecular system compared to previous work. The wave function changes smoothly with the structure, avoiding the discrete orbital selection problem previously solved via hand-crafted algorithms or Hartree-Fock. Additionally, we introduced memory-efficient implementation of the exponential envelopes, reducing memory requirements while accelerating convergence. Further, we presented a pretraining scheme for Pfaffians enabling initialization with Hartree- Fock a crucial step for molecular systems. Our experimental evaluation demonstrated that our Neural Pfaffian can generalize across different ionizations of various systems, stay accurate when enlarging datasets, and set a new state of the art by outperforming previous neural wave functions and the reference CCSD(T) CBS on the Tony dataset. These develop- ments open the door for new neural wave functions applications, e.g., to generate reference data for machine-leamee force fields density functional theory (Cheng et al. 2024 Gao et al. 2024).

---

Acknowledgments. We greatly thank Simon Geisler for our valuable discussions. Further, we thank Valerie Engelmayer, Leo Schwinn, and Aman Saxena for their invaluable feedback on the manuscript. Funded by the Federal Ministry of Education and Research (BMBF) and the Free State of Bavaria under the Excellence Strategy of the Federal Government and the Länder.
References
Acevedo, A., Curry, M., Joshi, S. H., Leroux, B., and Malaya, N. Vandermonde Wave Function Ansatz for Improved Variational Monte Carlo. In 2020 IEEE/ACM Fourth Workshop Deep Learning on Supercomputers (DLS), pp. 40-47, November 2020. doi:
Bajdich, M., Mitas, L., Drobný, G., Wagner, L. K., and Schmidt, K. E. Pfaffian Pairing Wave Functions in Electronic-Structur Quantum Monte Carlo Simulations. Physical Review Letters, 96 (13):130201, April 2006. ISSN 003 0031-9007, 1079-7114 doi: 10. 10.11005Physseeeectt......00000.
Bajdich, M., Mitas, L.., Wagner, L. K., and Schmidt, K. E. Pfaffian pairing and backflow wavefunctions for electronic structure quantum Monte Carlo methods. Physical Review B, 77(11):11512, March 2008. doi: 10. 100//Physee......1155.
Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milnn S., and Zhang, Q. JAX: Composable transformations of Python+NumlPy programs, 2018.
Casula, M. and Sorella, S. Geminal wavefunctions with Jastrow correlation: first application to atoms. The Journal of Chemical Physics, 119(13):6500666 6511, October 2003. ISSN 0021-9606, 1089-7690. doi: 10. 11.0//6.11133777
Casula, M., Attaccalite, C., and Sorella, S. Correlated geminal wave function for molecules: An efficient resonating valence bond approach. The Journal of Chemical Physics, 121(15);7110-7122 October 2004. ISSN 0021-9606. doi: 10. 17749/.
Ceperley, D., Chester, G. V., and Kalos, M. H. Monte Carlo simulation of a many-fermion study. Physical Review B, (07))3081-30999 October 1977. doi: 10. B..10//yyyeev....... 16.0081.
Chakravorty, S. J., Gwaltey, S. R., Davidson, E. R., Parpia, F. A., and p Fischer, C. F. Ground-state correlation energies for atomic ions with to 18 electrons. Physical Review A, (7(5))6649933777 May 1993. doi: 10. 103/PhysRev..47..60
Cheng, L., Szabó, P. B., Schätzle, Z., Kooi, D., Köhler, J., Giesbertz, K. H., Noé, F., Hermann, J.. Gori-Giorgi, P., and Foster, A. Highly Accurate Real-space Electron Densities with Neural Networks, September 2024.
Foster, J. M. and Boys, S. F. Canonical Configurational Interaction Procedure. Reviews of Modern Physics, 32(2):300-302 April 1960. ISSN 0034-6861. doi: N0.100//eeMMoPyy..2...0
Foulkes, W. M. C.,, Mitas, L.,, Needs, R. J., and Rajagopal, G. Quantum Monte Carlo simulations of solids. Reviews of Modern Physics, (3(1):33-833 January 2001. doi: 10. Moddee 73..33
Fu, W., Ren, W., and Chen, Variance extrapolation method for neural-network variational Monte Carlo, August 2023.
Gallier, J. Remarks on the Cayley Representation of Orthogonal Matrices and on Perturbing the Diagonal of Matrix Make it Invertible, November 2013.
Gao, N. and Günnemann, S. Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions. In International Conference Learning Representations, April 2022.
Gao, N. and Günnemann, S. Generalizing Neural Wave Functions. International Conference on Machine Learning, February 2023a. doi: 10.485550ar2x..2200..11111
Gao, and Günnemann, S. Sampling-free Inference for Ab-Initio Potential Energy Surface Networks.
In The Eleventh International Conference on Learning Representations, February 2023b.
10

---

Gao, N. and Günnemann, S. On Representing Electronic Wave Functions with Sign Equivariant Neural Networks. In ICLR 2024 Workshop on A14DifeernnialEquatiins Science, March 2024.
Gao, N., Köhler, J., and Foster, Folx Forward Laplacian for JAX, 2023.
Gao, N., Eberhard, E., and Günnemann, Learning Equivariant Non-Local Electron Density Functionals, October 2024.
Gasteiger, J., GroB, J., and Günnemann, S. Directional Message Passing for Molecular Graphs, In International Conference on Learning Representations. September 2019.
Gerard, L., Scherbela, M., Marquetand, P., and Grohs, Gold-standard solutions to the Schrödinger equation using deep learning: How much physics do we need? Advances in Neural Information Processing Systems, May 2022.
Goldshlager, G., Abrahamsen, N., and Lin, L. A Kaczmarz-inspired approach to accelerate the optimization of neural network wavefunctions, January 2024.
Han, J., Zhang, L.. and E. W. Solving many many-electron Schrödinger equation using deep neural networks. Journal of Computational Physics, 399:108929, December 2019, ISSN 0021-9991. doi:
10.1099 108922..
Hermann, J., Schätzle, Z., and Noé, F. Seep-neural-nettee solution of the electronic Schrödinger equation. Nature Chemistry, 12(10):891-88) 897,, October 2020. ISSN 1755-4330, 1755-4349. doi: 10.10388/41157-0000004444
Hermann, J., Spencer, J., Choo, K., Mezzacapo, A., Foulkes, W. M. C., Pfau, D., Carleo, G., and Noé, F. Ab initio quantum chemistry with neural-network wavefunctions. Nature Reviews Chemistry, (10):692-709, October 2023. ISSN 2397-3358. doi: 10. 10000///111000000000000010
Kim, J., Pescia, G., Fore, B., Nys, J., Carleo, G., Gandolfi, S., Hjorth-Jensen, M., and Lovato, A. Neural-network quantum states for ultra-cold Fermi gases, May 2023.
Klopper, W. Bachorz, R. A., Tew, D. P., and Hättig, C. Sub-meV accuracy in first first-principles computations of the ionization potentials and electron affinities of the atoms H to Ne. Physical Review A, (22200222503, February 2010. ISSN 1050-2947, 1094-1622. doi: 10. 10../ 81.022503.
Kosmala, A., Gasteiger, J., Gao, N., and Günnemann, S. Ewald-based Long-Range Message Passing for Molecular Graphs. Proceedings of the 40th International Conference on Machine Learning, pp. 17544-17563. PMLR, July 2023.
Le Roy, R. J.. Huang, Y., and Jary, C. An accurate analytic potential function for ground-state N2 from direct-potential-iit analysis spectroscopic data. The Journal of Chemical Physics, 125 (16):164310) October 2006. ISSN 0021-9606, 1089-7690. doi: 10.1063/1.235444
Li, R., Ye, H., Jiang, D., Wen, X., Wang, C., Li, Z., Li, X., He, D., Chen, J., Ren, W., and Wang, L. A computational framework for neural tetwork-baseee variational Monte Carlo with Forward Laplacian. Nature Machine Intelligence, 6(2):209-219, February 2024. ISSN 2522-5839. doi: 00.10888442256-004-007444
Lin, J., Goldshlager, G., and Lin, L. Explicitly antisymmetrizeed neural network layers for variational Monte Carlo simulation. arXiv:2112.0344 [physics], December 2021.
Lou, W. T., Sutterud, H., Cassella, G., Foulkes, W. M. C., Knolle, J., Pfau, D., and Spencer, J. S. Neural Wave Functions for Superfluids, July 2023.
Lyakh, D. I., Musiat, M., Lotrich, V. F., and Bartlett, R. J. Multireference Nature of Chemistry: The Coupled-Cluster View. Chemical Reviews, 11 112(1):1822243, January 2012. ISSN 0009-2665, 1520-6890. doi: 10. 10.1001/c20011117.
Martin, W. C. and Musgrove, Ground levels and ionization energies for the neutral atoms. 1998.
Mishchenko, K. and Defazio, Prodigy: An Expeditiously Adaptive Parameter-Free Learner, October 2023.

---

Motta, M., Ceperley, D. M., Chan, G. K.-L., Gomez, J. A., Gull, E., Guo. S., Jiménez-Hoyos, C. A.. Lan, T. N.. Li, J., Ma, F., Millis, A. J., Prokof'et ev, N. V., Ray, U., Scuseria, G. E., Sorella, S.. Stoudenmire, E. M., Sun, Q. Tupitsyn, I. S., White, S. R., 7gid, D., Zhang. S., and Simons Collaboration on the Many-Flectron Problem. Towards the Solution of the Many-Flectron Problem in Real Materials: Equation of State of the Hydrogen Chain with State-of-the-AN Many-Body Methods. Physical Review X, 7(3):031059, September 2017. ISSN 2160-3308. doi: 10.1103/ PhysRevX.7.031005
Pfau, D., Spencer, J. S., Matthews, A. G. D. G., and Foulkes, W. M. C. Ab initio solution of the many-electron Schrödinger equation with deep neural networks. Physical Review Research, 2(3): 033429, September 2020. doi: 10. 10.1: 10spPhysReekssearch2.233333
Pfau, D., Axelrod, S., Sutterud, H., von Glehn, I., and Spencer. J. S. Accurate computation of quantum excited states with neural networks. Science, 385(671 1): 377, August 2024, doi: 10.11666ciencc..d001133.
Ren, W., Fu, W., Wu, X., and Chen, J. Towards the ground state of molecules via diffusion Monte Carlo neural networks. Nature Communications, 14(1):1860, April 2023. ISSN 2041-1723. doi: 10. 10.1038/s41467-11 46700033144--0---7669999-
Richter-Powell, J., Thiede, L.,, Asparu-Guzik, A., and Duvenaud, D. Sorting Out Quantum Monte Carlo, November 2023.
Scherbela, M., Reisenhofer, R., Gerard, L., Marquetand, P., and Grohs, P. Solving the electronic
Schrödinger equation for multiple nuclear geometries with weight-sharing deep neural networks. Nature Computational Science, 2(5):331-341, May 2022. ISSN 2662-8457. doi: 10.1038/ 035588-022-00228-x.
Scherbela, M., Gerard, L., and Grohs, P. Variational Monte Carlo trained Neural Wavefunctions. In Thirty-Seventh Conference on Neural Information Processing Systems, November 2023.
Pondeet-Fine-tnntnn pre-
Scherbela, M., Gerard, L.., and Grohs, P. Towards transferable fermionic neural wavefunction for molecules. Nature Communications, 15(1):120, January 2024. ISSN 2041-1723. doi: 10. 1031/
Schütt, K. T., Sauceda, H. E., Kindermans, P.-J., Tkatchenko, A., and Müller, K.-R. SchNet - A deep learning architecture for molecules and materials. The Journal of Chemical Physics, 148(24): 241722, June 2018. ISSN 0021-9606, 1089-7690. doi:
Shazeer, N. GLU Variants Improve Transformer, February 2020.
Spencer, J. S., Pfau, D., Botev, A., and Foulkes, W. M. C. Better, Faster Fermionic Neural Networks.
3rd NeurIPS Workshop on Machine Learning and Physical Science, November 2020.
Szabo, A. and Ostlund, N. S. Modern Quantum Chemistry: Introduction Advanced Electronic
Structure Theory. Courier Corporation, 2012.
von Glehn, I., Spencer, J. S., and Pfau, Self-Attention Ansatz for Ab-initio Quantum Chemistry. In The Eleventh International Conference Learning Representations, February 2023.
Wilson, M., Gao, N., Wudarski, F., Rieffel, E., and Tubman, N. M. Simulations of state-of-the-an fermionic neural network wave functions with diffusion Monte Carlo, March 2021.
Wilson, M., Moroni, S., Holzmann, M., Gao, N., Wudarski, F., Vegge, T., and Bhowmik, A. Neural network ansatz for periodic wave functions and the homogeneous electron gas. Physical Review B, 107(23):235139, June 2023. doi: 10. D0.1103/PyssevBB.. 107.2351
Wimmer, M. Algorithm 923: Efficient Numerical Computation of the Pfaffian for Dense and Banded Seee-Syymetric Matrices. ACM Transactions on Mathematical Software, 38(4):30:1-30:17, August 2012. ISSN 0098-3500. doi: 10. 10.1145/2311300233133
You, Y., Li, J., Reddi, S., Hseu, J., Kumar, S., Bhojanapalli, S., Song, X., Demmel, J., Keutzer, K., and Hsieh, C.-J. Large Batch Optimization for Deep Learning: Training BERT 76 minutes. In Eighth International Conference on Learning Representations, April 2020.
12

---

Zhang, X., Wang, L., Helwig, J., Luo, Y., Fu, C., Xie. Y., Liu, M., Lin, Y., Xu, Z., Yan, K., Adams,
K., Weiler, M., Li, X., Fu, T., Wang, Y., Yu, H., Xie. Y., Fu, X., Strasser, A., Xu, S., Liu, Y., Du, Y., Saxton, A., Ling, H., Lawrence, H., Stärk, H., Gui, S., Edwards, C., Gao, N., Ladera, A., Wu, T., Hofgard, E. F., Tehrani, A. M.. Wang, R., Daigavane, A., Bohde, M., Kurtin, J., Huang, Q.. Phung. T., Xu, M., Joshi, C. K., Mathis, S. V., Azizzadenesheli, K., Fang, A., Aspuru-Guzik, A., Bekkers, E., Bronstein, M., Zitnik, M., Anandkumar, A., Ermon, S., Liò, P. Yu, R., Günnemann, S., Leskovec, J., Ji, H., Sun, J., Barzilay, R., Jaakkola, T., Coley, C. W., Qian, X., Qian, X., Smidt, T., and Ji, S. Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems, November 2023.
13

---

Table Number of envelope parameters for the full envelope and our memory efficient envelopes for an explanatory system.
π
Total
full our 640
1600
1600
3200
12800
13400
A Odd numbers of electrons
To handle odd numbers of electrons, extend the electron pair matrix ΦPtAPr even dimensions. We accomplish this by augmenting Apr with a learnable single-electron orbital Podd
Apr Ppr Apr
Pr
Oodd 0
(21)
o
To obtain a single additional orbital for the whole molecule, parameterize one orbital Podd, for each nucleus as in Eq. (4) and sum them up to obtain Podd = Σm Podd,
B Difference bottleneck envelopes
Similar to the bottleneck envelope from Piau et al. 2024), our efficient envelopes aim at reducing memory requirements. The bottleneck envelopes are defined
l Xbattleneck (rbz); Σ
Tim cxp (-00mmllll | R.m
(22)
While both methods share the idea of reducing the number of parameters, they differ in their implementation. Whereas the bottleneck envelopes construct a full set L. many-nuclei envelopes and then linearly recombine these to the final envelopes for each of K x N, orbitals, our efficient envelopes construct the final envelopes directly from a set of single-nuclei exponentials. Further, use different set of basis functions for each of the K determinants. In terms of computational complexity, the bottleneck envelopes require O O(N, N,, + O(K envelopes, while our efficient envelopes require O N..N.) operations. In practice, we found our efficient envelopes to be faster and converge better on all systems we tested. An ablation study is presented in App. Further, we observed numerical instabilities our envelopes as reported by Pfau et al. 2024
operations to compute the
Compared to the full envelopes, find memory efficient ones be slower but yielding better performance. This is likely due to the increased number of wave function parameters. The number of parameters for the full envelopes and our memory efficient envelopes is shown in Tab. for an example with N, = N. 20, N₂ 5, Na 16, N an = 8. The full envelopes' a, π scale both ONNaNN,NNo) while our memory efficient envelopes' a scales O.NNNNn.Newww/en O(NaN, Nenv/ /uu///neoo. No). In runtime, the full envelopes require ONNaN, N.. operations, while our memory efficient envelopes require ONNNN, m N.N..) operations. memory complexity, the full envelopes require N2", while our memory efficient envelopes require O(NaN N an. No).
and π scales
C Pretraining
To pretrain NeurPf, we solve the optimization problem from Eq. 20 The nested optimization problems are solved iteratively, where we first solve for E SO(N and AHF $ and then for the parameters of the wave function 0. We describe how we parametrize the special orthogonal group SO(N) and the antisymmetric special orthogonal group S and then how we solve the optimization problems.
To optimize over the special orthogonal group SO(N), we parametrize T via some arbitrary matrix T E RNX, Next, we obtain an antisymmetrized version of T via
T= 1 - (
(23)
14

---

We now may use T with the Cayley transform to obtain special orthogonal matrix
T= = f - (f+1) 7-
(24)
where / is the identity matrix. 7 is now a special orthogonal matrix where all eigenvalues are 1. To parametrize matrices with an even number of eigenvalues -1 as well, we simply multiply T with itself:
T TT
(25)
which gives us our final parametrization of the special orthogonal group SO(N) (Gallier 2013).
We follow Gallier ), to parametrize antisymmetric special orthogonal matrices S. In particular, parametrize some 7 using the procedure outlined above. To parametrize Aur. remains to
antisymmetrize 7 while preserving the special orthogonal property. We accomplish this by defining
Aur = TÎTT
(26)
where
0
1
0
0
0
0
0
diag ( ....)
0
0
0
0
1
(27)
0
0
-1
is the antisymmetric identity matrix. Since the product of special orthogonal matrices is special orthogonal and BAB" yielding an antisymmetric matrix for any special orthogonal matrix B, we have that Aur S is antisymmetric special orthogonal matrix.
Now that we can parametrize both groups with real matrices, we can simplify the optimization problem by performing gradient optimization for both T, AμF. and 6. We solve this problem alternatively, where we first solve for T and AμF by doing Npre steps of gradient optimization with the prodigy optimizer (Mishchenko & Defazio 2023) and then perform a single outer step on 0 with lamb optimizer (You et al. 2020) like previous works (Gao & Günnemann
2022 von Glehn
al. 2023
D Model architectures
We largly reuse the same architecture for the MetaGNN M (IR3 N4) θ and wave function praman IR N.x3 x θ + R as Gao & Günnemann (2023a). We canonicalize all molecular structures using the equivariant coordinate frame from Gao Günnemann (2022).
D.1 Wave function
Similar to Gao Günnemann (023a)), we use bars above functions and parameters to indicate that the MetaGNN M parameterizes these and that they vary by structure. We define our wave function as a -faffiw-PPaffian wave function like Kim et al. (2023):
K
(r) = cxp (J(r) Σ P& (7
(28)
As Jastrow factor J. R Nx3 IR embeddings and the fixed electronic cusp Jastrow from von Glehn et al. (202)):
linear combination of learnable MLP of electron
((r) Σ (L)(
1=1
+ Spar
1
par
2
(29)
4 Cepar + ri - r
Bami
Σ
Oranti + ||r: - r3
15

---

where h : IR RNX3 - R is the ith output of the permutation equivariant neural net- work, implemented via the Molecular orbital network (Moon) (Gao & Günnemann 2023a).
Ppar, Pami. Opar O'anti E RR are learnable scalars, and o; is the spin of the ith electron.
The orbitals ppr are defined as in Eq. (14) with Moon performing the following steps: We start with constructing electron embeddings based on electron-clectron distances and then proceed to aggregate these embeddings to the orbitals. The nuclei are updated through MLPs and finally diffused to the electrons, yielding the final electron embeddings.
The initial embedding h
of the ith electron is constructed as
N Σ
h (0)
gij W
-511) W
(30)
m(rs)
where denotes the Hadamard product and the Kronecker delta oz: as superscript indicates different parameters depending the identity between spin og and 0;-- r R + R. is learnable radial filter function, and is the activation function. Dif E R2 are the rescaled electron-electron
distances (von Glehn al. )023):
log (1 + 1/ FBll)
g
F - Fy 11 Fl1l
(31)
17 Fyll
is a normalization factor:
M
Zm
|
Rm 1|
=
exp
(32)
2
0 2
norm
172
We use the initial electron embeddings with nuclei embeddings and electron-nuclei distances to construct pairwise nuclei-electron embeddings representing edges in a fully connected graph:
e-n km = σ (h. (U) + 2m + 9.m Wm)
h
(33)
where 2. is the mith nucleus embedding, 9 E R are the rescaled electron-nuclei distances like in Eq. (3T). These embeddings are then aggregated with spatial filters twice: once towards the nuclei and once towards the electrons:
h (1))
Σ SEA"
h
I Im R.m)
(34)
(((mm))
M
m
p(r:)
hen
im
o I (Fr - Rmm),
(35)
m=1
h a (m, (1)
W + b).
(36)
We update the nuclei embeddings with L update layers:
h nxx(++1) (l))(()) + ((hh)())) hn (0)) + b()),
(37)
where denotes the opposite spin a, to obtain the final nuclei embeddings mL()), The final electron embeddings (L() are constructed by combining the message from the nuclei and the previous electron embedding:
h e(L) -a (n ( + (L) + bi) W + b2) + h2
(38)
where m, is the message from the nuclei to the ith electron:
M
m (L)
Σ σ ([ ([nmas(4)
(L)) W + Fiill (T:s -Rm)
(39)
(T.) m=
16

---

The spatial filters I' are defined as: A (z) = (x)
(40)
(z) exp
a Smi
2
D
W (zwi1)) + 6.. W + bl2))
(41)
i=1
Note that 3 is shared across all instances of I' is defined analogously to I but with fixed learnable parameters instead of MetaGNN parametrized ones.
D.2 MetaGNN
The MetaGNN M (IR. x N. θ takes the nucleus position R and charges Z as input and outputs parameters of the electronic wave function to adapt the solution to the system of interest. We follow Gao & Günnemann (2022 2023a) and implement it graph neural network (GNN) where nuclei are represented as nodes and edges are constructed based on inter-particke distances. The charge of the nucleus determines the initial node embeddings:
h Ez,
(42)
where E is an embedding matrix and Z; is the charge of the ith nucleus. These embeddings are iteratively updated via message passing in the following way:
k
[1+1]
f()) (k.
(43)
M
Σ
(0)
( R2),
(44)
fi ==1
o - 811 o o
= 1+ Σ
exp
(45)
BEN
where Eq. (43) describes the update function, Eq. 44 the message construction, and Eq. 45 learnable normalization coefficient. We implement the functions f and g via Gated Linear Units (GLU) (Shazeer 2020). As spatial filters, use the same in the wave function but additionally multiply the filters with radial Bessel functions from Gasteiger et al. (2019)
( (x) =3xx) W
(46)
2 sin
f:2
[ S2
Wan (a(ww()+bb ( b(2)
3(x)
exp
x
(47)
where f. are learnable frequencies, and cs smooth cutoff for the Bessel functions.
After L layers, we take the final node embeddings, pass them through another GLU, and then use a different GLU head for each distinct parameter tensor of the wave function want to predict. For edge-dependent parameters, like π or A, we first construct edge embeddings by concatenating all combinations of node embeddings. We pass these through GLU and then proceed like for node embeddings. For all outputs, we add default charge-dependent parameter tensor such that the MetaGNN only learns delta an initial guess depending on the charge of the nucleus.
D.3
Orbital parametrization
Our Pfaffian wave function enables to simply parametrize N. 2 MaxxNN, N2 orbitals rather than parametrizing exactly N As discussed in Sec. 4.4 we accomplish this by associating fixed number of orbitals with each nucleus. Here, provide detailed construction for all parameters of the orbital construction. For simplicity, we do not explicitly show the dependence on the kth Pfaffian. Note that we simply extend the readout by an N₂ sized dimension for each of the N Pfaffians from Eq. (3). Further, predict two sets of parameters, one for der and one for der in
17

---

Eq. 9). To parametrize the orbitals, we predict N orrh/nue orbital parameters for each of the N₂ nuclei. Concretely, the linear projection to WE from Eq. (14 are constructed as
W1 (k1)
W /ou (ki) W1 (kiz) E NRNNN
W
(48)
Worb//nue (KN.))
where wi RDD - R.N. learnable readouts of our MetaGNN. Similarly, we parametrize the envelope coefficients 0g from Eq. (6)
(k(k
SNanc/nuc (ki) (k2)
IRR
Newv
(49)
SNow /une (KN.))
where S RD - R are learnable readouts of our MetaGNN. The linear orbital weights connect orbitals. For this, we need to find a mapping
each nuclei-centered envelope to the non-atom-centert from each of the Nenv envelopes to each of the No orbitals. Since Nenv N Nawy /oue Nu and No = Norb/muc x Nn are predicted per nuclei, natural connection established via pair-wise atom function:
10,1 (k1, k1)
W L. Norh//nne nnn (k1,, 11)
11,1 (k22, k1)
N Naw / (k1, k())
New mueer Naneeraau /arb/auc (k1, k1) NoNo moe (k1, k2)
Neaw/mmend 1 (R22, ki)) 11,1 k2, k2)
E R.Ne
11,1 k2)
(50)
where Waj IRD RDD R learnable readouts of our MetaGNN. Similarly, establish the orbital correlations A from Eq. (14 by connecting each of the No orbitals to each other:
011,1 (kr, I k1)
01, O1,,Nrb//na /nne (ke, k22
011,1 (k2, k1)
Apc
oN 11 (k1, k2) 01111 kn, kz)
No Noett//ee-N /uc (k1, K2) 01, (o (ki, k2)
aNorr//mge (K2 ki) 0111 (k2, k2))
N.N.N.
(51)
Apr 5 (Apr ATr))
(52)
where di P x I + R are learnable readouts of our MetaGNN and Eq. (22) enforcing the antisymmetry requirements on A.
D.4
Changes to the MetaGNN
We performed several optimizations on the MetaGNN from Gao & Günnemann (2023a) that primarily reduce the number parameters while keeping accuracy. In particular, we changed the following:
We replace all MLPs with gated linear units (GLU) (Saazeer 2020
We reduced the hidden dimension from 128 64.
18

---

Table 2: Hyperparameters used for the experiments.
Hyperparameter
Value
Structure batch size
full batch 4096
Total electron samples
Epochs
10000
Learning rate
10 3 (1+ t 10 10-4)-3
Pretraining
Optimizer
Lamb
MCMC steps
Basis
STO-6G
Subproblem steps
Subproblem optimizer
Prodigy 1.0
Subproblem a
Subproblem 3
10
Steps
60000
Learning rate
0.02 (1 + L 10 10-4)-1
Optimization
Optimizer
Spring
MCMC steps
20
Norm constraint
10-3
Damping
0.001
Momentum
0.99 times mean deviation from median
Energy clipping
Hidden dim
256
E-E int dim
32
Ansatz.
Layers
Activation
SiLU
Determinants/Pfafiian
16
Jastrow layers
Filter hidden dims
116, 8]
Norb/uuc (H, He)
2
Norb/muc (Li, Be)
Pfaffinn
Norb/uuc (B, C)
Norb Norb/mue (N, O)
8
Norb/muc (F, Ne)
Nenv /nne
MetaGNN
Embedding dim
64
Message dim
32
Layers
Activation
SiLU
Filter hidden dims
[32, 16)
We reduced the message dimension from to 32.
We use bessel basis functions (Gasteiger et al. 2019) on the radius for edge filters.
We remove the hand-crafted orbital locations and the associated network.
We added LayerNorm before every GLU.
Together, these changes reduce the number of parameters from 13M to 1M for the MetaGNN while outperforming Gao & Günnemann (2023a) as demonstrated in Sec.
E
Experimental setup
19

---

Table Compute time per experiment measured in Nvidia A GPU hours.
Experiment
Time (GPU hours)
Ionization & affinity
224
N2
116
N2 + Ethene
124
TinyMol small
78
TinyMol large
E.1 Hyperparameters
We list the default parameters used for the experiments Tab. 2 Most of them were taken directly from Gao & Günnemann (2023a). We may have used different parameters for the experiments in Sec. 5 explicitly stated so. We implement everything in JAX (Bradbury et al. 2018)) To compute the laplacian V use the forward laplacian algorithm (Li et al. 2024) implemented in the folx library (Gao et al. 2023).
E.2
Source code
We provide the source code publicly on GitHub !
E.3 Compute time
Tab. lists the compute times required for conducting our experiments measured in Nvidia A100 GPU hours. Depending on the experiment, we use between and 4 GPUs per experiment via data parallelism. We typically allocated 32GB system memory and CPU cores per experiment. In terms of the number of parameters, the Moon wave function is large as in Gao & Günnemann (2023a) at IM parameters, and the MetaGNN shrank from 13M parameters to just TM parameters.
E.4 Preconditioning
The Spring optimizer (Goldshlager et al. 2024) is natural gradient descent optimizer for electronic wave functions V with the following update rule
0¹ 0f no
(53)
-0 -VO' + -1u
(54)
where À is the damping factor, μ is the momentum, is the learning rate, and Ô is the zero-centered Jacobian:
N
--0
O2,
(55)
N
1=1
vi
de
0
(56)
log (zx)
86
Since Ō E NxP where N is the batch size and P the number of parameters, the update in Eq. can be efficiently computed using the Woodbury matrix identity, which after some simplifications yields
54
=Ō(00 AT) -c+ (c+ -ōs-11 1
(57)
Our early experiment found it necessary to center the jacobian Ô per molecule rather than once for all. In single-structure VMC, the centering eliminates the gradient of the wave function along the direction where the amplitude of the wave function increases for all inputs. This direction does not
20

---

-0.42
our
-0.44
TAQ
Energy per atom (Fh.)
0.46
Globe + Moon
Globe FermiNet
-0.48
Hartree Fock
0.50
ATQMC 7> 00
Training data
0.54
26
Number of hydrogen
Fig. 6: Energy per atom of hydrogen chains with different lengths. The energy is computed with single NeurPf trained on the hydrogen chains with 6 and 10 atoms.
affect energies. Thus, instead of restricting the gradient from increasing in magnitude for all samples, we constrain it to not increase in magnitude for each molecule separately N.ote that the latter implies the first but not vice versa. For muuli--truccture VMC, we compute O as
Ni O
Ô Ō=
(58)
2i=
Na O:
where NI,..., NM are the index limits between molecular structures. To stabilize computations, we performed preconditioning in float64.
F Extensivity on hydrogen chains
Gao & Günnemann 2023a) and Scherbela et al. (2024) analyzed the behavior of their wave functions on hydrogen chains to investigate the extensivity of their wave functions. They did so by training the generalized wave functions on set hydrogen chains with and 10 elements. Then, they evaluated the energy per atom on hydrogen chains with different lengths. We replicated their experiment and trained a single NeurPf on the hydrogen chains with and 10 atoms and evaluated the energy per atom on hydrogen chains of increasing lengths.
Fig. shows the energy per atom of hydrogen chains with different lengths for various methods, Globe+Moon and Globe=FermiNet from Gao & Günnemann (202a), Scherbela et al. (2024), Hartree- Fock (CBS), the AFQMC limit for an infinitely long chain (Motta al.) 7017), and NeurPf. It is apparent that NeurPf outperforms Globe+Moon and Globe+FermiNer significantly by achieving significantly lower energies outside of the training regime. Compared Scherbela et al. (2024). NeurPf generally performs better on longer chains, achieving errors below the Hartree-Fock baseline. However, we observe significantly higher errors in the shortest chains in NeurPf.
These results indicate that NeurPf is better at generalizing to longer chains than previous works despite including additional Hartree-Fock calculations like Scherbela al. (2024).
G Metal ionization energies
In addition to the results in Sec. [] where we train on all second-row elements and their ionization and affinity potentials, we here train single NeurPf on a set metals and their ionization energies. This demonstrates that Neural Pfaffians also scale to heavier 3rd and 4th row elements. Fig. 7 shows the ionization energy during training. apparent, that NeurPf can learn solution for all states simultaneously.
21

---

Na
Mg
Al
K
Ca
Ton. (Ba)
10-2
10-3
10
10a
10a
10
10
10a
10
109
10
10
10
10
105
10
109
Fig. Ionization energies of metal atoms. The ionization energies are computed with single NeurP trained on the neutral and ionized atoms. Reference energies are taken from Martin & Musgrove
1998).
Small
Large
-95.44
155.500
Energy (Fi.)
155.525
-95.45
155.550
-95.46
155.575
95.47
155.600
20
40
60
25
50
75
Time (h)
Time (h)
(FermiNet)
our (PsiFormer)
Globe
Fig. 8: Energy convergence function of time.
H TinyMol convergence in time
In Fig. 8 we show the runtime effect of choosing different embedding and antisymmetrizer. We test our default model, our (FermiNet), our (PsiFormer) and Globe + Moon on both TinyMol datasets. For any time budget, all variants of NeurPf converge to lower energies than Globe.
I Convergence ablation studies
Here, provide additional ablation studies to further investigate the performance of NeurPf and our efficient envelopes. In particular, we train four different models on the small TinyMol dataset: NeurPf, NeurPf with the envelopes from Spencer et al. (2020), NeurPf with the envelopes from Pfau lt al. 2024), and an AGP-based generalized wave function.
The total energy during training is shown in Fig. The left plot shows the convergence regarding the number of steps, and the right plot shows the convergence in terms of time. We observe that NeurPf convergence is consistently faster than the other methods in terms of the number of steps and time. One further sees the importance generalizing Eq. (9) via the Pfaffian as the AGP-based wave function does not converge the same accuracy NeurPf. The bottleneck envelopes from Pfau et al. (2024) do not only converge to worse energies but are also slower per step than our efficient envelopes from Sec. 4.2
Model ablation studies
22

---

2863.80
2863.85
Total energy (E6)
2863.90
2863.95
AGP
Full Env.
-2664.00
our
Bottleneck Euv.
Efficient Env.
2864.05
5000
10000
15000
20000 Step
25000 30000
60
Time (h)
Fig. 9: Ablation study on the small TinyMol dataset. The y-axis shows the sum of all energies in the dataset. The left plot shows the convergence in terms of the number of steps. The right plot shows the convergence in terms of time, our Full Env. shows NeurPf with the envelopes from Spencer al. (2020) and Bottleneck Env. uses the bottleneck envelopes from Plau et al. (2024)
Small molecules
Large molecules
10a
10a
E - (T) CBS (m.b,)
103
10
102
Pf(()AT)
Pf(DD 47)
10
CCSD(T)
-2
10
250
500
1k
2k
4k
8k
16k
32k
250
500
1k
2k
4k
8k
16k
32k
Training Step
Training Step
Fig. 10: MonyMol ablation with fixed and learnable antisymmetrizer.
Small molecules
Large molecules
E - (T) CBS (mB)) 4
10
103
10
10a
our (Moon) our (FermiNet) our (PsiFormer)
10a
10
CCSD(T)
-2
100
250
500 1k
2k
4k
8k
16k
32k
250
500
1k
2k
4k
8k
16k
32k
Training Step
Training Step
Fig. 11: Ablation study on the small MinyMol dataset with different embedding networks.
23

---

Table TinyMol energies compared CCSD(T) in m.n-
Small C₂H₂
Large
Method (Steps)
CNH
2
C₃H4
CN
CNOH
CO2
Globe (32k)
5.2
12.3
10.7
62.3
45.8
40.4
42.7
TAO (32k)
1.1
4.5
6.6
18.7
21.0
41.9
19.6
our (32k)
-3.7
0.1
-2.1
12.7
5.5
3.1
5.0
our (128k)
-4.2
-1.5
-3.7
1.4
-3.8
-6.9
-8.2
Small molecules
Large molecules
60
Our (32k steps)
(32k steps)
TAD (pretrained, 32k steps)
Four FoosD (mfa))
10
20
0
10
CNH
C2H1
COH2
C314
CN2H2
ONOH
CO2
Fig. 12: Boxplot the energy per molecule both TinyMol small and large datasets for NeurPI, TAO, and the pretrained TAO from Scherbela et al. (2024). Each boxplot contains results from 10 structures for the given molecule. The line indicates the mean, the box the interquartile range, and the whiskers the 1.5 times the interquartile range.
J.1 Learnable antisymmetrizer
We picked Pf(QAAT) as parametrization because it generalizes Slater determinants and many alternative parametrizations. For instance, by choosing A (-) !) and ② ( Φ2)
Pf((-AT7)
Pf Φ2 We investigate the impact of having A being fixed/learnable Fig. [0] The results suggest that having A being learnable is a significant factor in our Neural
Pfaffinn's accuracy.
J.2 Embedding network
Since NeurPf is not limited to Moon, we performed additional ablations with FermiNet (Pīau et al. 2020) and PsiFormer (von Glehn al. 2023) as the embedding. The results in Fig. 11 show Neural Pfaffians outperforming Globe and TAO with any of the three equivariant embedding models. Consistent with Gao Günnemann (203a)) Moon the best choice for generalized wave functions.
K TinyMol results
Here, we provide additional data analysis and error metrics for the TinyMol dataset. First, we show in Tab energy per molecule for the small and large TinyMol datasets for NeurPí, Globe, and TAO. To estimate the remaining error, we also train another NeurPf for 128k steps. The results show that NeurPf consistently outperforms TAO and Globe all molecules in both datasets.
Second, we show the error per molecule for both the small and large TinyMol datasets Fig. 12 plot all models after 32k steps of training. It is apparent that NeurPf consistently results in lower, i.e., better, energies than TAO on all molecules in both datasets. Even the pretrained TAO outperformed by NeurPf on all but four structures of C3H4 in the large TinyMol dataset.
24

---

Small molecules
Large molecules
E - (CCSD(() CBS (mEp) 4
10a
10
10
100
102
TAO
10
CCSD(T)
-2
100
250
500
1k
2k
4k
8k
16k
32k
250
500
1k
2k
4k
8k
16k
32k
Training Step
Training Step
Fig. 13: TinyMol results with pretraining on the training set.
Small
Large
10
ECCSD(T) (mE
joint training
-5
separate training
CCSD(T) CBS
-10
250
1k
4k
16k
64k
256k
1M
250
1k
4k
16k
64k
256k
1M
Total Steps
Total Steps
Fig. 14: Comparison of the energy per molecule on the TinyMol dataset for training jointly on all structures training model per structure.
L Pretraining on the TinyMol dataset
The TinyMol provides additional pretraining set of 360 structures (18 molecules, 20 structures each). Like Scherbela et al. (2024), we pretrain our model on the training set of the TinyMol dataset and then finetune the two test sets. Interestingly, we find the Spring optimizer be unstable when swapping molecules from step to step and, thus, use CG- (G-preconditioning like Gao & Günnemann (2023a) during pretraining. While yielding small benefit on the small molecules, difference to the Hlrtree-Foock pretrained model on the large molecules as shown in Fig. [3 On the small structures, the unpretrained NeurPf's energies are 5.7 m mEE lower. NeurPf also surpasses the pretrained TAO after just 8k steps. Compared to the pretrained TAO on the large structures, NeurP surpasses TAO after 16k steps and achieves 5.4 E lower energies after 32k steps.
find notable
M Joint vs separate training
To estimate the benefit of training a generalized wave function compared to training model per molecule, we compare the convergence of the total energy on the TinyMol dataset for both approaches depending on the total number of training steps. As training a separate model for each of the 70 Mony test molecules computationally beyond the scope of this work, we select on structure per molecules and train model for each of the molecules. We use the same NeurPf with MetaGNN for both approaches. The results are shown Fig. [4] We observe that for lower step numbers, quite beneficial to train generalized model. Though, this benefit vanishes for higher step numbers, and
25

---

2
2.8
3.0
3.1
3.4
5.0
N in molecule 2
4
3.0
2.9
3.1
3.4
5.1
8
3.1
3.1
3.0
3.5
5.3
16
3.4
3.4
3.5
3.7
5.6
32
5.0
5.1
5.3
5.6
6.9
3
2
4
16
32
N₂ in molecule
Fig. 15: Time per training step depending on the number of electrons in two molecules.
Forward
10
Pf(
(e)
100
time in ms Graident
10
100
Laplacian
10
100
20
40
60
80
100
Number of electrons (Ne)
Fig. 16: Time for the forward pass, gradient and Laplacian computation of determinant our Pfaffian implementation.
training a model per molecule yields lower energies. We attribute this to the fact that the generalized model has to learn a more complex representation that is not necessary for each molecule individually. Further, the per-molecule energy estimates are quite unstable due to the small shared batch size. Developments like SSceebela et al. (2023) may improve NeurPf training well.
N Training time by batch composition
Here, we benchmark the total time per step for two-molecule batch. We test all combinations of two molecules with N1. N22 2, 4, 8, 16, 32. While we find small runtime increase when processing small molecules jointly in Fig. [5] for larger systems, see the runtime per step converge to the geometric mean of the individual runtimes.
Pfaffian runtime
we benchmark our implementation for Pf(TATT) (inel. the matrix multiplications) against the standard operation of det & for 10 to 100 electrons. We implement the Pfaffian in JAX
In Fig. 16
26

---

while highly optimized CUDA kernels are available for the determinant. In summary, both share the same complexity ( but the Pfaffian is approximately 5 times slower.
P
Broader impact
Highly accurate quantum chemical calculations essential for understanding chemical reactions and materials properties. Our work contributes to this development by providing accurate neural network quantum Monte Carlo caleulations at broader scales thanks to generalized wave functions. While this may used to distill more accurate force fields or axchange-corrclatin societal impact of our work is primarily in the scientific domain due to the high computational cost of neural network VMC. To the best of our knowledge, our work does not promote any negative societal impact more than general theoretical chemistry research does.
functionals for DFT, the
27

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We claim the following in our abstract and introduction: (1) Neural Pfaffian
are applicable to any molecular system. As outlined in Section 4.4 we ensure this by
parametrizing the orbitals be always larger than the number of electrons. (2) Neural
Pfaffian can learn all second-row element systems' ground state, ionization, and electron affinity energies. We demonstrate this in our first experiment Section (3) Neural Pfaffian outperforms Globe on the nitrogen dimer. See the second experiment Section (4) We
outperform CCSD(T) CBS on the small structures in TinyMol and TAO by factors of 10 and
6 on small and large structures, respectively. See the third experiment in Section S
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
At the end of Section 4.4 we list the limitations of our work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limmtations" section in their paper.
The paper should point out any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
28

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
Our paper does not contain theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Section 4 gives the mathematical definition of our new contribution. Ap- pendix D details the exact model definitions. Appendix Eists all hyperparameters, and as we explain Appendix E.2 we provide the source code reviewers and publish publicly upon publication.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
29

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide the source code via OpenReview to the reviewers as mentioned in Appendix E.2 The code will be made publicly available upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurIPS code and data submission guidelines (https://nips... cc/
public/guider /wwideee/ooeeuumiiisseeeeeeeeeee Coweeuumisssooooooooo.o.oo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips.
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: At the experimental setups (Section 5). we list the original references for structures and energies. Hyperparameters and additional details are listed in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Docs the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[No]
Justification: As common deep learning-based quantum Monte Carlo literature, we do not repeat experiments for different seeds due to their computational cost, see Appendix E.3 and their generally low deviations across runs. We omit error bars due to numerical integration these are typically below the readability threshold so 0.1 mEn.
30

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (c.g., Normally distributed errors).
should be clear whether the eπror bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
show in tables
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
We list compute resources and time Appendix E.3
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpppp///eerrppp.c////lll//ttte
Answer: [Yes]
Justification:
Our work respects the NeurIPS Code of Ethics in every aspect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
31

---

Justification:
We discuss this in App. P
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), faimesss considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We strongly believe that there is no higher danger of misuse for our work than for traditional methods in computational chemistry, especially not at the scale where neural wave functions are applicable.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We list all sources for our molecular structures and reference energies the appropriate places in Section 5 Further, we cite other codes we base our implementation of in Appendix E.2 and
Guidelines:
The answer NA means that the paper does not existing assets.
32

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, papprssitphheeeocc thoode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pane re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Upon publication, we will publish our source code new asset publically under the MIT license. Before that, we provide an early version to the reviewers via OpenReview, see Appendix
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Our research does not involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
33

---

Justification:
Our research does not involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly berween institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
34


Reviews:
[{"review_id": "eAV0OxWy4r", "reviewer": "Reviewer_7TwG", "summary": "This paper proposes NeurPf, a novel approach that replaces the standard determinant structure with a Pfaffian-based structure that allows systems of varying sizes to be represented with a single neural wave function. The key idea of the new ansatz is that given a large enough skew-symmetric matrix $A$, we have $\\text{Pf}(BAB^\\top) = \\det(B) \\text{Pf}(A)$ for invertible matrix $B$, allowing anti-symmetry to be broadcasted from the neural orbitals to the final output. This makes it possible to set the number of output orbitals $N_o$ into a fixed value that is irrelevant to the system size $N_e$ (so long as $N_e \\leq N_o$). Several details on implementing the NeurPf, including architecture selection, envelops, and computation, are discussed. Further, the authors ran experiments on the second-row atoms and small molecular systems to verify the effect of the proposed method.", "questions": "This is irrelevant to my rating, but right now the number of dimension $N_o$ must be larger than the size of the maximum systems $N_e$. What do you think we should do as the system size $N_e$ goes up to 100, e.g. in Psiformer and LapNet paper?", "limitations": "Irrelevant.", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- I think the major contribution, i.e., using Pfaffian to overcome the size consistency issue for varying systems, is quite novel and compelling. It is not only a direct generalization of the Slater determinant (which means that it can be directly applied to most existing architectures), but it allows systems with varying sizes to be represented with one set of parameters.\n- The paper is well-written and organized, making it easy to follow and understand.\n- The experiments that the authors consider, while not large enough systems (will be discussed later), indeed demonstrate the potential of the proposed method. I especially like the joint training experiments on all atoms of the second row, which serve as solid evidence that the proposed NeurPf ansatz can be used for multiple systems.", "weaknesses": "I think the following weaknesses are important to be addressed for the sake of a strong paper:\n\n- First, the authors emphasize the generalization ability of NeurPf, but it looks like all experiments they conduct are a sense of joint training (which I agree is still good to pursue). That said, the authors demonstrate the potential of training one network for multiple models but have not shown that the trained model can be generalized to unseen (but relevant) systems (probably, I admit, with some fine-tuning required). I think it will be necessary for the authors to demonstrate the capacity of generalization since the advantage of NeurPf is exactly to represent multiple systems together. For instance, training on 7/8 of the second row systems and generalizing to the rest, training on the ionized systems of some of the atoms and generalizing to the ionized systems of the rest, etc.\n- To me, NeurPf is, instead of a separate algorithm, more of an ansatz modification that can be applied to existing ansatzes, e.g., Ferminet, PsiFormer, LapNet, Moon, and Globe. Applying the method to all those architectures empowers them to be trained jointly on varying systems, as well. Unfortunately, the results of this combination are lacking, and the comparison the authors presented is only between the proposed method applied by Gao et al. 2023a.  I think is will be important if the authors show that \n  1. If we apply NeurPf to Ferminet, Psiformer, LapNet, Gao et al. 2023a, and TAO, then the joint training performance is similar to separate training, while the efficiency is much better. \n  2. For Fig 5, the results of all existing methods other than TAO should also be presented.\n  3. Comparison between NeruPf on different architectures to demonstrate which one is / is not compatible with the proposed architecture.\n- Can the authors add a concrete analysis of the computation efficiency of NeurPf? Specifically, how is it compared to a fixed-size Slater determinant (if $N_e = N_o$)? In order to train all systems together, we have to use the largest size (which implicitly increases the computation cost of the smaller systems). How will this influence the overall efficiency? Some plots or tables (instead of a number in texts) are preferred.\n- The authors should apply NeurPf on metal atoms (whose structures are complex but similar, and whose ionized energies are important to compute) to see how well it goes. For instance, Training on Na, Mg, Al, K and Ca. These systems are within 20 electrons (smaller than C(NH)_2), and should be feasible to train. The intuition is that since metals share similar electron organization structures to some extent, the proposed method should be able to capture the similarity and hence outperform separate training.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their invaluable feedback and suggestions. We hope to address their concerns. Firstly, we would like to highlight the broad range of new experimental evidence we present in the general comment. The following details how the experiments relate to the reviewer's comments.\n\n**TinyMol baselines**\\\nIn addition to the results from [1], we also trained Globe (+ Moon) from [4] on the TinyMol datasets and plot the convergence in Figure 1 of the general response. While starting similarly to our NeurPf, Globe does not converge to the same energies but saturates at energy errors at around $9.4mE_h$ and $47.8mE_h$ on the small and large datasets, respectively.\n\nUnfortunately, we cannot provide FermiNet or PsiFormer energies for all structures as these require $\\approx$ 6k A100 GPU hours each.\n\n**Embedding**\\\nWe agree with the reviewer's suggestion to ablate the embedding method of our NeurPfs. To demonstrate that they also perform well with different embedding methods, we train them with FermiNet [2], PsiFormer [3], and Moon [4] on both TinyMol test sets. We omitted LapNet due to its similarity with PsiFormer but happily add it to the next version of the paper.\n\nThe results are depicted in Figure 2 of the general response. NeurPfs outperform Globe and TAO independently of the choice of embedding network. Though consistent with the results from [4], Moon performs best in generalized wave functions.\n\n**Transferability**\\\nWe agree that transferability is an exciting aspect of generalized neural wave functions but would also like to stress that it has not been the focus of this work as it typically cannot achieve chemical accuracy. Nonetheless, we are happy to present additional experiments. Like Scherbela et al. [1], we first train our NeurPf on the TinyMol training set and then transfer it to the unseen molecules in the test sets.\n\nThe results are depicted in Figure 4 of the general response. These suggest that even after pretraining, both methods still require significant fine-tuning to lower errors, but only NeurPf can reach chemical accuracy.\n\nIn addition to the TinyMol results, we would like to point the reviewer to the hydrogen chain experiment in Appendix E, where we extrapolate to larger hydrogen chains without finetuning.\n\n**Joint vs. separate training**\\\nWe compare separately optimized wave functions to our NeurPf trained on the 30 and 40 structures, respectively. Since training separate wave functions for all 70 molecules in TinyMol exceeds our computational resources ($\\approx$ 6k A100 hours), we picked one structure for each of the 7 molecules. We trained a separate NeurPf for each to estimate the convergence.\n\nThe results are shown in Figure 5 of the general response. At a fixed cost, our generalized wave function generally offers higher accuracy than separately optimizing wave functions. However, we also find that the additional degrees of freedom (higher ratio of parameters/molecule) and specialized optimization offer better final accuracies for separate optimization.\n\n**Computational efficiency**\\\nWe measure the compute time of the Pfaffian operation in Figure 8 of the general response. Our Pfaffian is five times slower than the determinant. This is primarily due to optimized CUDA kernels for the latter. Note that here, we only measure the Pfaffian and determinant, not the rest of the network.\n\nWe benchmark the effect of the batch composition on the time per training step in Figure 9 of the general response for different batches composed of two molecules. There, we see a small overhead for small structures, while for large $N_e$, the time per batch converges to the geometric mean of the individual structures.\n\nWhen training systems of different sizes, we optimize with various techniques. We work with flattened representations for the embedding network. For the Pfaffian operation (or determinant), we switch to sequential processing for each molecule in a batch (but batch different conformer). This also allows us to use different $N_o$ for each molecule. We want to highlight that this maintains a high level of parallelism thanks to the batch of electronic configurations per molecule.\n\nFor a comparison to APG, $\\det (\\Phi_\\uparrow\\Phi_\\downarrow^T)$, we would like to point the reviewer to our ablation study in Appendix F. AGPs are a special case of Pfaffians. There, the AGP wave function is faster and reaches 32k steps in 70h compared to our Pfaffian's 95h. However, the AGP cannot match the accuracy of our NeurPf.\n\n**Number of orbitals**\\\nWhen going to large systems, the number of orbitals must increase with the number of electrons. As described in Section 4.4 and further detailed in Appendix C.3, we accomplish this by predicting a set of orbitals per atom:\n>[...] we grow the number of orbitals $N_o$ with the system size by defining $N_\\text{orb/nuc}$ orbitals per nucleus, as depicted in Fig. 2.\n\nThus, a system with twice the number of atoms (assuming they are the same atoms) has twice the number of orbitals, while the generalized wave function still has the same number of parameters. The computational scaling of neural network wave functions (incl. FermiNet/PsiFormer/...) to hundreds of electrons remains an issue for future work. Still, NeurPf remains well-defined independent of the system size, thanks to the orbitals growing with system size.\n\n**Final remarks**\\\nAgain, we thank the reviewer for their detailed assessment and suggestions for improving our manuscript. We hope to have addressed their concerns and look forward to an engaging discussion period. We appreciate any further feedback or questions from the reviewer.\n\n\n[1] Scherbela et al. \"Towards a transferable fermionic neural wavefunction for molecules\"\\\n[2] Pfau et al. \"Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks\"\\\n[3] von Glehn et al. \"A Self-Attention Ansatz for Ab-initio Quantum Chemistry\"\\\n[4] Gao et al. \"Generalizing Neural Wave Functions\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MaeYXPjwqv", "reviewer": "Reviewer_K3cB", "summary": "In this paper, the authors propose using Pfaffians instead of Slater determinants to learn a generalized neural wave functions so that the permutation antisymmetry enforcement can be better addressed. Empirical study shows one single proposed model can generalize to various systems (second-row elements) with chemical accuracy. For the nitrogen potential surface prediction, the proposed model outperforms previous work Globe. The proposed model outperforms CCSD(T) and previous work TAO on the TinyMol dataset which contains hundreds of samples.", "questions": "- In Fig. 3, why don't you train the model as many steps as FermiNet did?\n- There are other works on generalized wave functions mentioned in the related-work section, such as PESNet. Why aren't they compared in the experiments?\n- If I understand correctly, for the large molecules in Fig. 5, none of the neural wave functions outperforms CCSD(T) baseline?", "limitations": "The authors provided a section describing the limitations of the current work.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- By replacing Slater determinants, the proposed method avoids discrete and manual orbital selections, and hence is overparameterized, fully learnable, and applicable to any molecular systems.\n- Techniques such as memory-efficient envelopes, pretraining by Hartree-Fock, and generalization, are investigated to improve the efficiency and application of the proposed method.", "weaknesses": "I am not capable of discovering any weaknesses beyond the ones listed in the Limitation section, or the limitation of the entire neural wave function domain.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive feedback on our manuscript and would like to take the opportunity to address the few concerns that were raised. Firstly, we would like to highlight the broad range of new experimental evidence in the general comment. The following details how the experiments relate to the reviewer's concerns.\n\n**Comparison to Globe**\\\nTo include more baselines in our empirical evaluation, we trained Globe [1] on both TinyMol datasets and plotted the convergence in Figure 1 of the general response. While Globe is initially close to our NeurPf, it converges to higher, i.e., worse energies.\n\n**Comparison to PESNet**\\\nWe want to stress that PESNet [2] can only perform generalization across different geometric configurations, while Neural Pfaffians tackle the more complex problem of generalization across arbitrary molecular compounds. Nonetheless, we are happy to add comparisons of PESNet on the N2 energy surface. We also added FermiNet [3] from [4] to cover a broader range of methods. However, it should be noted that Globe (Ethene) and our (Ethene) have been trained on a more challenging augmented dataset, while PESNet is only optimized on the N2 energy surface directly. For FermiNet, each structure is optimized independently.\n\nThe results in Figure 8 of the general response demonstrate NeurPf's high accuracy on energy surfaces.\n\n**Comparison to CCSD(T) CBS**\\\nIn Figure 5 of the manuscript, we replicated the setting from [5] and, thus, only trained for 32k steps. However, neural wave functions typically require between 100k and 200k steps to converge [3,6]. Therefore, we extend the training of our Neural Pfaffian to 128k steps and compute energies for the converged model. The results are displayed in Table 1 of the general response. There, our long-trained NeurPf significantly outperforms CCSD(T) CBS on 3 of the 4 large molecules while being within chemical accuracy ($\\leq 1.6mE_h$) on the last one.\n\n**Extended training on second-row elements**\\\nAs the reviewer suggested, we trained our Neural Pfaffian for 200k steps on the second-row elements in Figure 6 of the general response. These results strongly suggest that a single Pfaffian can learn the ionization potentials with higher accuracy with additional training.\n\n**Final remarks**\\\nWe hope to have answered the reviewer's questions and look forward to an engaging discussion. We appreciate any further feedback and questions from the reviewer.\n\n\n[1] Gao et al. \"Generalizing Neural Wave Functions\"\\\n[2] Gao et al. \"Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions\"\\\n[3] Pfau et al. \"Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks\"\\\n[4] Fu et al. \"Variance extrapolation method for neural-network variational Monte Carlo\"\\\n[5] Scherbela et al. \"Towards a transferable fermionic neural wavefunction for molecules\"\\\n[6] von Glehn et al. \"A Self-Attention Ansatz for Ab-initio Quantum Chemistry\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RldTSmRtXG", "reviewer": "Reviewer_3Wr9", "summary": "This paper proposes a new ansatz (Neural Phaffian) for parameterizing wave functions. The new ansatz improves the expressive power by making it possible to increase the number of orbitals. It is also beneficial to tasks like generalization between different systems. The effectiveness is demonstrated with plenty of experiments.", "questions": "1.\tIs it obvious that Neural Pfaffian is better than the sum of parameterized orbital determinants $\\sum_k |(\\phi_{i,k}(r_j;r_{-j}))_{i,j}|$ that has a similar number of parameters? By ‘better’, I mean both expressive power (i.e. the least energy that the model can attain with sufficiently long optimization time) and convergence (i.e. the energy achieved within a fixed time period/number of iterations).\n2.\tRegarding the ablation study on envelopes, it turns out that the full envelope(green) is less costly per iteration than the red and yellow curves. This is weird because the model with efficient envelope contains fewer computations. Furthermore, the red line reaches lower energy than the green line. This is also weird because the model/wave function with full envelope is richer in expressiveness and thus has the potential to attain a lower energy.\nPlease correct me if I misunderstood this part.\n3.\tCould the authors give a possible explanation on why Neural Pfaffian’s result gets significantly worse in the 2-hydrogen case? Intuitively this setting is easier than the case with a larger number of hydrogens.\n4.\tRegarding L157, when the paper addresses systems with odd numbers of electrons, the approach of concatenating an additional learnable vector appears unnatural. The structure of the wave functions would change significantly when a system gains or loses an electron.  \n\nOverall, I enjoy reading this work and vote for a strong accept.", "limitations": "Yes.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "-\tThe paper targets important tasks in physical sciences. It is laudable that the authors not only address ground-state energy calculation but also consider ionization energy and generalization among different systems, which are of more practical significance.\n-\tHandling anti-symmetry with Pfaffian is a very novel and clever idea! How to enforce anti-symmetry is one of the most important problems in this task, and for decades we have not been so far from original Slater determinants. This work provides rich insights and opens up great opportunities for future research.\n-\tThe authors present comprehensive empirical studies to demonstrate the advantages of their model.", "weaknesses": "-\tAs mentioned in L250-258, Neural Pfaffian is slower than a comparably-sized wave function with Slater determinants. It would be better if the authors could add training-convergence plots with time instead of iterations when comparing Neural Pfaffian with baseline methods. \n-\tThe metric of summing over all energies in a dataset, as in Fig 7, is quite weird. I do not think this metric makes much sense. Additionally, it is possible that some systems with large absolute energy values dominate the curve. It is not convincing enough to showcase the advantage in **most** systems in the dataset.\n-\tThere are several typos. I highly recommend the authors read the paper thoroughly again to fix all the typos. Just to list some of them:\n1.\tL23: It should be $ \\langle \\Psi | \\hat{H} | \\Psi \\rangle$.\n2.\tL120: ‘inferring’.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are delighted by the reviewer's positive feedback and want to address the remaining concerns. Firstly, we would like to highlight the broad range of new experimental evidence we present in the general comment. The following details how the experiments relate to the reviewer's concerns.\n\n**Time convergence plot**\\\nFor our new ablation studies with NeurPf (+ Moon/PsiFormer/FermiNet) and Globe (+ Moon), we added convergence plots regarding compute time in Figure 10 of the general response.\n\nOne can see that despite the additional computational overhead through the Pfaffian, our NeurPf is approximately as fast as Globe with the same embedding. This comes mainly from the fewer operations, as we do not require Globe's extra message-passing steps from atoms to orbitals.\n\n**Total energy**\\\nWe agree with the reviewer that the total energy of all elements in the training set is an imperfect measure. For the ablation study, we decided to plot the total energy as this represents the optimization objective and is less noisy than individual molecular energies. Further, all energies are within the same order of magnitude ($-78.5E_h$ to $-114.49E_h$).\n\nNonetheless, to better communicate the error per molecule, we added Table 1 in the general response to attribute the error on a per-molecule basis. Generally, models that perform well on one of the molecules also perform well on the others. We also would like to highlight the results in Figure 8 of the manuscript, where we break down the error per molecule with error bars indicating the different structures for finer details. Figure 8 shows that our Neural Pfaffian results in more consistent relative energies than TAO.\n\n**Pfaffian vs. determinant**\\\nGreat question; we will extend the Appendix with the following discussion to clarify this. While it is a simple result to show that a Pfaffian can generalize Slater determinants, i.e., a Pfaffian can represent every Slater determinant, it is non-obvious that Pfaffians are naturally better in convergence or expressiveness. Empirical evidence in classical QMC [1] finds little to no improvements in molecular systems. However, in non-molecular systems, Pfaffians had greatly improved accuracy [2].\n\nIn our new ablation study in Figure 3 of the general response, we replace the learnable $A$ in $\\text{Pf}(\\Phi A\\Phi^T)$ with a fixed one and investigate the impact on convergence. Here, we find that the parametrization is an essential factor in the accuracy of our Neural Pfaffians.\n\nIn summary, it is unclear whether Pfaffians are generally better suited for modeling molecular systems. However, as we demonstrate in this work, they can achieve identical accuracy and are well-suited for generalized wave functions.\n\n**Envelopes**\\\nWe appreciate the keen eye of the reviewer; the classical envelopes are indeed faster than our memory-efficient envelopes. While our envelopes and Pfau et al. [3] aim to reduce memory requirements, they require more operations. In particular, the full envelopes require $O(N_bN_dN_nN_e^2)$ operations. In contrast, our memory efficient envelopes require $O(N_bN_dN_nN_\\frac{\\text{env}}{\\text{atom}}N_e^2)$ (for $N_o=N_e$) operations where $N_b$ is the batch size, $N_d$ is the number of determinants, $N_n$ the number of nuclei, $N_e$ the number of electrons and $N_\\frac{\\text{env}}{\\text{atom}}$ is the number of envelopes per atom in our memory-efficient envelopes.\nOur envelopes primarily reduce the memory from $O(N_bN_dN_nN_e^2)$ for the full envelopes to $O(N_bN_dN_nN_\\frac{\\text{env}}{\\text{atom}}N_e)$ where $N_\\frac{\\text{env}}{\\text{atom}}\\ll N_e$.\n\nWe most likely attribute the empirical performance to the increased number of wave function parameters. While the $\\sigma$ tensor is reduced from $N_d \\times N_n \\times N_e$ to $N_d \\times N_n \\times N_\\frac{\\text{env}}{\\text{atom}}$, the $\\pi$ tensor is enlarged from $N_d \\times N_n \\times N_e$ to $N_d \\times N_n \\times N_\\frac{\\text{env}}{\\text{atom}} \\times N_e$. For instance, for $N_e=20, N_n=4, N_d=16, N_\\frac{\\text{env}}{\\text{atom}}=8$, we get the following parameter counts:\n||$\\sigma$|$\\pi$|Total|\n|-|-|-|-|\n|full|1600|1600|3200|\n|our|640|12800|13440|\n\nWe will update Appendix A to better reflect memory and compute requirements in the context of the full envelopes.\n\n**Hydrogen chain results**\\\nWe agree with the reviewer that the H2 case is the simplest of all structures. However, it is arguably the most distinct from the other structures as the chain has no \"middle\" elements. We hypothesize that due to this, it has the lowest accuracy as no fine-tuning has been performed in this experiment.\n\n**Odd numbers of electrons**\\\nWe are happy to present an alternative solution to deal with odd numbers of electrons in the general comment and would appreciate the reviewer's opinion on this. In short, instead of appending a learnable vector to the orbital matrix, we pad the orbitals $\\Phi$ in both dimensions with an identity block to obtain $\\hat{\\Phi}=\\begin{pmatrix}\\Phi&0\\\\\\\\0&1\\end{pmatrix}$. Additionally, we also pad the antisymmetrizer $A$ to $\\hat{A}=\\begin{pmatrix}A&1\\\\\\\\-1&0\\end{pmatrix}$ such that one obtains $\\text{Pf}(\\hat{\\Phi} \\hat{A}\\hat{\\Phi}^T)\\propto\\det\\Phi$ if $\\Phi$ is square.\n\nWe repeat the experiment with this new formulation on the second-row elements experiment in Figure 6 of the general comment. We find little difference. Going forward, we will adopt the new padding technique as it requires no additional parameters.\n\n**Final remarks**\\\nWe will make sure to correct any typos in our manuscript. We hope to have adequately addressed the reviewer's concerns and questions. We welcome any additional feedback from the reviewer and eagerly await their response.\n\n[1] Bajdich et al. \"Pfaffian pairing and backflow wavefunctions for electronic structure quantum Monte Carlo methods\"\\\n[2] Kim et al. \"Neural-network quantum states for ultra-cold Fermi gases\"\\\n[3] Pfau et al. \"Natural Quantum Monte Carlo Computation of Excited States\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FtjXx4BLLg", "reviewer": "Reviewer_F2V3", "summary": "In this paper, the authors propose a novel Pfafifian-based anti-symmetrization method to represent the generalized wavefunction in quantum chemistry. Unlike the traditional Slater determinant-based anti-symmetrization, the Pfaffian-based method offers greater flexibility in selecting the number of orbitals. This flexibility is crucial for generating a universal wavefunction representation for molecular systems with varying number of electrons. The authors also introduce a new pretraining scheme that addresses the rotational symmetry in Hartree-Fock (HF) solutions, thereby improving the quality of the initial guess in their method. They validate the effectiveness of their approach through experiments on atomic systems and the TinyMol dataset.", "questions": "There are other ways to construct a skew-symmetric matrix, such as $\\Phi_1^{\\top}\\Phi_2-\\Phi_2^{\\top}\\Phi_1$, where $\\Phi_1,\\Phi_2\\in\\mathbb{R}^{N_{0}\\times N}$ are the output of different equivariant networks. The reviewer is curious about why the authors choose $\\Phi^{\\top}A\\Phi$ in Neural Pfaffians.", "limitations": "See weakness part", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1.\tTo the review's knowledge, this is the first study that uses Pfaffian as an anti-symmetrization method in the area of solving the Schrödinger equation with neural networks.\n2.\tThe proposed pretraining scheme effectively avoids orbital disorder in HF solutions, ensuring that the initial guess for the wavefunction is more accurate and stable. This scheme can be integrated with the other methods in solving the Schrödinger equation, facilitating further development and refinement in this area.\n3.\tThe performance of the proposed method is not affected by the uncorrelated data, indicating its potential to incorporate more training data to improve the transferability of the proposed method.", "weaknesses": "1.\tThe primary techniques in this paper are the Pfaffian anti-symmetrization method and the associated pretraining scheme. However, in the TinyMol dataset evaluation, the equivariant part of the proposed method differs from that of the baseline, making it difficult to attribute performance difference solely to the new anti-symmetrization method. Additionally, the performance of the proposed method is not consistently superior to the baseline. The reviewer recommends an ablation study to clearly demonstrate the effectiveness of the proposed method.\n2.\tTo handle odd numbers of electrons, the authors introduce a learnable padding vector to prevent Pfaffian collapse. While this approach appears effective in in-distribution experiments (e.g., affinity and ionization potential of atomic systems), the reviewer is concerned about its efficacy in out-of-distribution scenarios, particularly when generalizing to unseen systems with different electron numbers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their invaluable feedback and hope to address their concerns. Firstly, we would like to highlight the broad range of new experimental evidence we present in the general comment. The following details how the experiments relate to the reviewer's concerns.\n\n**Ablation studies**\\\nTo better isolate the contribution of our Neural Pfaffians, we performed several new ablation studies on the TinyMol dataset. In particular, we also trained Globe (+ Moon) [1] on the TinyMol dataset and combined our Neural Pfaffians with FermiNet [2] and PsiFormer [3].\n\nGlobe shares the same embedding technique as our Neural Pfaffian and, thus, only differs in how to enforce the fermionic antisymmetry. The results of Globe are depicted in Figure 1 of the general response. There, we can see that while Globe starts similar to our Neural Pfaffians, it cannot reach similar accuracies and converges to significantly higher, i.e., worse, energies.\n\nSince the Neural Pfaffian is not restricted to Moon as an embedding technique, we perform an ablation study where we replace Moon with FemriNet and PsiFormer, respectively. The convergence plots are depicted in Figure 2 of the general response. There, we can see that our Neural Pfaffian outperforms TAO and Globe, independent of the choice of embedding model. However, consistent with [1], Moon performs better in generalized wave functions.\n\nAdditionally, we would like to point the reviewer to the ablation study in Appendix F, where we replace the Pfaffians with AGPs $\\Psi=\\det(\\Phi_\\uparrow\\Phi_\\downarrow^T)$, a special case of the Pfaffian that is faster to evaluate. The rest of the network stays the same. There, we find that AGPs are significantly faster to compute, though they cannot match the accuracy of our Neural Pfaffians.\n\n**Comparison to CCSD(T)**\\\nOur NeurPf does not match the CCSD(T) energies in Figure 5 of the paper, as convergence typically requires 100k-200k steps [2,3]. We only trained for 32k steps to match the setup from [4]. To better illustrate final convergence, we added an evaluation to Table 1 of the general response, where we train a NeurPf for 128k steps. Our NeurPf comes within chemical accuracy ($\\leq 1.6mE_h$) or surpasses CCSD(T) even on the larger dataset.\n\n**Odd numbers of electrons**\\\nWe are happy to present an alternative solution to deal with odd numbers of electrons in the general comment and would appreciate the reviewer's opinion on this. In short, instead of appending a learnable vector to the orbital matrix, we pad the orbitals $\\Phi$ in both dimensions with an identity block to obtain $\\hat{\\Phi}=\\begin{pmatrix}\\Phi&0\\\\\\\\0&1\\end{pmatrix}$. Additionally, we also pad the antisymmetrizer $A$ to $\\hat{A}=\\begin{pmatrix}A&1\\\\\\\\-1&0\\end{pmatrix}$ such that one obtains $\\text{Pf}(\\hat{\\Phi} \\hat{A}\\hat{\\Phi}^T)\\propto\\det\\Phi$ if $\\Phi$ is square.\n\nWe repeat the experiment with this new formulation on the second-row elements experiment in Figure 6 of the general comment. We find little difference. Going forward, we will adopt the new padding technique as it requires no additional parameters.\n\n**Construction of skew-symmetric matrix**\\\nThis is a great point raised by the reviewer. We agree with the reviewer that there are various ways of parametrizing skew-symmetric matrices. We decided to go with $\\Phi A\\Phi^T$ as a general construction. For instance, $A=\\begin{pmatrix}0&I\\\\\\\\ -I&0\\end{pmatrix}$ and $\\Phi=(\\Phi_1 \\hspace{1em} \\Phi_2)$ corresponds to the reviewer's suggestion $\\Phi A\\Phi^T=\\Phi_1\\Phi_2^T - \\Phi_2\\Phi_1^T$. We experimentally verify the advantage of having $A$ being fully learnable in Figure 3 of the general response, where we compare our approach to a fixed non-learnable $A$.\n\nThe results indicate that having $A$ as learnable grants an accuracy benefit in the later stages of training. During the development of our method, we experimented with several other parametrizations, e.g., $A$ being learnable block-diagonal or other fixed forms like $A=\\text{diag}\\left(\\begin{pmatrix}0&I\\\\\\\\-I&0\\end{pmatrix}, ...\\right)$. Still, all of these resulted in very similar training trajectories to the one depicted in Figure 3 of the general response. We also experimented with parametrizing the skew-symmetric matrix for the Pfaffian directly via pair-orbitals $\\text{Pf}(A)$ with $A_{ij}=\\phi(h_i, h_j) - \\phi(h_j, h_i)$ but found this to be numerically unstable for molecular systems, especially with heavier atoms. To better communicate this, we will add this experiment among all the other new experimental evidence to the paper.\n\n**Final remarks**\\\nWe hope to have addressed the reviewer's concerns and were able to isolate our contribution better with our additional experimental evidence. We are happy to discuss further concerns and look forward to an engaging discussion.\n\n[1] Gao et al. \"Generalizing Neural Wave Functions\"\\\n[2] Pfau et al. \"Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks\"\\\n[3] von Glehn et al. \"A Self-Attention Ansatz for Ab-initio Quantum Chemistry\"\\\n[4] Scherbela et al. \"Towards a transferable fermionic neural wavefunction for molecules\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their invaluable feedback and great suggestions for additional experimental evaluation. We enriched our work with several ablation studies, which we present in the attached PDF. We will add all results to the manuscript.

**Fig 1: TinyMol baselines**\
In addition to the TAO results, we trained Globe [1] on the TinyMol datasets and added it to our evaluation. While Globe is initially close to our NeurPf, it convergences slower and to significantly higher, i.e., worse, energies.

**Fig 2: Embedding**\
Since NeurPf is not limited to Moon, we performed additional ablations with FermiNet [2] and PsiFormer [3] as the embedding.

Our Neural Pfaffians outperform Globe and TAO with any of the three equivariant embedding models. Consistent with [1], Moon is the best choice for generalized wave functions.

**Fig 3: Skew-symmetric construction**\
We picked $\text{Pf}(\Phi A \Phi^T)$ as parametrization because it generalizes Slater determinants and many alternative parametrizations. For instance, by choosing $A=\begin{pmatrix}0 & I\\\\ -I&0\end{pmatrix}$ and $\Phi=(\Phi_1 \hspace{.5em} \Phi_2)$, one obtains the parametrization suggested by Reviewer F2V3 $\text{Pf}(\Phi A \Phi^T)=\text{Pf}(\Phi_1\Phi_2^T - \Phi_2\Phi_1^T)$.
We investigate the impact of having $A$ being fixed/learnable in Figure 3.

The results suggest that having $A$ being learnable is a significant factor in our Neural Pfaffian's accuracy.

**Fig 4: Transferability**\
We want to stress that we focus on direct optimization in this work as it currently provides the only path toward chemical accuracy in generalized wave functions.
Nonetheless, we replicated the setup of TinyMol and pretrain our NeurPf on the TinyMol training set before finetuning on the test sets.
The results show that any method requires significant finetuning. However, only our Neural Pfaffians can match the reference calculations.

**Fig 5: Joint vs separate training**\
We compare separately optimized wave functions to our Neural Pfaffian trained on the 30 and 40 structures, respectively. We plot the total number of steps on the x-axis and the mean difference to CCSD(T) CBS on the y-axis. Since training separate wave functions for all 70 molecules in TinyMol exceeds our computational resources, we picked one structure for each of the 7 molecules. We trained a separate Neural Pfaffian (with Moon) for these 7 to estimate the errors.

At a fixed cost, our generalized wave function generally offers higher accuracy than separately optimizing wave functions. However, we also find that the additional degrees of freedom (higher ratio of parameters/molecule) and specialized optimization offer better final accuracies for separate optimization.

**Fig. 6: Odd numbers of electrons**\
We propose a new solution to address the reviewers' concerns regarding handling odd numbers of electrons. Starting from the classical Slater determinant where $\Phi$ is square and $\Psi=\det\Phi$:

Let $\Phi\in R^{N\times N}$ be the orbitals for odd $N$ electrons and $A\in R^{N\times N},A=-A^T$.
For $\hat{\Phi}=\begin{pmatrix}\Phi&0\\\\0&1\end{pmatrix},\hat{A}=\begin{pmatrix}A&1\\\\-1&0\end{pmatrix},\text{Pf}(\hat{\Phi}\hat{A}\hat{\Phi}^T)\propto\det\Phi$.

In our Neural Pfaffians, we generalize this to $\Phi\in R^{N\times D},A\in R^{D\times D}, \hat{\Phi}\in R^{N+1\times D+1},\hat{A}\in R^{D+1\times D+1}$.

We train our new approach on the second-row elements and show the training energies in Figure 5. As suggested by Reviewer K3cB, we increased the number of training steps to 200k to match FermiNet. The results suggest little difference between the appending of a learnable vector and the new dimension augmentation. Given the avoidance of additional learnable parameters, we use the new parametrization as default.

**Fig. 7: N2 baselines**\
We added FermiNet results from [5] and PESNet [4] as reference energies.

**Fig 8: Pfaffian runtime**\
We benchmark our implementation for $\text{Pf}(\Phi A\Phi^T)$ (incl. the matrix multiplications) against the standard operation of $\det\Phi$ for 10 to 100 electrons. We implement the Pfaffian in JAX while highly optimized CUDA kernels are available for the determinant. In summary, both share the same complexity of $O(N^3)$, but the Pfaffian is approximately 5 times slower.

**Fig 9: Runtime by batch composition**\
Here, we benchmark the total time per step for a two-molecule batch. We test all combinations of two molecules with $N_e^1,N_e^2\in\{2,4,8,16,32\}$. While we find a small runtime increase when processing small molecules jointly, for larger systems, we see the runtime per step converge to the geometric mean of the individual runtimes.

**Fig 10: Convergence by time**\
For NeurPf with FermiNet, PsiFormer, and Moon in addition to Globe (+ Moon), we show convergence by the number of steps. For any time budget, all variants of NeurPf converge to lower energies than Globe.

**Tab. 1: TinyMol energies**\
We list energy differences to CCSD(T) after training for Globe, TAO, and our NeurPf for 32k steps to match the setup from [6]. However, since NN-wave functions typically require 100k-200k steps to converge [2,3], we add a NeurPf trained for 128k steps.

The results show that among generalized wave functions that are optimized on each of the sets, our Neural Pfaffians achieve the lowest energies in 32k steps. Once further converged, our neural Pfaffians also reach or surpass CCSD(T) CBS on the larger structures.

[1] Gao et al. "Generalizing Neural Wave Functions"\
[2] Pfau et al. "Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks"\
[3] von Glehn et al. "A Self-Attention Ansatz for Ab-initio Quantum Chemistry"\
[4] Gao et al. "Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions"\
[5] Fu et al. "Variance extrapolation method for neural-network variational Monte Carlo"\
[6] Scherbela et al. "Towards a transferable fermionic neural wavefunction for molecules"

Meta-review: The paper proposes a novel neural many-electron wavefunction ansatz based on the construct of Pfaffian to guarantee the required anti-symmetry. It seems to be the first working neural ansatz other than Slater-determinant-based neural ansatzes, and has the advantage of a unified representation of wavefunctions of different molecular systems (different number of electrons, atom species and positions, etc.). The paper empirically shows that Neural Pfaffians indeed achieve overall more accurate results than existing methods in the cross-system joint training scenario. A few detailed technical designs (e.g., a proper HF initialization) are also presented. The contribution of the paper is acknowledged by all the reviewers. Reviewers also raised a few concerning points, especially the transferrability to unseen molecules, real-time computational cost, and handling odd numbers of electrons, which have been addressed or turned out acceptable given resource limit and the status of the field. Given the potential of opening a new possibility in this field, I recommend accept for this submission.
Decision: Accept (oral)