Towards Antisymmetric Neural Ansatz Separation
Authors: Aaron Zweig
Joan Bruna
Venue: Submitted to ICLR 2023 (2023)
Keywords: antisymmetric
Slater
Jastrow
quantum chemistry
separation

Abstract: We study separations between two fundamental models (or \emph{Ansätze}) of antisymmetric functions, that is, functions $f$ of the form $f(x_{\sigma(1)}, \ldots, x_{\sigma(N)}) = \text{sign}(\sigma)f(x_1, \ldots, x_N)$, where $\sigma$ is any permutation. 
These arise in the context of quantum chemistry, and are the basic modeling tool for wavefunctions of Fermionic systems. 
Specifically, we consider two popular antisymmetric Ansätze: the Slater representation, which leverages the alternating structure of determinants, and the Jastrow ansatz, which augments Slater determinants with a product by an arbitrary symmetric function. We construct an antisymmetric function that can be more efficiently expressed in Jastrow form, yet provably cannot be approximated by Slater determinants unless there are exponentially (in $N^2$) many terms. This represents the first explicit quantitative separation between these two Ansätze.

Under review as a conference paper at ICLR 2023
TOWARDS ANTISYMMETRIC NEURAL ANSATZ SEPA- RATION
Anonymous authors Paper under double-blind review
ABSTRACT
We study separations between two fundamental models (or Ansätze) of anti- symmetric functions, that is, functions of the form ( (N)) sign(o)f(z1,.. ......... .N), where is any permutation. These arise in the con- text of quantum chemistry, and are the basic modeling tool for wavefunctions of Fermionic systems. Specifically, we consider two popular antisymmetric Ansätze: the Slater representation, which leverages the alternating structure of determi- nants, and the Jastrow ansatz, which augments Slater determinants with product by arbitrary symmetric function. We construct antisymmetric function that can be more efficiently expressed in Jastrow form, yet provably cannot be ap- proximated by Slater determinants unless there are exponentially (in N many terms. This represents the first explicit quantitative separation between these two Ansätze.
INTRODUCTION
Neural networks have proven very successful parametrizing non-linear approximation spaces in high-dimensions, thanks to the ability of neural architectures leverage the physical structure and symmetries the problem at hand, while preserving universal approximation. The successes cover many areas of engineering and computational science, from computer vision (Krizhevsky et al., 2017) to protein folding (Jumper et al. 202T).
In each case, modifying the architecture (e.g. by adding layers, adjusting the activation func- tion, etc.) has intricate effects the approximation, statistical and optimization errors. An im- portant aspect in this puzzle is first understand the approximation abilities of certain neural architecture against a class of target functions having certain assumed symmetry (LeCun et al., 1995 Cohen et al. 2018). For instance, symmetric functions that are permutation-invarian f(II.... for all Tı,... IN and all permutations o: [1 + 11, N
ie
(Z (x()))
can be universally approximated by several neural architectures, e.g DeepSets (Zaheer et 2017) or Set Transformers (Lee et al. 2019); their approximation properties (Zweig & Bruna 2022 thus
offer a first glimpse on their efficiency across different learning tasks.
In this work, we focus on quantum chemistry applications, namely characterizing ground states of mody-body quantum systems. These are driven by the fundamental Schröndinger equation, an eigenvalue problem of the form
-
where H is the Hamiltonian associated to particle system defined over product space Ω and V is the wavefumction. a complex-valued function Vi Q - C whose squared modulus (Z)..... xx)) describes the probability of encountering the system in the state (I1,..., ,N) e Q&N A particularly important object is to compute the ground state, associated with the smallest eigenvalue Н. On Fermionic systems, the wavefunction satisfies an additional property, derived from Pauli's exclusion principle: the wavefunction is antisymmetric, meaning that
(F sign(a) ((1,

---

Under review as a conference paper at ICLR 2023
The antisymmetric constraint is an uncommon one, and therefore demands unique architectures to enforce it. The quintessential antisymmetric function is a Slater determinant (Szabo & Ostlund, $2 + C, they define rank-one =N) == Ilis f3 (2ry). The
2012), that now briefly describe. Given functions fi...., tensor mapping fi @ ....& 6 JN &&N + C by (fi @ ...... @ Slater determinant is then the orthogonal projection of a tensor rank one function into antisymmetric space. In other words, the rank one tensor fi
fN is projected A(fi @ @ fN) NI >xeex(11) (-1)" @ .... ...& fa(N). In coordinates, this expression becomes
(1) f2(21)
[((N) f2(2N)
A(i 0 & & (N)( IN)
det
N!
(x1(
fN
which shows that is antisymmetric following the alternating property of the determinant.
The Slater Ansatz is then simply linear combination of several Slater determinants, of the form F(x) = EiSL A(f @.... @ fx), similarly as shallow (Euclidean) neural network formed as lin- ear combination of simple non-linear ridge functions. While this defines a universal approximation class for antisymmetric functions (as a direct consequence of Weierstrass universal approximation theorems for polynomials), the approximation rates will generally be cursed by the dimensional- ity of the imput space, as also the case when studying lower bounds for standard shallow neural networks Maiorov & Meir (998).
In the case of particles in l or C, it is classical that all antisymmetric functions can be written as a product of symmetric function with the Vandermonde (see Section 3). This setting is gener- ally considered much easier than settings with higher-dimenssonal particles, as this Vandermonde factorization no longer applies, though there are still ansätze that mimic this formulation (Han et al.) 2019b).
A more powerful variant is the Jastrow Ansatz, where each Slater determinant is 'augmented" with symmetric prefactor (Jastrow] 1955), ie G p A(fi @ ... @ fN) where p is permutation-invarian Clearly, G' is still antisymmetric, since the product of an antisymmetric function with a symmetric one is again antisymmetric, but grants more representational power. Other parametrisations building from Jastrow are popularly used in the literature, e.g. backflow (Feynman & Cohen 1956), which models particle interactions by composing the Slater determinant with permutation equivariant change of variables. Among practitioners, it is common knowledge that the Slater Ansatz is in- efficient, compared to Jastrow or other more advanced parameterizations. Yet, there is no proven separation evinced by particular hard antisymmetric function. We note that the Jastrow ansatz is strictly generalized by backflow (see Section 3), so separations between Slater and Jastrow would have immediate consequences for separations from the stronger architectures as well.
In this work, we are interested understanding quantitative differences in approximation power between these two classes. Specifically, wish find antisymmetric target functions G such that G can be efficiently approximated with the Jastrow ansatz, i.e. approximated to e error in the infinity norm with some modest dependence on the parameters N and e by a single Slater determinant with a single symmetric prefactor, yet no Slater representation can approximate G for reasonably small widths. This question mirrors the issue of depth separation in deep learning theory, where one seeks functions that exhibit separation between, for example, two layer and three layer networks (Eldan Shamir 2016). as well as recent separations between classes of symmetric representations (Zweig
& Bruna 2022).
Main Contribution: We prove the first explicit separation between the two ansätze, and construct an antisymmetric function G such that:
In some norm, G cannot be approximated better than constant error by the Slater ansatz, unless there are Ne) many Slater Determinants.
G can be written in the Jastrow ansatz with neural network widths bounded by poly(N) for specific activations, in NO(N) using complex ReL.U
2

---

Under review as a conference paper at ICLR 2023
2
RELATED WORK
2.1 MACHINE LEARNING FOR QUANTUM CHEMISTRY
Numerous works explore how use neural network parameterizations effectively solve Schrödinger's equation. These include works in first quantization, which try to parameterize the wavefunction v/ directly (Pfau et al. 2020; Hermann et al., 2020), and second quantization, where the wavefunction is restricted to an exponentially large but finit--dmennional Hilbert space, and then the problem is mapped to spin system (Carleo & Troyer 2017).
2.2 ANTISYMMETRIC ANSÄTZE
Numerous architectures enforce antisymmetry. In this work we focus primarily on the Slater ansatz and Jastrow ansatz, but others exist and used in practice, with associated guarantees of universal- ity (Han et al. 2019a). The backflow ansatz enables interaction between particles while preserving antisymmetry (Luo & Clark 2019). More recently, ansatz that introduces hidden additional fermions was introduced in Moreno et al. (2021).
2.3 ARCHITECTURE SEPARATIONS
A large body of work studies the difference in expressive power between different neural network ar- chitectures. These works frequently center on the representational gap between two-layer and three- layer networks (Eldan & Shamir 2016 Daniely 2017). Relatedly, several works have considered the representational power of different networks architectures constrained to be symmetric (Wagstaff
et al. 2019 2022
Zweig& & Bruna 2022).
The most closely related work to ours is Huang et al. (2021). which proves a non-constructive limit the representability of the backflow ansatz, but requires exact representation rather than approximation some norm. Conversely, Hutter (2020) demonstrates the universality of a single backflow ansatz, but requires highly discontinuous term that may not be efficiently representable with neural network.
PRELIMINARIES AND MAIN THEOREM
3.1 ANTISYMMETRIC ANSÄTZE
We consider N particles restricted to the complex unit circle. That is, x E Q with Ω = {= E C; |z| = 1). We denote the tensor product & where, for f.g Ω C, have f & g 5 c such that (f & (z, y) f())g(v). Given a permutation a E Sx. and æ e S2W, we denote by a. (I) (N())) S the natural group action.
Let A denote the antisymmetric projection, defined via:
A(o1 @ 2 )N)
Σ (-1)% @ ... (o(N)
N!
dESN
So up rescaling we can consider Slater determinants as terms of the form A(ó .... Óv). Each on is called an orbital. Intuitively, Slater determinant is the simplest way to write an antisymmetric function, inheriting the antisymmetry property from the determinant itself.
Thus the Slater determinant ansatz with terms can be written as:
1.
F
Σ A( E @ fk)
(3)
3

---

Under review as a conference paper at ICLR 2023
Similarly, the Jastrow ansatz (with only term)
(Jastrow 1955) can be written
G p
gN)
where p is symmetric function, namely p(a.x) p(x) for any and x. is immediate verify
that G' is antisymmetric. Finally, the Backflow ansatz (Feynman & Cohen 1956) (considering again
a single term) is deñneed
G(z)= A(ā ... (x)(()))))(
(5)
where Ω2 + 12 is an equivariant flow, satisfying ((0.x) a. (ax), and where general f2 might be higher-dimensional than Ω.
In partieular, verify that
Φ : Ω + (Cx2)
(6)
- +<=) = (1p (x): ..... (p' (x);
equivariant. Given a collection of N orbitals $1 ON - C. we verify that the Jastrow Ansatz G = p. A(i 8 ... @0 gN) can be written as G = A(ă @ ... EN t with Cj C x Ω - C defined as O (2,x) = 9₁ (x). Thus, the Jastrow ansatz can be recovered as a particular case of the more general backflow ansatz. Therefore, quantitative separations between Slater and Jastrow automatically imply the same rates for Backflow.
3.2 INNER PRODUCTS
To measure the distance between the Slater Determinant ansatz and the Jastrow ansatz, we need an appropriate norm.
For univariate functions f.g: S + C, define the inner product:
U.g) (2T) [ 22n
)g()0)(())))
For functions acting on N particles, F,G (S1) - C. the associated inner product is
1
I
(F,G): (2T) 10,2ma0
F(e G( F(e"))
(9)
Let us introduce the notation that for x E CN and E N ra I 2. of the Fourier basis may be written as (Ix x²) = Sag.
Then the orthogonality
3.3 THEOREM STATEMENT
With this, we may state our main result explicitly:
Theorem 3.1. Consider Slater ansatz with L terms:
F = Σ a...
(10)
parameterized by orbitals fh St - C, and Jastrow ansatz.
G p=p.a p=p-AAlp A(g) 8 ... gN)
(11)
parameterized by orbitals gn S1 C and symmetric Jastrow factor p: (81)N C.
4

---

Under review as a conference paper at ICLR 2023
(a) A (5,4, 1)
(b) X (3,2,2,2,1)
Figure 1: Example of Young diagram and conjugate partition
Let N 2 and > 6 Then there is a hard function with | such that G parameterized by neural networks with width, depth, and weights scaling in O Oppoy((N .)) that can approximate G:
|| (G-Gli G
(12)
but, for number of Slater determinants
L S E.
3
min F G||
(13)
10
Note the constraint ||G]| = 1 is just prevent vacuous solutions by scaling G' to be arbitrarily big. We will describe the exact network structure of G' in Section A.4
We also remark that NO(N) < N2, this is true separation. Furthermore, the construction only requires (N()) parameters due to the analysis of generic complex ReLU activation. For particular choices of the activation, the network only requires poly(N) parameters. Finally, the restriction of G to only have one determinant is artificial, here to demonstrate the nature of the separation. In practice, the Jastrow ansatz allows for multiple determinant terms in learning.
3.4 SCHUR POLYNOMIALS
To build up the difficult function G, we use several identities related to the symmetric Schur poly- nomials. First, introduce partitions they will be used to index Schur polynomials:
Definition 3.2. An integer partition À is non-increasing, finite sequence of positive integers À Z A2 2 ... 2 AR. The weight of the partition is given by [A] = Σ >; The length of a partition ((A) is the number of terms the sequence. We call a partition even if every > even.
Partitions can represented by their Young diagram, see Figure Furthermore, we will need the notion of conjugate partition: Definition 3.3. Given partition the conjugate partition is gotten by reflecting the Young diagram of X along the line =z. We call partition doubly even if X and X are both even.
First, introduce the Vandermonde written
V(x)
(I, x:.) iÇj
(14)
Then we denote the Schur polynomial indexed by partition À as:
xx = V(x)
((x) det
x
Ay N --
((A) S N
(15)
((A) 1())
Given two partitions and μa, the following fact follows easily from linearity of the determinant:
(sx V, V) N! 8xp=
(16)
We will in the sequel assume even, then we can cite the following formal identity of a particular Pfaffian:
5

---

Under review as a conference paper at ICLR 2023
Theorem 3.4 (Sundquist (996)
Theorem 5.2, Ishikawa et al. (2006) Corollary 4.2).
Σ
SA V = Pf
3; 3i
(17)
1- 1- x
À doubly even
.N. A.O1 0 ... .NN))
(18)
1 2/2}
i<j
where we set the Φ maps to be:
N/ (1
N N/2+1sisN
(x12,) (1)))=
(19)
PROOF SKETCH
We give here sketch of the tactic of the proof. If we simplify the problem to question of antisym- metric tensor products of vectors instead functions, then inapproximability Slater determinants amounts to finding a high-rank antisymmetric tensor, that cannot be approximated in the l norm by a small number rank-one tensors.
The usual trick for this problem is to flatten all the tensors and rewrite them as matrices, so the problem reduces to approximating high rank matrix by low rank one, which is solved by SVD. To make the SVD tractable, the high-rank matrix is typically taken to be diagonal.
In our setting, cannot simply choose tensor that will be diagonal after flattening, because the constraints of antisymmetry will enforce certain matrix elements to be equal. turns out we can focus on a particular subtensor, where it's possible to flatten diagonal matrix, while nevertheless keeping our hard function representable by the Jastrow ansatz.
Indeed, the hard function G can be written exactly in the Jastrow ansatz by the above identity:
G
C
Σ
M1A++ NIN
82 V CVN!
A(ol") @ & OW))
VN! doubly
r An7a3
izj
(20)
for some choice of o maps and C, with [r|
1.
Let us explain the signficance of restricting the support to doubly even Schur functions. Let ó (N N 2,...,1,00, Then by simply canceling the Vandermonde factor, for an appropriate partition X, we have:
sx(x) V(x) = det x2;
Furthermore, is doubly even, then + will take the form (2a + 1, 1, 2b + 2b ...) with alternating odd and even terms with the odd term above the subsequent even term. See Figure for an example.
The significance of this structure is that, by knowing only the odd values of + ö, the even values are determined. We will make essential use this property to flatten an antisymmetric tensor to a matrix with sets of odd indices rows, and sets of even indices columns. For this matrix, the above structure implies diagonality, and from there we can proceed with a usual proof low-rank approximation from SVD.

---

Under review as a conference paper at ICLR 2023
(a) (4,4,2,
(b) A + 6= 67,
Figure 2: À and À + for doubly even. Note that À + 8 (6 1, 1) (6.
EXPERIMENTS
We illustrate the nature of this exponential separation finite case, specifically by seeking to learn our hard function in the Slater ansatz and Jastrow ansatz with N = 4, particles. In particular, we try to learn the hard function with the two ansätze (rescaled with the C term this constant extremely small to give normalization in the L2 norm).
Antilsy mmetrrc Leam Erro
Anttlyy Leaminn Error
0.5
0.25
Slater oc justoow
020
0.20
بلل الملا ملللة اللاابل
0:5
0.
910
الل
Lلاe le whk
0.00
300000
0.00
1500c0
160000
120000
(a) All training iterations with N
(b) Last quarter of iterations with N
Antisymmetric Leam na Erro with NG
Antioy minerric Leaming Error with N
Slater 20
0.14
Slater 20
0.75
State-9D
Swiee-90
b0 ASE
0.125
hommatedMs MSE
0.300
8050
25000
50000
75000 200000
125000 150000 175000 200000
150000
160000
170000
180000
200000
200000
decations
(c) All training itcrations with =
(d) Last quarter of iterations with N
Figure 3: Training MSE for Slater ansatz of varying number of determinants and Jastrow ansatz of one determinant.
We consider the empirical approximation of mean squared training error in || where compare learning with the Slater ansatz and a large number of determinants, vs. the Jastrow ansatz with a single determinant. We parameterize each orbital with three layer neural network with hidden width 30, and activation the complex ReLU that maps a + bi + (eLU((a) + ReLU We parameterize the Jastrow term with the Relational Network (Santoro et al. 2017) and multiplication

---

Under review as a conference paper at ICLR 2023
pooling where all networks are three layers and all hidden widths are also 30. Learning rate is set to 0.0005 in all runs, for 200000 iterations of full batch gradient descent on 10000 samples drawn i.id. from the complex unit circle. We plot the normalized MSE, i.c. MSE divided by the error attained by the naive zero function.
The results are given in Figure Each Slater ansatz is labeled by the number of determinants it parameterized with. We observe that one Jastrow determinant suffices where a large number of Slater determinants fails to achieve low MSE, alluding the exponential nature of the separation as the number of particles increases.
DISCUSSION
6.1 PROOF LIMITATIONS
The proof technique relies on finding a symmetric function that supported exclusively on doubly Schur polynomials. This is established in the Pfaffian identity given in Theorem 3.4 This yields function that requires exponentially many Slater determinants but may be written exactly in Jastrow form.
However, the large magnitude of the Jastrow factor precludes efficient approximation in the infinity norm. This cannot be overcome by changing the value of approaches 1, the magnitude of support on high dimensional doubly even Schur polynomials increases while simultancously the magnitude of the Jastrow factor increases. So must choose sufficiently large in order to guarantee the induced matrices are effectively high-rank.
An alternative tactic would be control approximation in the L2 norm given by || calculating the L2 norm is most easily done after decomposing into the orthogonal basis of multino- mials, which is challenging when multiplying terms together in the Jastrow product.
||- However,
This proof also only uses I-dimensional particles to evince separation. A nearly identical proof could be employed for higher-dimensionall particles by only utilizing the first component, but there would be no dependence on the dimension d. Understanding the simultaneous dependence on N and d would therefore require new proof technique.
6.2 OPEN QUESTIONS
The main result this work represents a first step in understanding separations between relevant antisymmetric ansätze. We conclude with discussion of the natural open questions in this domain.
Stronger Separation It would be ideal to strengthen this bound to show polynomial efficiency in approximating with the Jastrow ansatz, or to alternatively prove this not possible.
Practical Wavefunctions Extending the analysis to consider wavefunctions that appear in more practical applications would be informative. For example, in the one-dimensional case, famously the eigenfunctions of the Sutherland model known (Langmann, 2005), and the representability of these functions in particular is an open question.
More Powerful Ansätze The separation we demonstrate in this work is between the two simplest ansätze. Demonstrating separations among the more expressive network architectures, for example the backflow or hidden fermion models discussed previously, would prove the merit of these more complicated and time-intensive methods.
8

---

Under review as a conference paper at ICLR 2023
Learnability separations Our current separation concerns exclusively the approximation proper- ties of the two parametric families of antisymmetric functions, and as such neglects any optimization question. It would be interesting to integrate the optimization aspect in the separation, similarly as (Safran & Lee 2022) for fully-connected networks.
REFERENCES
Martin Arjovsky, Amar Shah, and Yoshua Bengio. Unitary evolution recurrent neural networks. In International conference on machine learning, pp. 1120-1128, PMLR, 2016.
Andrei Caragea, Dae Gwan Lee, Johannes Maly, Götz Pfander, and Felix Voigtlaender. Quantitative approximation results for complex-valued neural networks. SIAM Journal Mathematics of Data Science, 4(2):553-580, 2022.
Giuseppe Carleo and Matthias Troyer. Solving the quantum many-body problem with artificial neural networks. Science, 355(6325):602 60066666666666 2017.
Taco S Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical cnns. arXiv preprint arXiv:1801. 2018.
Amit Daniely. Depth separation for neural networks. Conference on Learning Theory, pp. 690- 696. PMLR, 2017.
Ronen Eldan and Ohad Shamir. The power of depth for feedforward neural networks. In Conference on learning theory, pp. 907-90 1779900. PMLR, 2016.
RP Feynman and Michael Cohen. Energy spectrum of the excitations in liquid helium. Physical Review, 102(5):1189, 1956.
Jiequn Han, Yingzhou Li, Lin Lin, Jianfeng Lu, Jiefu Zhang, and Linfeng Zhang. Universal approx- imation of symmetric and anti-symmetric functions. arXiv preprini /9/1v:/9/,.01765, 2019a.
Jiequn Han, Linfeng Zhang, and E Weinan. Solving many-electron schrödinger equation using deep neural networks. Journal of Computational Physics, 399: 108929, 2019b.
Jan Hermann, Zeno Schätzle, and Frank Noé. Deep-neural- solution of the electronic schrödinger equation. Nature Chemistry, 12(10):991-897 2020.
Hang Huang, JM Landsberg, and Jianfeng Lu. Geometry of backflow transformation ansatz for quantum many-body fermionic wavefunctions. urXiv preprint 1rX11:2111.10314, 2021.
Mareus Hutter. On representing (anti) symmetric functions. arXiv preprint arXi::2007. 2020.
Masao Ishikawa, Soichi Okada, Hiroyuki Tagawa, and Jiang Zeng. Generalizations of cauchy's determinant and schur's 's pfaffian. Advances in Applied Mathematics, 36(3):251-287, 2006.
Robert Jastrow. Many-body problem with strong forces. Physical Review, 98(5):1479, 1955.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger,
Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-588) 2021.
Alex Krizhevsky, Ilya Sutskever. and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Communications of the ACM, 60(6):84-99 2017.
Edwin Langmann. A method derive explicit formulas for elliptic generalization of the jack polynomials. arXiv preprint math-ph/05 1015, 2005.
Yann LeCun, Yoshua Bengio, et al. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361(10):199)) 1995.
Juho Lee, Yoonho Lee, Jungtack Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-bused permutation-invant invariint neural networks. In Interna- fional Conference Machine Learning, pp. 3754-3753. PMLR, 2019.

---

Under review as a conference paper at ICLR 2023
Di Luo and Bryan K Clark. Backflow transformations via neural networks for quantum many- sany-body wave functions. Physical review letters, 122(22):226401, 2019.
Vitaly Maiorov and Ron S Meir. Approximation bounds for smooth functions in e (r/sup d/) by neural and mixture networks. IEEE Transactions on Neural Networks, 9(5):96-9778 1998.
Attila Maróti. On elementary lower bounds for the partition function. Integers: Electronic J. Comb. Number Theory, 3:A10, 2003.
Javier Robledo Moreno, Giuseppe Carlco, Antoine Georges, and James Stokes. Fermionic wave functions from neural-network constrained hidden states. arXiv preprint arXiv:2111.04220, 2021.
David Pfau, James S Spencer, Alexander GDG Matthews, and W Matthew Foulkes, Ab initio solution of the many-electron schrödinger equation with deep neural networks. Physical Review Research, 2(3):033429, 2020.
Itay Safran and Jason Lee. Optimization-be based separations for neural networks. In Conference on Learning Theory, pp. 3-64. PMLR, 2022.
Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. A simple neural network module for relational reasoning. Ad- vances neural information processing systems, 30, 2017.
Thomas Sundquist. Two variable pfaffian identities and symmetric functions. Journal of Algebraic Combinatorics, 5(2): 135-148, 1996.
Attila Szabo and Neil $ Ostlund. Modern quantum chemistry: introduction advanced electronic structure theory. Courier Corporation, 2012.
Edward Wagstaff, Fabian Fuchs, Martin Engelcke, Ingmar Posner, and Michael A Osborne. On the limitations of representing functions on sets. In International Conference on Machine Learning, pp. 6487-6494. PMLR, 2019.
Edward Wagstaff, Fabian B Fuchs, Martin Engelcke, Michael A Osborne, and Ingmar Posner. Uni- versal approximation of functions on sets. Journal of Machine Learning Research, 23(151)::-56, 2022.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander Smola. Deep sets. Advances in neural information processing systems, 30, 2017.
Aaron Zweig and Joan Bruna. Exponential separations in symmetric neural networks. arXiv preprint wwXiv:2206.01266 2022.
10


Reviews:
[{"review_id": "j_Ur5bzLsbS", "reviewer": "Reviewer_Gasa", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work studies a quantitative separation between two fundamental models of\nanti-symmetric functions in quantum chemistry: The Slater ansatz and Jastrow\nansatz. The authors construct a target function (Equation 20) using a subtle\nproperty of Schur polynomials to show that the Jastrow ansatz can approximate\nthis function with exponentially fewer variables than the Slater ansatz. This\ngives the first explicit separation between the two ansatze. Finally, the\nauthors begin to illustrate this separation property empirically by fitting a\nmodified version of this target function with neural network-based relaxations\nof the two ansatze.", "summary_of_the_review": "This paper makes a nice theoretical contribution about the difference in the\nrepresentation power of the Slater ansatz and Jastrow ansatz. A witness function\nis constructed using advanced techniques from algebraic combinatorics, and some\nexperiments are included to illustrate this difference. Overall, this is a good\npaper but it does not seem appropriate for ICLR as written.", "strength_and_weaknesses": "**Strengths:**\n- This work constructs an antisymmetric function that provably shows a\n  difference in the expressiveness of the Slater and Jastrow ansatze given a\n  comparable number of variables.\n- The techniques used for this result use clean and powerful tools from\n  algebraic combinatorics.\n\n**Weaknesses:**\n- This is primarily a math/mathematical physics paper. There are connections to\n  representation learning (i.e., representing the target function via neural\n  network-based ansatze), but they seem limited in scope.\n\n**Suggestions:**\n- [page 2] suggestion: It might be slightly more direct to first give the\n  mathematical definition of *Slater determinant*, and then describe what is\n  happening in words. The determinant-based definition is very direct and easy\n  to parse.\n- [page 7] Given the nature of this paper, the experiments used to explore\n  Theorem 3.1 are very under-developed. The main results are interesting and\n  exciting, but this does not fully come through from the experiments.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Q2lCY6Q7pV", "reviewer": "Reviewer_Cp2s", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies quantitative separations between two families of antisymmetric functions: The Slater Ansatz which is a linear combination of Slater determinants and the more powerful Jastrow Ansatz where each Slater determinant is augmented by a symmetric prefactor (and hence remains antisymmetric). In this area of research, it is widely known that Jastrow ansatz is more efficient (in terms of expressivity) compared to Slater, yet no separation was known. The paper provides such a separation in terms of expressivity power.\n", "summary_of_the_review": "The paper deals with the approximation power of some families of antisymmetric functions and provides a separation between two well-known families. I believe that the contribution is marginally beyond acceptance, since the separation by itself and the technical work in order to obtain it do not seem sufficient to give a higher score to the paper. ", "strength_and_weaknesses": "The main result of this paper is a “hard” antisymmetric function that has the following properties:\n\n(a) the function cannot be approximated better than some constant with $o(\\exp(N^2))$ many Slater determinants, where $N$ is the number of particles (i.e., dimension of the space). This means that its Slater ansatz is quite inefficient.\n\n(b) the function can be written in the Jastrow ansatz using a single Slater determinant and a prefactor which is parametrized by neural networks of width, depth and weights of order either $O(N^N)$ with ReLU activations or $\\mathrm{poly}(N)$ with other activations.\n\nThe paper is well-written and its contribution is clear. The hard instance's construction seems nice and relies on some known tools from representation theory. Essentially, the provided separation adds to other results related to architecture separations in terms of approximation power. Regarding technical novelty, the only contribution lies in the construction of the hard instance. However, it is not clear to me whether the construction and the technical novelty can place the paper higher than marginally above the acceptance threshold. It would be nice if the authors could further motivate the studied problem. \n\nIn short, I believe that the paper is marginally above the acceptance threshold, yet I am willing to increase my score depending on authors' response. \n\nI would like to close with some questions:\n\nQ1. Could the authors provide some ML applications of these antisymmetric ansatze? Do they only arise in quantum chemistry applications?\n\nQ2. Could the authors further comment on the last open question they propose?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "OeiF5BsB25", "reviewer": "Reviewer_PU8s", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the separation of two antisymmetric ansätze, i.e. Slater ansätze and Jastrow ansätze.", "summary_of_the_review": "Separation of antisymmetric ansätze is an interesting topic. However, several claims in this paper is not fully justified.", "strength_and_weaknesses": "The topic of this paper is interesting and the writing is mostly clear.\n\nHowever I have concerns about the soundness of theory and experiments.\n\n1.\nAfter Theorem 3.1, authors claim that \"For particular choices of the activation, the network only requires $poly(N)$ parameters.\" However, I cannot find a proof in the main paper and supplement.\n\n2.\nIn the experiment, authors don't use the hard function they choose in theory because \"the Jastrow term $\n\\prod_{i<j} \\frac{1}{1-r^4 x_i^2 x_j^2}\n$ can lead to very large function values\". On the other hand, their theory guarantee that $\\lVert G\\rVert=1$, which indicates that function values should not be too large. Why there is such seemingly disagreement?\n\n3.\nIn the experiment, two figures for $N=4$ and $N=6$ are shown. In the supplementary material, another picture called `plot_2.png` is also shown but not included in the main paper. This experiment shows small gap between Slater ansätze and Jastrow ansätze. Why there is only large gap in $N=4$ experiment?\n\nTypos:\npage 6: some choice of -> same choice of.\npage 6: sign[i]ficance.\npage 7: hidden weights -> hidden width.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject