From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: Previous authors have developed parameterized function models specialized to capture structure in estimation problems arising in quantum chemistry.  Two prominent examples of these, which can be viewed as neural network models, are called the Slater Ansatz and Jastrow Ansatz.  This paper provides a separation result on the expressive power of these models, describing a function that can be approximated much more efficiently using the Jastrow Ansatz than with the Slater Ansatz.  They illustrate this separation through simulation experiments.

The fundamental novelty of the analysis was assessed to be limited.  The authors did not do enough to motivate this research for an ICLR audience.

justification_for_why_not_higher_score: Please see the above.

justification_for_why_not_lower_score: N/A

From: ICLR.cc/2023/Conference/Paper4699/Reviewer_Gasa
title: Followup response
comment: Thank you to the authors for all of your reviewer responses and for updating the experiments. I increased my score from 3 --> 5.

From: ICLR.cc/2023/Conference/Paper4699/Reviewer_Cp2s
title: Thanks for the response.
comment: First, I would like to thank the authors for the response. As far as my score is concerned, I would like to keep it unchanged.

From: ICLR.cc/2023/Conference/Paper4699/Reviewer_PU8s
title: I Appreciate Authors' Response
comment: Their comment resolves my concern therefore I change my score to 6.

From: ICLR.cc/2023/Conference/Paper4699/Authors
title: Rebuttal
comment: Thank you for your review, we address questions and concerns below.

Thanks for the suggestion, we agree the Slater ansatz could use a bit more exposition when we introduce it.

We also feel the experiments as given before were somewhat unsatisfying.  As discussed in the response to reviewer PU8s, we have updated our experimental setup to now use the true hard function (up to a choice of scaling), which we feel should make the empirical results a more honest reflection of the theory.

On the point of the theoretical contribution and its appropriateness for ICLR, we would argue that the ICLR call for papers includes "theoretical issues in deep learning".  As discussed in the response for reviewer Cp2s, there are many new ansätze applied in ML for quantum chemistry, and this work represents a first step in understanding the expressive power of these models.

From: ICLR.cc/2023/Conference/Paper4699/Authors
title: Rebuttal
comment: Thank you for your review, we address your questions and comments below.

As to motivation, we are driven by the broad spectrum of new antisymmetric ansätze in the literature, including backflow-driven models like FermiNet[1] and PauliNet[2], the hidden fermion model[3], models based on Pfaffians[4], etc.  These models differ on several axes, in terms of time complexity, empirical expressiveness, and qualitative performance on different molecules.

Very little is known about provably differences in expressiveness between these models; we consider this work a first step towards understanding comparison of these models, beginning with Slater vs Jastrow as even this simpler case was previously unstudied.

To answer Q1: yes, the primary application of antisymmetric ansätze is in fermionic systems in quantum chemistry, of which the four papers cited are several examples of applying these models in ML for chemistry.

To answer Q2: our last open question (about learnability of our hard function) is difficult because even the broader question of learning any non-trivial antisymmetric function is unknown.  In particular, say learning the Vandermonde is not obvious.  Though there are results about learnability of some functions with 2-layer neural networks, a Slater determinant where each orbital is a 2 layer network is far more complicated structurally than a 2 layer network itself, so it's not yet clear what results can be transported to the antisymmetric setting.



[1] Spencer, James, et al. "Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks." Bulletin of the American Physical Society 65 (2020).

[2] Hermann, Jan, Zeno Schätzle, and Frank Noé. "Deep-neural-network solution of the electronic Schrödinger equation." Nature Chemistry 12.10 (2020): 891-897.

[3] Robledo Moreno, Javier, et al. "Fermionic wave functions from neural-network constrained hidden states." Proceedings of the National Academy of Sciences 119.32 (2022): e2122059119.

[4] Bajdich, M., et al. "Pfaffian pairing and backflow wavefunctions for electronic structure quantum Monte Carlo methods." Physical Review B 77.11 (2008): 115112.

From: ICLR.cc/2023/Conference/Paper4699/Authors
title: Rebuttal
comment: Thank you for your review, we address your concerns below.

You are correct that we mentioned the proof of Theorem 3.1 using specific activations but were not explicit in proving it.  We have rectified this with a more explicit discussion of this fact (essentially the proof amounts to taking the Jastrow terms and orbitals in the defintion of the hard function G and considering the somewhat artificial setting where the activations are exactly chosen equal to these functions).

We agree that using a surrogate hard function is somewhat unsatisfying in our experiments, and we have updated all experiments to use the true hard function (up to a choice of scaling), which we feel should make the empirical results a more honest reflection of the theory.

To your point about the norm of the Jastrow factor, we note that our Jastrow factor is normalized to have L_2 norm equal to one, but this does not automatically control the infinity norm, which indeed can reach exponentially large values.

The plot_2.png in the supplement was an unused result under the choice of N = 2 set elements.  We omitted this plot because as you are correct that at such a small set size the separation result is not clear: because the separation is exponential in N, this fact is not surprising and the plots for larger values of N are more important for evincing the separation.

Thank you for pointing out the typos!

From: ICLR.cc/2023/Conference/Paper4699/Reviewer_PU8s
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper studies the separation of two antisymmetric ansätze, i.e. Slater ansätze and Jastrow ansätze.
strength_and_weaknesses: The topic of this paper is interesting and the writing is mostly clear.

However I have concerns about the soundness of theory and experiments.

1.
After Theorem 3.1, authors claim that "For particular choices of the activation, the network only requires $poly(N)$ parameters." However, I cannot find a proof in the main paper and supplement.

2.
In the experiment, authors don't use the hard function they choose in theory because "the Jastrow term $
\prod_{i<j} \frac{1}{1-r^4 x_i^2 x_j^2}
$ can lead to very large function values". On the other hand, their theory guarantee that $\lVert G\rVert=1$, which indicates that function values should not be too large. Why there is such seemingly disagreement?

3.
In the experiment, two figures for $N=4$ and $N=6$ are shown. In the supplementary material, another picture called `plot_2.png` is also shown but not included in the main paper. This experiment shows small gap between Slater ansätze and Jastrow ansätze. Why there is only large gap in $N=4$ experiment?

Typos:
page 6: some choice of -> same choice of.
page 6: sign[i]ficance.
page 7: hidden weights -> hidden width.

clarity,_quality,_novelty_and_reproducibility: The writing is mostly clear.
The topic and theory appear novel to me.
summary_of_the_review: Separation of antisymmetric ansätze is an interesting topic. However, several claims in this paper is not fully justified.
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4699/Reviewer_Cp2s
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper studies quantitative separations between two families of antisymmetric functions: The Slater Ansatz which is a linear combination of Slater determinants and the more powerful Jastrow Ansatz where each Slater determinant is augmented by a symmetric prefactor (and hence remains antisymmetric). In this area of research, it is widely known that Jastrow ansatz is more efficient (in terms of expressivity) compared to Slater, yet no separation was known. The paper provides such a separation in terms of expressivity power.

strength_and_weaknesses: The main result of this paper is a “hard” antisymmetric function that has the following properties:

(a) the function cannot be approximated better than some constant with $o(\exp(N^2))$ many Slater determinants, where $N$ is the number of particles (i.e., dimension of the space). This means that its Slater ansatz is quite inefficient.

(b) the function can be written in the Jastrow ansatz using a single Slater determinant and a prefactor which is parametrized by neural networks of width, depth and weights of order either $O(N^N)$ with ReLU activations or $\mathrm{poly}(N)$ with other activations.

The paper is well-written and its contribution is clear. The hard instance's construction seems nice and relies on some known tools from representation theory. Essentially, the provided separation adds to other results related to architecture separations in terms of approximation power. Regarding technical novelty, the only contribution lies in the construction of the hard instance. However, it is not clear to me whether the construction and the technical novelty can place the paper higher than marginally above the acceptance threshold. It would be nice if the authors could further motivate the studied problem. 

In short, I believe that the paper is marginally above the acceptance threshold, yet I am willing to increase my score depending on authors' response. 

I would like to close with some questions:

Q1. Could the authors provide some ML applications of these antisymmetric ansatze? Do they only arise in quantum chemistry applications?

Q2. Could the authors further comment on the last open question they propose?
clarity,_quality,_novelty_and_reproducibility: The paper is clear and the contribution is well-presented. 
summary_of_the_review: The paper deals with the approximation power of some families of antisymmetric functions and provides a separation between two well-known families. I believe that the contribution is marginally beyond acceptance, since the separation by itself and the technical work in order to obtain it do not seem sufficient to give a higher score to the paper. 
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: Not applicable
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4699/Reviewer_Gasa
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This work studies a quantitative separation between two fundamental models of
anti-symmetric functions in quantum chemistry: The Slater ansatz and Jastrow
ansatz. The authors construct a target function (Equation 20) using a subtle
property of Schur polynomials to show that the Jastrow ansatz can approximate
this function with exponentially fewer variables than the Slater ansatz. This
gives the first explicit separation between the two ansatze. Finally, the
authors begin to illustrate this separation property empirically by fitting a
modified version of this target function with neural network-based relaxations
of the two ansatze.
strength_and_weaknesses: **Strengths:**
- This work constructs an antisymmetric function that provably shows a
  difference in the expressiveness of the Slater and Jastrow ansatze given a
  comparable number of variables.
- The techniques used for this result use clean and powerful tools from
  algebraic combinatorics.

**Weaknesses:**
- This is primarily a math/mathematical physics paper. There are connections to
  representation learning (i.e., representing the target function via neural
  network-based ansatze), but they seem limited in scope.

**Suggestions:**
- [page 2] suggestion: It might be slightly more direct to first give the
  mathematical definition of *Slater determinant*, and then describe what is
  happening in words. The determinant-based definition is very direct and easy
  to parse.
- [page 7] Given the nature of this paper, the experiments used to explore
  Theorem 3.1 are very under-developed. The main results are interesting and
  exciting, but this does not fully come through from the experiments.
clarity,_quality,_novelty_and_reproducibility: The paper is written clearly. The main theoretical contribution is indeed
novel, and the experiments appear to be reproducible.
summary_of_the_review: This paper makes a nice theoretical contribution about the difference in the
representation power of the Slater ansatz and Jastrow ansatz. A witness function
is constructed using advanced techniques from algebraic combinatorics, and some
experiments are included to illustrate this difference. Overall, this is a good
paper but it does not seem appropriate for ICLR as written.
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold