Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper demonstrates that by leveraging symmetrization of variational states and employing Lenczos recursion, neural network quantum states can surpass the current state-of-the-art accuracy for representing the ground state of the J1-J2 model on the square lattice. The main finding is that instead of using more intricate NN anzatz (e.g., ConvNets, Graph NN, or Transformers), the paper proposes that using the much simpler RBM anzatz coupled with symmetrization and Lanczos steps is sufficient to represent complex wave-functions. With that method they show that they can obtain much better ground state energy estimation across the entire range of the J2/J1 parameter and specifically near the point of highest frustation (~ 0.5).
strengths_and_weaknesses: The paper is well written and offers a simple introduction to the subject matter that is a bit outside of the common domain of ML. The main strength of the paper are its impressive results, improving on the previous best results by a large factor on a problem that is considered exceptionally difficult. Moreover, it is noteworthy that these improvements are obtained by using relatively simple RBMs, as opposed to more intricate architectures (e.g., ConvNets or Transformers).

However, the presented methods are not novel and are based on previous works. Specifically, it appears that the symmetrization method is identical to those used by prior papers including with NN-based approaches [1,2], and to the extent of my knowledge it is a well-known widely-used technique in the field. Similarly, Lanczos iteration for improving the accuracy of a given ground-state approximation was suggested by other works and was specifically used in the past to improve results on the J1-J2 model [3]. It is worth noting that to the best of my knowledge this is the first use of this method with NN quantum states. While the authors do properly cite prior works, it is not sufficiently emphasized that they merely implement these ideas for NN quantum states -- something that should have been more clearly articulated in the introduction and abstract.

As a final note, while the paper clearly shows the superiority of their proposed method on the given task, there is not a lot of effort spent on attributing these gains to the various choices. Some abalation study (even if done only on the 6x6 case) could go a long way to clarify the contribution of each element (e.g., employing Lanczos on a non-symmetric RBM and the ConvNet model from [1], or using RBMs with a symmetric matrix to account for translation symmetry vs. brute-force symmetrization).

[1] - Choo et al., Two-dimensional frustrated J1−J2 model studied with neural network quantum states, PRB, 100:125124, Sep 2019.

[2] - Sharir et al., Deep Autoregressive Models for the Efficient Variational Simulation of Many-Body Quantum Systems, PRL 124:020503, Jan 2020.

[3] - Iqbal et al., Spin liquid nature in the Heisenberg J1-J2 triangular antiferromagnet, PRB 93:144411, Apr 2016.
questions: * On the topic of symmetries it would have been helpful for the authors to discuss other approaches to brute-force symmetrization, e.g., symmetric constructions of RBMs / ConvNets / Graph NN. It is especially interesting that applying brute-force symmetrization to RBMs worked better than the ConvNet used in [1] which follows translational symmetry by construction and uses the same brute-force symmetrization only for the C4 symmetries. It would be great if the authors could comment on why that might be the case given that both models follow the same symmetries and one (the ConvNet) is supposedly more expressive (prior to using Lanczos). Moreover, given that the set of methods advocated by the paper could be applied to any anzatz, have the authors considered using ConvNets or any other NN architecture to verify whether the improved results are due to the use of simpler RBMs, or comparing the use of brute-force symmetrization vs. inherently symmetric models (whether using symmetric RBMs, or symmetric ConvNets)?

* There is a slight mismatch between the usual meaning of invariance to symmetries and how you define your symmetrized anzatz. Specifically, one usually refers to symmetric invariant function as one where f(x) = f(Tx) for all T in some set of symmetric operators, but in this case it appears that this requirement is soften to be equal in amplitude but it does allow for a shift in phase, and so it is more akin to equivariant property. Could the authors please elaborate on this point to clarify what is the property they wish to enforce and why?

* "point group symmetries" might not be clear for all readers, and it would be great to be specific on what it means in this context (rotations + reflections).

* While not mandatory, given that it is a method defined by prior works, it would be helpful to include a derivation of equations 14-18 in the appendix to this work.
limitations: The authors have properly addressed the limitations of their approach, and the difficulty of scaling it to larger lattices. My only comment would be that the limitations paragraph at the end should also include the cost of symmetrization in the training step (so it should be N^3 not N^2), and that they should also repeat the cost of the Lencoz step. I would also add that it might be challenging to scale this method to larger NN, which might be necessary to solve certain cases with high accuracy.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes a symmetry-projected variational solution in the form of linear combinations of simple restricted Boltzmann machines, which allows one to explore states outside of the original variational manifold and then increase representation power. Also, an expansion in terms of Krylov states using a Lanczos recursion is used to further improve quantum state accuracy. Experiments is conducted on Heisenberg J1-J2 model on square lattice. 
strengths_and_weaknesses: **Strengths:**
- *clarity*: This paper is overall well written and easy to follow. The method and results are clearly presented.
- *significance*: The proposed neural network quantum state can obey the internal symmetries of the quantum model and the point group symmetries of the lattice. This could make the neural network being more effective to represent quantum state by satisfying internal constraints of quantum many-body problem. Also, Heisenberg J1-J2 model has the infamous sign problem due to frustration that cannot be handled well by traditional numerical methods. So applying neural network to this problem is of great importance to quantum physics.

**Weaknesses:**
- *originality*: Based on my background and the paper itself I cannot judge if the method of this work is novel, but apparently this work misses some relevant works, such as the following:
  - *Liang, Xiao, et al. "Solving frustrated quantum many-particle models with convolutional neural networks." Physical Review B 98.10 (2018): 104426.*
  - *A. Szabó and C. Castelnovo. Neural network wave functions and the sign problem. Physical Review Research, 2(3):033075, 2020.*
  - *Kochkov, Dmitrii, et al. "Learning ground states of quantum Hamiltonians with graph networks." arXiv preprint arXiv:2110.06390 (2021).*

   And authors don’t explain the difference between the proposed method and those works (mentioned in related works) that also try to restore symmetry. And how is the performance comparison with these methods also considering symmetry, not just one CNN method.


- *quality*: Authors primarily compare the results with a specific CNN method. Could authors explain why only compare with this specific deep learning method?. Also, experiments are only performed on square lattice. Since the compared CNN method has performed so good on square lattice, the proposed method can only show very small improvements. I think authors could show the advantage of the proposed method over other methods on some other difficult quantum systems, such as several different lattices where the ground state is harder to learn, etc.


questions: I was aware of another recent work on solving quantum many-body problem over various lattices [1]. Have authors tried the proposed methods on other types of lattices, such as triangular and honeycomb, etc? How is the performance of proposed neural network wave function on these lattices?

[1] Kochkov, Dmitrii, et al. "Learning ground states of quantum Hamiltonians with graph networks." arXiv preprint arXiv:2110.06390 (2021).

limitations: Limitation of scaling up to larger systems is well addressed. Could authors explain more on the generality of this method on different type of lattices or even random graphs?
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In the present paper, the authors introduce a method to model the wave function of quantum systems, in particular spin systems in the $J_1$-$J_2$ model. They use a restricted Boltzmann machine (RBM) and form linear combinations of them in order to incorporate certain symmetries of the investigated system. Furthermore, the authors introduce a method to determine the ground state utilizing the Lanczos recursion to iteratively improve the estimate.

They compare their method on spin lattices of various sizes to baseline methods such as Quantum Monte Carlo and a CNN based method on the basis of comparing the ground state energies and the spin structure factor obtained through the different procedures. Their method is competitive with or outperforms the baselines.
strengths_and_weaknesses: **Strengths**

In general, the paper is concise and well written, making it pleasant to read and understand. 

The authors utilized the physical knowledge about the symmetries of the system and incorporate it into the model, which is a common theme in applying machine learning models to problems in the natural sciences.

Different concepts from machine learning (RBMs), advanced statistics (Fubini-study metric), and numerics (Lanczos recursion) are combined to form a new algorithm capable to compute the ground state energy of a spin system, which is competitive or outperforms its baselines.

**Weaknesses**

I am not familiar with algorithms for estimating the ground state energy of quantum systems, so I cannot judge how big or small the improvement of this procedure over existing baselines is. However, the difference of the results is often very small, only manifesting in the fourth or fifth digit, so it is not clear to me whether this is actually significant.

The authors mention that the key for such algorithms is that they use a small amount of memory and compute. A naive implementation would use an exponential amount of memory, while this procedure has a controllable number of variables. However, they do not discuss in detail how much compute and memory competing procedures use. They only mention in the conclusion that their method is slightly more expensive than the CNN-based method while having a better accuracy, but it is unclear what this means. 

**Conclusion**

Given my criticism and my lack of knowledge in the field, I am unsure whether to accept or reject this article. For me, it is important to clarify the amount of memory and compute needed by different methods, putting their respective performances in perspective. I am willing to raise my score if this concern is addressed appropriately. 
questions: * How much memory and compute do other methods use?
* How do they scale with the system size?
limitations: The authors state that the computational cost of their method scales quadratically with the system size, which limits their ability to tackle larger systems.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper applies Lanczos step improvements over a shallow neural quantum state based on RBMs. 
Symmetries are also exploited successfully, thanks to projections in the relevant symmetry sectors. 
The authors report results that are highly competitive with the state of the art on a challenge benchmark (J1-J2 model in 2d). 
strengths_and_weaknesses: To my knowledge, this is the first application of the Lanczos-step style improvements to neural quantum states. This idea has been applied in the past to other variational states, and carries a cost that is exponential with the number of Lanczos steps. 

The technique reported here allows to improve significantly on previously reported "pure" neural quantum states results. 
The main limitation of the approach is the known lack of "size extensive" scaling of the Lanczos iterations, which make them ineffective for larger systems approaching in the thermodynamic limit. However, there could be cases of finite clusters where the improvement offered is still important. 

My main criticism is the lack of comparison with the current best-known result on the model (more in the questions), reported in Nomura and Imada, PHYSICAL REVIEW X 11, 031034 (2021).  


questions: 1. While the authors report on a comparison on the 6x6 model, it is crucial to understand what happens on the larger 10x10 model, if they want to claim new SOTA results on the J1-J2 model. The paper by Nomura and Imada in Table 2 reports the relevant energy to compare to. How does the Lanczos-step approach compare? 

2. The effect of symmetries seems absolutely crucial to improve the bare RBM results. What does it happen if one imposes translation symmetries in the weights instead of summing over the group explicitly? 
limitations: yes
review_title: null

Meta-review: metareview: This submission proposes to apply Lanczos step improvements over a shallow neural quantum state based on restricted Boltzmann machines (RBM). The authors report highly competitive empirical results with the state of the art on one challenge benchmark: the J1-J2 model.  The reviews are mixed for this contribution: while the provided empirical results look promising, a clear and comprehensive comparison with SOTA's results was lacking in the initial submission, which was later improved during the interaction between the authors and the reviewers. We think these suggested changes should be included in the revision.  Most of the reviewers see the novelty of using Lanczos step improvements and feel this technique could be generally applicable. However, the authors should also discuss the limitation of this technique, especially the size consistency of the approach and the fact that returns will be smaller and smaller on larger lattices, explicitly in the revision. 
confidence: Less certain
Decision: Accept