Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The locally-balanced approach of Zanella has recently proven to be useful for MCMC sampling in discrete state spaces. The article proposes two main innovations when compared to recent articles building on the approach of Zanella:
1. tuning of the "balancing function" $g(t) = t^{\alpha}$ and length-scale $\sigma$ of the kernel $K_\sigma$. The tuning of the parameters $(\alpha, \sigma)$ is done quite naturally by attempting the maximise the "Expected Squared Jumping Distance" during an adaptation phase.
2. take second order effect by choosing a global "preconditioning" matrix such that $f(y) \approx f(x) + \nabla f(x)^\top (y-x) + \frac12 \langle (y-x), W (y-x) \rangle$. The choice of the global matrix $W$ is done during an adaptation phase by minimising $\|W(y-x) - (\nabla f(y) - \nabla f(x))\|^2$, which is quite cheap computationally to implement.

Using a second order approximation of the log target leads to sampling from a Ising-like distribution. As has already been discussed in other article [1], this can be approached by using the standard Gaussian integral trick (i.e. Hubbard-Stratonovich transformation). In order to implement this approach, a diagonal matrix $D$ such that $W+D$ is positive definite needs to be chosen. The authors propose to choose $D$ so that $W+D$ as "isotropic" as possible, which is a sensible and standard strategy that appears to work well in this case.


### References
1. Rhodes B, Gutmann M. Enhanced gradient-based MCMC in discrete spaces. arXiv preprint arXiv:2208.00040. 2022 Jul 29.
strength_and_weaknesses: ## Strengths
The paper reads well and the proposed approached is very natural and convincing. Furthermore, the simulations are quite well executed.

## Weakness
I do not see any particular weakness.
1. Indeed, the proposed methods is extremely natural after one has read the recent paper [1] but the article seems (?) to have been written at the same time.
2. I am not really sure we need the discussion at the bottom of page 3 (eg. Equations 5/6/7) since this does not seem to add much to the text. But it certainly does not hurt.

### Very Minor comments
1. The authors seem to parametrize Gaussian distributions with mean and standard-deviation $N(\mu, \sigma)$ instead of mean and variance $N(\mu, \sigma^2)$, which is a bit awkward (is it common in some communities?).
2. Should it be $K_\sigma$ instead of $H_\sigma$ in Equation 5. Also, $k_\sigma$ is used of $K_\sigma$ on page 2.

### References
1. Rhodes B, Gutmann M. Enhanced gradient-based MCMC in discrete spaces. arXiv preprint arXiv:2208.00040. 2022 Jul 29.
clarity,_quality,_novelty_and_reproducibility: ## Clarity:
the article is clearly written and very easy to follow.

## Quality:
The proposed method is convincing and natural. It appears to empirically work extremely well.

## Novelty:
Relatively weak: the tuning part of $\alpha,\sigma$ is entirely straightforward. The tuning of $W$ is interesting. The use of the Gaussian trick is also very standard.

## Reproducibility:
Enough detail to reproduce everything.
summary_of_the_review: The article proposes an extension of the work of Zanella to take second order effect into account (i.e. preconditioning) when sampling from discrete distributions, which is very important in practical scenarios (i.e. preconditioning can lead to very large improvements in sampling efficiency). Even if the approach is relatively straightforward, it works well and the numerical simulations are convincing.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper provides a non-local extension of the locally balanced samplers for improved sampling efficiency. The authors start by providing an analysis of existing approaches to improve the sampling efficiency of locally balanced samplers. They then introduce the any-scale balanced sampler in a step by step manner. Numerical results showing the effectiveness of the proposed algorithm conclude the paper.
strength_and_weaknesses: The proposed algorithm is clearly explained, I personally appreciated the step by step manner in which the algorithm is presented. In terms of performances, the algorithm offers notable improvements over its locally balanced counterparts. Results are interpreted and the limitations of the algorithm are clearly stated.

The authors highlighted rather well themselves the limitations of the proposed algorithm. I would actually count this as a strength, not a weakness. Personally, I don't see any other weaknesses.
clarity,_quality,_novelty_and_reproducibility: The authors clearly state their goals, show that their proposal fulfils their claims and are not shy of highlighting the limitations of their proposal. The paper is in my opinion clear and of very good quality. The results are new, thus the paper ticks the novelty criterion. I believe that enough details are given in the paper and the supplementary material to also tick the reproducibility criterion.

There are some minor typos and errors:
- page 2, paragraph above section 2, typo neumerical
- page 4, paragraph above section 3.2, typo hyperparamter
- page 5, paragraph beneath eq. 15, typo should gives
- page 5, paragraph "A common approach ...", shouldn't j=2,...,d instead of j=1,...,d ?
- page 7, paragraph "In this experiment ...", last sentence, it is said that more results are given in Table 2 and Table 3, however, it should be mentioned that they are to be found in the appendix since on the next page we have Table 2 and Table 3 which don't contain results for the Ising model
summary_of_the_review: The paper proposes an any-scale balanced sampler, a non-local extension of locally balanced samplers. The proposed algorithm is clearly presented, numerical results that show the advantages of the proposed sampler are presented and interpreted. Limitations are clearly discussed. Good paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed an any-scale balanced proposal for discrete distribution. Motivated by recent findings that non-local proposals improve sampling efficient, the proposed method closes the gaps of the choice of weight function and the accuracy of first order gradient approximation. Comparison to several baseline method demonstrate the efficiency of the proposed method.
strength_and_weaknesses: Strength:

1. The paper is clear and well written.
2. non-local proposals are helpful for efficient sampling of discrete distributons and the paper introduces several techniques to deal with the associate problems.


Weaknesses:

1. The preconditioning (using $D$ instead of $\sigma^2 I$) seems to be a useful trick. The same trick can be applied to other baseline method (for example, DLP can also use anisotropic diagonal as well), additional comparison to these variants would better illustrate the merits of the second order quadratic approximation.

2. Lack of a comparison to the precondition version PAVG of Rhodes and Gutmann, 2022.
clarity,_quality,_novelty_and_reproducibility: The paper is clearly writtern. However, there is a lack of originality as the main idea seems to be explored in a previous paper by Rhodes and Gutmann, 2022. 
summary_of_the_review: The paper provides useful tricks for hyperparameter choice but lacks originality. Also, a very important baseline method that is highly related to the proposed method is missing (PVAG in Rhodes and Gutmann, 2022).
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper considers two improvements to a recent, locally-balanced sampler approach of Zanella used in the context of discrete state spaces. Precisely, the authors propose a natural to tune $\alpha$ in the balancing function $g(t) = t^{\alpha}$ and the variance $\sigma$ in the length-scale of the kernel $K$. They also propose a way to fit a global "preconditioning" matrix for a second-order term in the Taylor expansion of $f$, which is relatively cheap to compute and intended to aid in the case where the steps are large so that the first-order expansion is inaccurate. 

The reviewers all thought that the ideas of the paper were clean, easy to implement, and described well. The experiments were also encouraging and executed well. The main concern was the amount of overlap with a (presumably concurrent) work by Rhodes and Gutman, but the decision was made that due to the fact that the works were simultaneous, and comparison to the paper was provided, this is ok. 
justification_for_why_not_higher_score: The reviewers concluded that paper had interesting ideas, though neither the novelty of the algorithmic improvements, nor the experimental gains merited a recommendation higher than a poster. 
justification_for_why_not_lower_score: The ideas of the paper improve a line of work that is quite successful at designing discrete space samplers and deserves a place at the conference.
Decision: Accept: poster