Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper analyzes a recently proposed non-equilibrium importance sampling approach to integrating / estimating the partition function of intractable densities. This approach provides an unbiased estimator with bounded variances typically much better than the variances provided by other estimators (which are in some cases infinite). Some experiments on basic multivariate densities including asymmetric and symmetric MoG, and Neal's funnel in 10d, provide empirical support.
strengths_and_weaknesses: Originality
The NEIS method seems to be new to the ML/stats community. The predecessor work in physics is only a couple years old. Connecting this work to Neural ODEs and neutral transport maps is a novel contribution, as are the variational approximations enabling these connections.

Quality
The math is developed reasonably well and connections to existing work are shown (might also think about connections to diffusion models, neural transport HMC, neural optimal transport). A possible complaint might be that the math is developed fairly quickly/compactly in the body of the paper, with >30pg of detail reserved to supplementaries (which I've only browsed).
A weaker portion of the paper, in my view, is the experiments. On the positive side, the experiments demonstrate fairly well the unbiased and low variance nature of this estimation approach. However, I would like to see some richer densities than [hierarchical/mixture] gaussians. Multiple libraries exist that could provide other interesting densities (inference gym, ppl bench, posterior db). One reason the authors might have chosen these particular densities is for analytically computable Z, which enables scaling all plots to a 0-1 size. But I do see a log-Gaussian cox process in the anon github, which suggests other densities might have been considered? The supplementary indicates that the method takes 1-2h to fit/optimize on these densities, which calls into question the fairness of comparison with 100 transitions of AIS (which should take only seconds unless I've misunderstood something). If we ran AIS for 1h, could we get much more exhaustive exploration and tighter variance? I might like to see plots of Z bias against number of density evaluations for each of the methods considered.

Clarity
I would have been helped by adding some prose about usage of $\nabla$ to the notation section. Presumably it is used throughout as an operator, but some notations like $\nabla \cdot$ (e.g. eq 5) are unfamiliar.

Significance
Because the approach seems to be fairly expensive and is only compared to much less expensive AIS, I am somewhat unclear about its practical significance. I do think the approach seems academically interesting to the NeurIPS audience and the numerical experiments indicate that, at least in cases where we are willing to pay compute, this provides an interesting new point on the frontier of approximate inference fidelity vs cost. I wonder whether comparisons with a trainable-RealNVP-reparameterized AIS/SMC might give a more fair comparison in terms of cost.
questions: Why were these particular densities chosen for numerical examples?

If we ran AIS for 1h, could we get much more exhaustive exploration and tighter variance?

How does the training approach compare/contrast with diffusion models, where we are training a map that pushes a perturbed sample back toward the original sample?

Is the bias on Neal's funnel attributable to the variational approximation, or some other approximation in the procedure?
limitations: The academic nature of the examples is noted. Perhaps the expense of training ought to be more explicitly called out.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper approaches the problem of normalization constant estimation for the case when the density of the target distribution is given as an unnormalized function. To be precise, the paper builds upon the Non-Equilibrium Importance Sampling (NEIS) [1]. Unlike the conventional Importance Sampling, NEIS takes the samples from the reference density (known normalized density) and propagates them along some vector field, which we are free to choose. One can obtain the resulting estimator of the normalization constant as the properly weighted average of the target density (unnormalized) along the sampled trajectories.

The main contribution of this paper is the following. The authors use the fact that NEIS yields an unbiased estimate for any choice of the vector field. Using this degree of freedom, they learn a parametric model of the vector field that minimizes the variance of the resulting estimator.

The main theoretical result of the paper is the achievability of the zero-variance estimate of the normalization constant. Namely, when we take the infinite time for the trajectory integration, there exists a vector field that yields a zero-variance NEIS estimator. Furthermore, the authors demonstrate that every local minimum for this problem is actually global. In practice, the authors consider only finite-time estimators, though.

For the empirical evaluation, the authors consider 2d, 10d mixtures of Gaussians, and 10d Funnel distribution. They demonstrate that the proposed method outperforms AIS and the original NEIS in a reasonable setting. Moreover, for the Funnel distribution, the authors parameterize the vector field as a linear model, significantly reducing the computational efforts.

1. Grant M. Rotskoff and Eric Vanden-Eijnden. Dynamical computation of the density of states and Bayes factors using nonequilibrium importance sampling. Phys. Rev. Lett., 122(15):150602, 2019. doi: 10.1103/PhysRevLett.122.150602.

strengths_and_weaknesses: Overall, the paper does a decent job on every single aspect of a NeurIPS publication. It's clearly presented. It focuses on the intuition of the method in the main body and exhaustively describes all the details in the appendix. The theory is novel and immediately results in a motivated application. One might say that the empirical evaluation is not extensive. However, it's neat (properly compared with relevant algorithms) and demonstrates the clear superiority of the approach.

I feel that the weakest point of the method is its computational burden. Namely, backpropagation through the trajectories of ODEs is computationally expensive. Also, the unbiased estimate is formulated as a continuous integral, which, in practice, has to be discretized. While the latter seems to be not an issue since the integral is 1d, I could imagine that the computational efforts spent on learning might overweight the performance gains during the evaluation. I think that a comparison of the methods with the fixed computational budget is essential here (I haven't found the corresponding numbers in Appendix I).
questions: In the section "Strengths and Weaknesses," I gave my suggestion for the improvements of the experimental section.

Regarding the approach, I have the following question for the authors. Assuming the network can generate arbitrary vector fields, does it make sense to change the time interval $[t_-, t_+]$?
limitations: As I've mentioned in "Strengths and Weaknesses,"  the computational side of the proposed method is not fully addressed, and I would expect a comparison of the methods with the fixed computational budget.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper considers non-equilibrium importance sampling (NEIS), wherein samples from a base distribution are evolved along flow lines and a particular quantity is averaged along these flow lines.  This results in a theoretically unbiased estimate of a normalizing constant (like in standard importance sampling), and since standard importance sampling can be derived as a special case of NEIS, this approach should allow one to construct lower variance estimators.  The authors derive a PDE which is satisfied by vector fields that produce optimal (zero variance) importance schemes, but then use a neural ODE approach to minimize the empirical variances of samples.  Finally, a few problems are considered where the NEIS method greatly reduces the variance of normalizing constant estimators compared to adaptive importance sampling.
strengths_and_weaknesses: Strengths:

* Overall I really enjoyed the paper.  The main results were technical, but clearly presented, and the approach seems to be promising.

* The theoretical results in the appendix were very thorough and I appreciated the examples of explicitly solvable dynamics and other concrete examples.

Weaknesses:

* A major weakness of the present paper, in my opinion, is the general lack of discussion of the impact of discretizing the flow along the vector field.  All of the theoretical results are about the idealized flow, but in practice the ODEs have to be solved numerically.  Presumably this introduces complicated biases and if the vector field is learned using a neural net, there is no obvious reason that the learned ODE needs to be amenable to easy and accurate numerical integration.  More discussion of and investigation of the impact of discretization would strengthen the paper, in my opinion.

* A minor point -- 3 different methods of using the neural ODE framing (directly estimating an arbitrary vector field, estimating a scalar potential and then taking gradients, or the divergence-free form) but then results were only presented from one of these for each toy problem (other than training curves in Figure 9).

* The training of the neural ODE seems difficult -- the two trials seem to rarely converge to similar optima in Figure 9, and the runtime of the training seems quite long -- in the hours that it takes to train the neural ODE to construct the vector field, many more samples could be drawn from the proposal distribution and used in a less fancy importance sampling scheme.  It would be good to consider comparisons of the various estimators in terms of accuracy per fixed amount of wall time (e.g., how does AIS or vanilla IS do if it just samples continuously for the amount of time it takes to train the NEIS vector field?).

* One other aspect that I felt was neglected was the importance of the proposal distribution, $\rho_0$.  In standard importance sampling, how to choose $\rho_0$ (from some family) is well-understood -- we want it to be as close to $\rho_1$ as possible, where closeness is measured by the chi divergence.  In the NEIS case, the optimal $\rho_0$ is less obvious -- presumably choosing it close to $\rho_1$ is useful, but one also wants the vector field to be "nice" and easy to learn.

Minor comments and Typos:

* Line 16: "order" --> "orders"
* Line 45: "based on Jarzynski" --> "based on the Jarzynski"
* Line 75: "order" --> "orders
* Line 77: "lead into" --> "lead to"
* Line 83: "one use in" --> "one used in"
* Line 168: "sampling procedure" --> "a sampling procedure"
* Line 188: "will necessarily produces" --> "will necessarily produce"
* Line 211: I believe the plural of "anstaz" is "anstazes"
* Line 227: "minimize" --> "minimizing"
* Figure 2b caption: "descend" --> "descent"
* Line 289: "descend" --> "descent"
* The notation $|\cdot|$ is used throughout.  I believe that this is the $\ell_2$ norm, but that should be made explicit.
* It should be made explicit that the minimization of equation (15) is over $V$ 
questions: * How were the flow fields chosen for the direct GD method for Figures 2, 3 and 4?  in Figure 2 it is $-\nabla U_1$, in Figure 3 it is $-5\nabla U_1$, in Figure 4 it is $-2\nabla U_1$.
* Why was a neural ODE not used in the funnel toy model?  In general one does not know a priori what parameterization of the vector field will be good, and so using the neural ODE on the funnel model would be a good test-case for the behavior of the neural ODE + NEIS approach to pathological distributions that may arise in practice.
* In figures 2, 3, and 4 the NEIS-based estimators (especially Direct GD in figures 2 and 3, and the Linear (before model  and even Linear (after) model to some extent in Figure 4) show substantial bias.  This is at odds with the theoretical result that regardless of the vector field NEIS should be unbiased.  Is this due to the discretization of the ODE?  Or are there just rare events where a very large normalizing constant is estimated and the runs presented here got "unlucky" in having exclusively underestimated normalizing constants?  In any case, this disconnect between the theoretical result that all of these estimates should be unbiased and the observed substantial bias should be discussed / addressed.
limitations: In terms of limitations, please see the discussion above in "Strengths and Weaknesses" about issues surrounding discretization.  I agree with the authors that there are no obvious negative societal impacts of the work.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper has two main contributions. First, a theoretical analysis of Non-Equilibrium Importance Sampling (NEIS), a technique used to get unbiased estimates (in theory) of expectations and normalizing constants under intractable distributions, based on performing averages averages along the flow generated by the evolution of a dynamical system (given as a first order ODE). The main theoretical results proved in the paper involved showing the existence (under certain assumptions) of an optimal velocity field that yields an estimator with zero variance, and a connection with optimal transport maps. And second, the paper explains how NEIS with a learnable velocity field parameterized by a NN can be used in practice by training the method's parameters to minimize the estimator's variance, and shows an empirical evaluation on three low-dimensional models.
strengths_and_weaknesses: Strengths: The paper is written in a clear way, and presents original theoretical findings. The theoretical analysis answers relevant questions about NEIS, including conditions for the existence of the optimal dynamics that lead to an estimator with zero variance. All statements are clearly stated including all assumptions, and most assumptions are adequately discussed in the main text.

Weaknesses: In my opinion, the bigger weaknesses involve the empirical evaluation, the claims made around it, and the method's practicality. For instance:

- The conclusion claims that the empirical evaluation shows the method's effectiveness in high dimensions, when the evaluation is done on three models of dimensionalities 2, 10 and 10.

- The method appears to be extremely slow in practice, even for the simple 2d model, but this is not mentioned in the main text. It takes 1.5 hours to train for 200 SGD steps on the two dimensional model (mixture of two Gaussians) and 5 hours on one of the ten dimensional model (mixture of 4 Gaussians). I understand that the method uses neural networks, and simulations were done using CPUs, but all NNs are fairly small, of around 2 layers with 10 neurons each, so I don't think that should be a big bottleneck even when using CPUs. (Are training times on the funnel model, for which NNs are not used, reported?).

- The training uses minibatches of thousands of samples per training step, which may indicate that there are some issues with the variance of the objective and its gradients. This affects the cost of the method. Is it necessary to use such large minibatches?

- The vanilla estimator is taken to be importance sampling with a standard Gaussian as proposal, which is expected to work very poorly, specially since it is an extremely bad approximation of the targets used. I think that replacing this vanilla baseline with a diagonal Gaussian trained with VI, for instance, would provide a more interesting comparison.

Overall, I feel the empirical evaluation does not really show the practicality of the method, somewhat contradicting the claims made in the paper (if any of the statements above is wrong, please let me know). I think that more training details should be included in the main text explicitly (training times, sizes of the networks, steps used to numerically evaluate integrals and simulate dynamics, etc), as they are extremely relevant and currently only present in the Appendix. Also, I think that a more explicit discussion regarding the method's cost should be included, specially since the objective is somewhat complex, involving two nested integrals whose integrands depend on the values taken by a dynamical system at different times.
questions: - Have you tried using smaller batch-sizes?

- If I understand correctly, theoretical results (prop 3.1, prop 4.1) involve the formulation for equation 10, which integrates time from $-\infty$ to $\infty$. Do the results hold for the case where finite integration limits are used (i.e. eq. 8), which is the one used in practice?
limitations: See above regarding limitations.
review_title: null

Meta-review: metareview: This paper studies the properties of a non-equilibium importance sampling method for unnormalized densities. The reviewers unanimously agreed the paper is well-written and the contribution is novel. Several reviewers expressed concerns on the practicality of the algorithm and that the empirical results do not adequately show the benefits of the method. Overall, the reviews felt that the rebuttal did not adequately address their concerns. 

After discussion, the reviewers agreed that the paper is a worthwhile contribution and unanimously recommended acceptance. The final revision of the paper should address the limitations of the method more explicitly and tone down the claims about the empirical contribution and the method’s practicality. Another reviewer expressed that the paper needs to address the issue that in practice, numerical integration is required to solve ODEs and can lead to longer running times. Please revise the paper carefully based on the reviewers' feedback.

confidence: Certain
Decision: Accept