Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: I have the rebuttal of the authors, the paper improved indeed and some point on role of M is better clarified now although it is still a bit convoluted. The paper would be stronger if the analysis shows any theoretical advantage to the presented method. I think the author put a good effort in addressing some of my concerns and I m raising my score to 6. 

 
####
Summary of the paper:

The paper proposes stein self repulsive dynamics for sampling from an unnormalized distribution. The method starts by using Langevin dynamic for up to time $Mc$ and then uses those pasts samples to guide the trajectories of the langevin sampling to explore new areas of the densities  using the stein witness function between the current particles  and the past samples( similar to Stein Variational gradient descent). 

The paper analyses the mixing properties under standard assumptions of the potential of the Boltzmann distribution and the kernel used in the Stein discrepancy, and shows convergence to the boltzman distribution as the number of particles goes to infinity and the step size goes to zero. 

Authors validate their methods and show that it indeed explores on a synthetic example new areas wrt to pure langevin dynamics. Applications in sampling from the posterior of bayesian neural networks and contextual bandits compare the performance of the proposed method to langevin dynamic and pure stein descent favorably. 

Clarity/presentation :

The paper is well written and the intuition are well presented . 

The notation $\hat{delta}_{M} $ is not great in denoting direct measures, please using another symbol. 

My main concerns with the paper are the following:

- the definition of $\bar{\delta}_{M}$ averages only $M$ particles choosen for $M$ time stamps. Something is off here for the continuous approximation to work at each time stamp you need $N$ particles and then you have a past horizon $M$. As $M\to \infty$ this does not matter, but I think in your implementation you are considering at each time step $N$ particles and you average on a horizon of size $M$. is this correct? Please clarify?

- The theorem show only asymptotic behavior and don't quantify the intuition behind the paper , that the "coverage of the samples" is higher.
 Can you for instance bound the wasserstein  distance between the pure langevin and Stein Repulsive dynamic , and between SVGD and your method, as it was done in "The promises and pitfalls of Stochastic Gradient Langevin Dynamics". Basically you can find a coupling between trajectories of your methods and the langevin dynamics and bound the wasserstein distance between the two methods. This will be insightful to see if one would mix faster with respect to the other one.  

While the appendices of the paper are lengthy I don't think they explain the most selling point of the method, since they are asymptotic and there is a confusion between the time horizon for the past , and the number of particles.  if $M$ is  $\infty$ in this defintion , we are just running langevin dynamic, and not using Stein witness function. I see an important issue in this definition of the time horizon, I think this time horizon should be finite, and that from time steps less we sample N particles , and we let this $N$ go to infinity.

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper proposed another variant of Langevin dynamics, called “Stein self-repulsive dynamics,” which simultaneously decreases the auto-correlation of Langevin dynamics and eliminates the need for running parallel chains in SVGD. They combined Langevin dynamics with Stein variational gradient descent and theoretically justified that the proposed method successfully converges to the stationary distribution with only a single chain, unlike SVGD. The proposed method decreases the auto-correlation of Langevin dynamics, so the proposed method increases the sample efficiency.

The paper is well-written. The idea of the proposed method is natural, which is incorporating the functionality of SVGD to reduce the auto-correlation of Langevin dynamics. The idea is intuitive and justified by their theoretical analysis. The authors also well- placed their work in the literature, as described in Section 3. The intuitive explanation of the proposed method is given in Section 3.

I have one technical question as follows. If the authors reply appropriately, I will raise the score to accept.

In Theorem 4.3, the result holds for any k and M. The authors claim that if we take a limit of M -> ∞ with fixed k, the practical dynamics converges to the discrete-time mean-field limit, in Section 4. However, to state the result of Theorem 4.3, k should be bigger than M c_\eta from the dentition of \tilde{\rho}_k^M, as shown under the equation (4). How do we take a limit of M -> ∞ ? Does k also go ∞?

Minor comments:
- The definition of g should depend on only \theta_k^I and \hat{\delta}_k^M, not \theta_k^k.
- The equation (1) should hold for any \theta’, not \theta.
- The equation (1) should contain \rho, not p.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: The papers described how to use the repulsive term used with standard SVGD within MCMC/SGLD. Briefly, the paper proposes to use a (damped version of) the SVG repulsive term between the current position of a SGLD trajectory and the empirical distribution defined by the trajectory.

The approach is interesting and natural. Unfortunately, I do not think that the experiments are convincing. 

(1) Since the authors are advertising the Bayesian framework, the choices of metrics such as test RMSE or test LL are not adapted
(2) in the UCI dataset examples, it is indeed extremely difficult to explain the enhanced performances? Is it because of multimodality? Better exploration of a mode? Comparison to a single NN? Comparison with an ensemble of NN? 
(3) I would have been much more convinced by a set of well-chosen and controlled experiments. The 2-dimensional examples are far too low-dimensional to be convincing. Higher-dimensional Gaussian? Higher-dimensional mixtures? Non-linear tractable problems in higher dimensions? Influence of the tuning parameters (RBF parameter? alpha? step-size in SGLD)? Computational issues? Subsampling-effect? etc...

The proposed method is interesting and has a lot of potential. I would like to suggest the authors to spend more times on careful and controlled numerical experiments (Bayesian NN are not very good for this purpose) -- with convincing numerics (which would give more reasons to delve into the proofs) the method can be very promising.

review_title: Official Blind Review #1

Meta-review: This paper proposes a new sampling mechanism which uses a self-repulsive term to increase the diversity of the samples.

The reviewers had concerns, most of which were addressed in the rebuttal. Unfortunately, none of the reviewers genuinely championed the paper. Since there were a lot of good submissions this year, we had to make decisions on the borderline papers and this lack of full support means that this submission will be rejected.

I highly encourage you to keep updating the manuscript and to rebusmit it to a later conference.
Decision: Reject