Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The authors extend gradient flow methods for variational inference to handle inequality constraints. These inequality constraints can capture fairness, safety, interpretability, and other such desiderata. They introduce two methods to solve this. First, they introduce a 'primal-dual gradient method': they formulate the equivalent minimax problem to the constrained optimization, and then they derive the dynamics for gradient descent ascent of the zero-sum game. Convergence is shown by explicitly providing a Lyapunov function. Next, they introduce a 'constraint controlled gradient descent', which first finds the feasible set, and then stays within the feasible set afterward (inspired by controlled barrier functions (CBFs) from control). Convergence is shown similarly to the work in CBFs.
main_review: Originality: This work introduces the problem of variational inference subject to inequality constraints. The tasks are new. Whereas the methods are not new, they are combined in a novel fashion to solve a new, well-motivated problem.

Quality: The methods used are appropriate for the problem at hand, and the theoretical claims are (mostly) technically sound. However, one alarming issue is the fact that Assumption 3.7 is missing from the draft (it abruptly ends before the assumption is given). This MUST be fixed. However, the methods used are relatively standard and the results are intuitive; again, the main contribution of this paper is the new problem formulation.

Clarity: The paper is well-written; it provides a good overview of the methods, both formally and intuitively. It provides intuition for how the results are derived, as well as full rigorous proofs in the appendix. 

Significance: The problem introduced is definitely of significance to the community and society at large. 

Major issues: 
Assumption 3.7 is incomplete, ending with "...we have". This MUST be addressed.

Minor notes (did not affect evaluation):
- Typo in the title: 'Trusthworthy'
- Line 52: 'contorl'
- Equation 3: $\hat y(\theta; \theta)$.
- Line 86: $r_q$ should be bolded for consistency.
- Line 108: I recommend the authors describe the median trick in greater detail.
- Line 180: 'qudratic'
- Line 224: 'Gaussian mixture toy' -> 'Gaussian mixture toy model'
- Line 225: 'thrustworthy'
- Line 285: 'manifold' -> 'manifolds'

limitations_and_societal_impact: The work provides algorithms to include constraints in variational inference, and provides theoretical guarantees. However, as stated by the authors, the theoretical analysis primarily considers the mean-field limit. Additionally, the theorems are for continuous-time convergence, and the conditions for the discretizations to follow the continuous-time flow are not provided. Nevertheless, this paper provides a significant contribution in its own right, and these limitations are more good directions for future work rather than a shortcoming of the current paper.

Additionally, it's unclear how practical these algorithms would be in real-world settings, as the simulations provided are primarily small-scale. However, a deeper evaluation of this would be a paper in its own right, and is outside the scope of this paper, whose primary goal is theoretical development.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper presents SVGD variants that can handle constraints on the optimized distribution in the form of an expectation under it. The methods are based on Lagrangian multiplier (primal-dual form) and constraint controlled optimization, respectively. Their convergence analyses in the context of a constraint are developed. Experiment verifies a good fit to the target distribution with the satisfaction of the constraint.
main_review: Under a critical view, the considered task seems not to require fancy techniques. The primal-dual form transforms the target distribution to satisfy the constraint, and the constraint control version enforces the constraint to hold in every optimization step (similar to projection in each step). So I did not feel lighted on its originality. Nevertheless, the paper has done this in some fine details, including convergence analyses that take the constraint into consideration, and a good introduction and application results on tasks that require such a constraint.

One technical question: The authors mentioned that the conditions of CCGF is milder than the primal-dual version. Does that mean that CCGF indeed has a better convergence guarantee in more scenarios, or is it just a proving technique issue?

Although the organization is logical and intuitive explanations of math are provided, the writing of the paper can be further improved. To name a few:
  - Line 2, "to handling". Line 21, "which frame". Three lines above Line 106, "to be an". Line 112, missing "(". Line 144, "holds". Line 153, "$E$" but not "\mathbb{E}". Line 166, "direction". Second line in Page 6, "in addition". Line 174, "allows us to have". Line 202, "defined in (17), we". Line 225, "trustworthy".
  - Line 130: "The empirical distribution of ... is used to approximate ... in (19)": I suppose the samples are $\{\theta_i\}$ but not $\{\xi_i\}$, and Eq. (9) but not (19).
  - In Eq. (10), what is $\eta$? A step size parameter is not expected in such a continuous-time equation.
  - Eq. (14): $D(q_t, p^*_{\lambda_t})$ but not $D(q_t \Vert p^*_{\lambda_t})$.
limitations_and_societal_impact: Addressed.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes to modify existing approximation posterior inference algorithms to satisfy an additional moment constraint that might be related to trustworthiness, fairness, or something else. Two algorithms are proposed, which can be used with either a discretized Langevin algorithm or Stein variational gradient descent. Convergence properties of the underlying gradient flows are also characterized. 
main_review: Originality: Overall the work seems quite original and creative. The moment constraint is introduced by extending the usual KL minimizing gradient flow problem as a minimax problem. This leads to a rather elegant “primal-dual” gradient flow formulation, which is discretized to obtain the first proposed algorithm. The second method is obtained as a trust-region–type method using a linearization of the constraint. In addition to proposing two algorithms, convergence results for the (continuous-time, infinite-particle limit) of both algorithms are provided. 

Quality: The paper appears to be technically sound. However, the results concerning the SVGD flow are misleading. This is because most of the theory requires a log Sobolev inequality to hold. However, it appears that an LSI essentially never holds for SVGD [DNS19]. Thus, the results as they apply to SVGD are essentially vacuous. Moreover, we should not expect SVGD to behave well for multimodal distributions, since the KSD does a poor job detecting the relative mass of different mods. Given all this, it is unsurprising that the Langevin algorithms are empirically superior. But the superiority of the Langevin algorithms is never clearly acknowledged. If anything, the author(s) seem favorably disposed to the SVGD approach, despite its clear theoretical and empirical inferiority. 

Clarity: While the motivation of the paper is clear, the theoretical results are extremely dense and hard to digest – particularly for a reader who isn’t already familiar with the relevant background material on gradient flows. In light of the better performance of the Langevin-based algorithms, to improve readability I would suggest moving all of the technical SVGD content and the general framework to the appendix and focus on the more digestible Langevin case. 

Significance: The results and approaches developed seem to be potentially quite useful and could open up a number of interesting new avenues for future work. Thus, I’m inclined to accept the paper. Though I do strongly encourage the author(s) to revise the paper as suggested above. I think making the paper easier to read will also make it more impactful as it will reach a broader audience. 


[DNS19] Duncan, A., Nuesken, N. & Szpruch, L. On the geometry of Stein variational gradient descent. arXiv.org, arXiv:1912.00894 [stat.ML], (2019).
limitations_and_societal_impact: These are appropriately addressed
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose two ways to extend Unadjusted Langevin Algorithm (ULA) and Stein Variational Gradient Descent (SVGD) for sampling from a distribution known up to its normalization constant, under moment constraints. The first one is a primal-dual gradient descent scheme,
while the second one is a controlled gradient descent one.

They provide convergence guarantees in continuous time for both schemes: a 1/T rate for a given optimality criterion Delta in the more generic case (Th 3.4 and 3.8), and linear convergence rate in KL if some form of log-concavity is satisfied for the current target p_{lamba_t}* (Th 3.5 and 3.9).

These methods perform reasonably well on synthetic and real world experiments, with a better performance for controlled-based methods.

main_review: The paper tackles the interesting problem of sampling under moment constraints. I am quite convinced about the importance of this problem, as motivated in the introduction or the experiments conducted (e.g. bayesian logistic regression with logic rules, or training fair bayesian neural networks).

They proposed two extensions of ULA and SVGD which are widely used in Bayesian inference, based on schemes from the optimization literature.
These ideas are natural but are novel up in sampling to the best of my knowledge. Also, the authors did a good job in providing both theoretical guarantees and empirical evidence for the relevance of these schemes for sampling.

My main concern is regarding the theoretical guarantees.
Regardig the first scheme, assumption 3.1 is problematic, as it has to be satisfied at all times for the current target distribution p_lambda_t *.
For instance, eq 14 corresponds to assuming the log-Sobolev inequality for p_lambda_t* when F_t=L2(q_t)/in the Langevin setting (satisfied for instance when p_lambda_t* is c_2 strongly log concave) as explained by the authors, or to the Stein log Sobolev inequality when F_t= RKHS /in the SVGD setting, which is hardly satisfied (see Lemma 36 in “on the geometry of Stein variational gradient descent” , Duncan, Nusken, 2019 - does not hold for a bounded kernel such as gaussian or IMQ if p_lambda_t* has exponential tails).

But maybe it is common in the euclidean optimization framework to require such assumptions on the current iterates to prove convergence for primal-dual gradient methods. Can the authors point to such references?I have a similar concern for the second scheme, and the requirement that the norm of s_{q_t, g} is non zero or lower bounded by c_ . 

Still, Theorem 3.5  and 3.9 are mainly valid for the Langevin setting, but are not likely to be satisfied for SVGD -> this should be written clearly.

Overall, I tend towards a weak accept.


Minor comments:

Section 2: 

When F_t is a RKHS, D is known to metrize convergence between q_t and p_0* only under specific conditions on p_0* and the reproducing kernel, see “Measuring sample quality” Gorham and Mackey, 2017. In particular it may fail for a gaussian kernel, which is widely used in practice. This is worth mentioning in the paper.

l101: whose time-discretisation -> one of the time discretisation leads to Unadjusted Langevin Monte Carlo algorithm. Many other time-discretizations of this flow exist, see  “Sampling as optimisation over the space of measures”, A. Wibisono, Colt 2018; or “Wasserstein Proximal Gradient” A. Salim, A. Korba, G. Luise, Neurips 2020.

Section 3:

eq 10: what is eta?
eq 11 and 12: p*-> po*?
l189 : grad log q -> grad loq po*
l196: same with p

Assumption 3.7 is not written entirely…
Figure 1: I would have been curious about the non-constrained baseline (ie classical langevin or svgd) to have a better idea of the attained loss and E_q[g(\theta)]

Section 4 :

l256 : i have troubles understanding how the distribution D_logic is created. For the points x that do not satisfy the logic constraint, it looks like this constraint penalizes the original KL objective as well; but maybe i’m getting it wrong.

What is the fairness loss and disparate impact? Why don’t you plot the (average) prediction loss of the BNN and the constraint violation as in the first experiments?

limitations_and_societal_impact: Limitations : cf comments, e.g. regarding theoretical guarantees and SVGD.
review_title: null

Meta-review: The reviewers agreed that this paper should be accepted. For the camera ready: aside from generally going through the reviews/responses and ensuring that all suggested changes are made, the reviewers had one more remark for the authors. In particular, while the reviewers appreciated the effort to compare Langevin/SVGD, they still strongly suggest that the authors suitably nuance the limitations of the SVGD theory. One idea brought forward by a reviewer that the others thought was acceptable is to state results for Langevin and defer SVGD to the appendix with careful statements of limitations. This highlights that they can be obtained in a very similar manner, but that they are less likely to hold in practice.

Decision: Accept (Poster)