Paper: A Primal-Dual Approach to Solving Variational Inequalities with General Constraints (ICLR-2024)

Reviewer comment (Theory):
The assumption that F(x) is monotone or strictly monotone may be overly stringent. This condition is not met in applications involving Generative Adversarial Networks.

Author response:
Monotone variational inequalities (VIs) are crucial in the VI literature and are considered a relatively broad problem class, surpassing min-max problems, for example. Their significance mirrors that of convex optimization problems in minimization literature, where optimization methods are initially validated on convex problems due to the complexity of studying non-convex problems.
Hence, it is essential and customary to initially examine the convergence properties of new algorithms on monotone VI problems, as evidenced by existing works (see [1-4], for instance). Extending our analysis to a broader class is left for future research.  Additionally, our experiments with GANs (Figure 3) suggest that ACVI performs effectively even in non-monotone problems.

Reviewer comment (Reproducibility):
The actual efficiency of the algorithm heavily depends on the solution approach for the x-subproblem, which is challenging to solve. The authors do not address the resolution of this general nonlinear system.

Author response:
In terms of our theoretical contributions, our convergence rate does take into account the complexity of solving the $x$-subproblem. In particular, if either EG or GD are used for the $x$-subproblem, the convergence is $O(1/\sqrt{K})$. It's worth noting that if the original problem is monotone, the $x$ problem becomes strongly monotone, making it easier to solve; you can find more details in our discussion on page 51.
Additionally, if general-form constraints are associated with the problem, there is no other alternative as a first-order method, as we elaborate in the introduction. 

From a practical standpoint, solving the $x$-subproblem approximately using unconstrained GDA or ED methods is straightforward—this is reflected in our experiments, which include wall-clock time and gradient query comparisons. Setting the iteration number for solving $x$-subproblem (i.e., $l_x^{(t)}$ in line 8 of Alg. 1)  to be a small constant yields good results. Refer to Figures 2-4, 11, and App. E for more details.

Please let us know if further clarification on the theoretical or practical aspects is needed.

Reviewer comment (Theory):
It is preferred to provide an explanation for assuming the "monotonicity" of $F$ on $\mathcal{C}_{=}$ and the "strictly monotonicity" of $F$ on $\mathcal{C}$.

Author response:
The monotonicity of $F$ on $C_=$ guarantees the existence and uniqueness of the solution of the non-linear system for the $x$-subproblem. More details can be found in Theorem B.3 in our paper or Theorem 1 in Yang et al., 2023. 

We require either strict monotonicity of $F$ or strict convexity of one of $\varphi_i$. This condition is necessary to ensure the "central path" exists—refer to Appendix B.3 for a more detailed discussion. It's worth noting that Remark 3 in Yang et al. explains that this assumption is relatively weak.

Please let us know if more clarifications are needed.

Reviewer comment (Writing):
The barrier map is undefined when it first appears in Algorithm 1.

Author response:
We've postponed the definition of the barrier maps to the convergence subsection to ensure clarity that the theorem specifically applies to those barrier maps (which are clickable in the algorithm). Our revised paper also refers to their equations when introducing Algorithm 1 in Section 3.  Thanks for bringing this to our attention.

Reviewer comment (Novelty):
The main issue I had in reading the paper was the comparison with Yang et al. I tried to follow the long proofs in that paper and to compare them with the present proofs. Many of the steps were similar, but I was still left wondering where the main improvements took place.

Author response:
We managed to circumvent the assumptions in Yang et al. by a simple yet not immediately apparent insight into understanding the point implicitly tracked by ACVI at each iteration. Indeed, (14) is one of the crucial steps -- we added more technical discussion below Lemma C.10 (paragraph *discussion*, page 28) in our revised paper,  where we describe our new analysis technique and compare them with the approach of Yang et al., 2023. Please let us know if additional clarification could be helpful. Thanks.

Reviewer comment (Writing):
A small typo: in the first two displays of page 23, the norms in the RHS are missing their squares (the mistake is fixed in the next display).

Author response:
Thank you for your attentive review of the proofs and for providing valuable feedback – it is greatly appreciated. We've corrected the mentioned typo in our revised paper; thanks for catching that.

Reviewer comment (Experiments):
Experiments are somewhat limited. The only examples are synthetic bilinear games, and GANs. It would be nice to also have an example motivated by computational game theory, or by multi-agent reinforcement learning, or applications to economics, where equilibrium-finding algorithms that are a special case of the variational inequality framework are important.

Author response:
Thanks; integrating those experiments could indeed help highlight the value of our contributions. However, due to space limitations, we opt for the standard experiments commonly found in related works. To complement our numerical simulations, we include experiments on GANs—a less conventional practice in theoretical papers.


It's worth noting that the bilinear games we employ have been extensively explored in computational game theory, reinforcement learning, and economics, as indicated by references [1-5]. The high-dimensional bilinear games, coupled with the probability simplex constraints (problem (HBG) in Section 5), are reminiscent of two-player zero-sum games [5]. In each turn, players select actions from predetermined probability distributions, akin to defining their policies. The effectiveness of our methods in addressing this particular problem subtly implies that they may perform well in the mentioned areas as well.

Reviewer comment (Presentation):
Figure 1 needs a legend, because right now it is confusing. I understand the red square is the constraint set and the yellow star is the Nash from the caption: what's the difference between the yellow and blue line? Which one is the initial point? It would save my time and improve clarity to have this labeled in full on the actual plot rather than partially explained in a manner that is buried inside the caption which has a lot more details that I may or may not be ready to delve into. The vector fields are also way too small, I can't tell on my display which direction any of the arrows are pointing.

Author response:
We've edited Fig. 1 in the uploaded revised version; thanks for raising that. The yellow curve is the central path, or in other words, the trajectory that the second-order interior point methods would follow. The dark blue trajectory is that of the corresponding method (in the subfigure's caption), where all methods are initialized at the same point depicted with a dark blue circle.

Reviewer comment (Presentation):
Font size in figures is a bit small, making it bigger would help see what is happening.

Author response:
Thanks for pointing that out; we will improve these for the final version.

Reviewer comment (Writing):
Notation is not always completely consistent, for instance min and argmin are sometimes italicized, sometimes not.

Author response:
We've corrected that; thanks.

Reviewer comment (Writing):
Can you comment a bit further on the role of the barrier function in ACVI, and in the theory of Section 3.2? The barrier function is already a part of ACVI, and therefore not a new component of inexact ACVI - is this correct? If so, why talk about it in detail before Theorem 3.2, rather than in the background section? Does Theorem 3.2 specifically require one of these two barrier functions, and would not work with others? If so, this should be stated more explicitly - right now it's buried in the definition of Algorithm 1, and one has to unroll that definition - including the line above saying "Choosing among these is denoted with $\wp$" - to realize this, which isn't great because it puts a key part of the logic outside of both the text which defines Theorem 3.2 and the text that defines Algorithm 1.

Author response:
In short, $\wp_1(\cdot)$ is the same as in Yang et al., and $\wp_2(\cdot)$ is new for I-ACVI.


In Yang et al., it is assumed that the $y$ subproblem can be solved exactly analytically.  In contrast, in I-ACVI, where the solution is obtained iteratively through gradient-based methods, we introduced $\wp_2$ to make the objective nicer. Specifically, $\wp_1(\cdot)$ is non-smooth, and while technically functional, there are no efficient methods to solve it.  We design $\wp_2(\cdot)$ to make the objective smooth;  this barrier term could potentially be helpful also for constrained minimization.  Please see App. C.4 for a more detailed discussion.


Yes. Theorem 3.2 requires one of these two barrier functions and would not work with others; in addition to Alg. 1, we noted it in Theorem 3.2, too.

Reviewer comment (Theory):
The last iterate convergence is characterized by the gap function and the difference of two primal iterates. This metric is different from the optimality gap used in Yang et al. (2023). I am not sure if they are comparable.

Author response:
We use the standard gap function as in Yang et al. (2023); see our def. 2.2, and def. 2 in Yang et al., as well as item 3 in Thm.2 and Thm.3 of Yang et al.(2023).

Reviewer comment (Theory):
It is less discussed why removing the Lipschitz assumption is important in theory and practice.

Author response:
Practical scenarios often entail operators lacking Lipschitz continuity, such as in GANs and multi-agent reinforcement learning. For example, if using relu (or a lot of other activations), the successive multiplication of coefficient matrices leads to a derivative that is not Lipschitz continuous. Relaxing this assumption broadens the scope for which the convergence guarantee holds. It's worth noting that a substantial body of work in both VI and minimization literature focuses on eliminating the Lipschitzness assumption; references [1-5], for instance.

Reviewer comment (Theory):
The method is limited to monotone operators.

Author response:
Monotone variational inequalities (VIs) occupy a pivotal position in the VI literature and are considered a relatively broad problem class, surpassing min-max problems, for example. Their significance is comparable to that of convex optimization problems in the realm of minimization literature, where, due to the complexity of non-convex problems, optimization methods are initially validated on convex problems.  
Hence, it is essential and customary to initially examine the convergence properties of new algorithms on monotone VI problems, as evidenced by existing works (see [6-9], for instance). Extending our analysis to a broader class is left for future research.  

Our experiments with GANs (Figure 3) indicate that ACVI performs effectively even in non-monotone problems.

Reviewer comment (Novelty):
Since this paper focuses on improving existing analysis, can the authors more formally state new analysis techniques compared with Yang et al. (2023)?

Author response:
Please refer to our earlier answer to W2.  
In addition to Thm. 3.1, which improves the mentioned analysis, we introduce two algorithms and study their convergence. This is in response to the limitation of the alg. in Yang et al. (2023) in handling the general case when the subproblem cannot be analytically solved and efficiently managing structured inequality constraints.

Reviewer comment (Experiments):
What is the computational complexity of inexact ACVI? How can this be compared with the one for ACVI in your experiments?

Author response:
To give an $\varepsilon$-accurate solution, the overall complexity of inexact ACVI is still $\mathcal{O}(1/\varepsilon^2)$ up to a log factor, as we discussed at the end of App. C.4. In our experiments, the complexity is also $\mathcal{O}(1/\varepsilon^2)$ since we fix the number of iterations to solve the subproblems to be a small constant number, which only adds constant to the complexity.

Reviewer comment (Experiments):
Any examples that lack Lipschitzness? and experiments?

Author response:
For example, the operators of Von Newmann's ratio game and Frasaken game study in Yang et al., 2023 (see Figure 2 and App. D therein) are not Lipschitz continuous.  
Operators lacking Lipschitzness are ubiquitous when using neural nets, such as in GANs and multi-agent reinforcement learning (albeit non-monotone); please see Figure 3 and Section 5.3 for our experiments on GANs.